localisation on Xml

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

localisation on Xml

bubblebenj
This post was updated on .
Hi,

I'd like to use an Xml <http://haxe.org/api/xml> file to store my
localisation.
I also intend to import it as a
resource<http://haxe.org/doc/advanced/resources>and use
haxe.xml.Fast to access datas.
Also these datas need to be accessed by neko, probably flash (but I'm not
sure I still need it for flash).

I was wondering if that could considerably slow down the server due to :

   - Xml parsing as highlighted in these discussion<http://haxe.1354130.n2.nabble.com/Xml-is-too-slow-td4930126.html> (maybe the slowness has
   been corrected)
   - xml.Fast. I don't know if it's supposed to be executed faster or if it
   just th way to write code that is faster.
   - embedding the whole xml as a resource could result in a high memory
   usage (plus xml.Fast instance)

I'd like your opinion on that.

Ho I forget, I've choosen xml format rather than other because :

   - I know it
   - I can edit it in Excel (one line per localisation id and one column for
   each language) and save it (no code required).
   - It's crossplateform in haXe.

Their may be other good choices, I'll be glad to hear about.

Ben

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
Sorry I forget a link :
  - Xml parsing as highlighted in these discussion<http://haxe.1354130.n2.nabble.com/Xml-is-too-slow-td4930126.html> (maybe the slowness has
   been corrected)

Ben
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Baluta Cristian
Wow, but how many words do you have to localize? I'm using haxe.xml.Proxy to do that, and i think you can use only with an xml.

On Fri, Aug 5, 2011 at 10:51 AM, bubblebenj <[hidden email]> wrote:
Sorry I forget a link :
 - Xml parsing as highlighted in these
discussion&lt;http://haxe.1354130.n2.nabble.com/Xml-is-too-slow-td4930126.html&gt;
(maybe the slowness has
  been corrected)

Ben

--
View this message in context: http://haxe.1354130.n2.nabble.com/localisation-on-Xml-tp6655601p6655620.html
Sent from the Haxe mailing list archive at Nabble.com.

--
haXe - an open source web programming language
http://haxe.org



--
Băluță Cristian
http://ralcr.com
http://imagin.ro

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
I actually don't have that much words to localize (I believe less than 300 items (words or sentences), to be large)
And at the moment I'm targeting only one language but If I can do something I can reuse later that worth looking into possible problems.
That said, I won't spend to much time on that.

I'll look into the haxe.xml.Proxy to see if it fits better to my needs.

Ben

On Fri, Aug 5, 2011 at 9:57 AM, Baluta Cristian <[hidden email]> wrote:
Wow, but how many words do you have to localize? I'm using haxe.xml.Proxy to do that, and i think you can use only with an xml.


On Fri, Aug 5, 2011 at 10:51 AM, bubblebenj <[hidden email]> wrote:
Sorry I forget a link :
 - Xml parsing as highlighted in these
discussion&lt;http://haxe.1354130.n2.nabble.com/Xml-is-too-slow-td4930126.html&gt;
(maybe the slowness has
  been corrected)

Ben

--
View this message in context: http://haxe.1354130.n2.nabble.com/localisation-on-Xml-tp6655601p6655620.html
Sent from the Haxe mailing list archive at Nabble.com.

--
haXe - an open source web programming language
http://haxe.org



--
Băluță Cristian
http://ralcr.com
http://imagin.ro

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Yanis Benson
In reply to this post by bubblebenj

It's rarely needed to use more than one language at once. So you usually do one language per file to speed up things. And the simpliest possible format would be just one name and string per line. If you want to use Excel(or whatever else) I'd recommend to write it in the format you want and then just to convert it to this simple format.

Also, I plan to write fast template engine. Yaht like with all the cashing and possible speedups out of the box. I had an experience of writing such engine for PHP in the past and it was beating everything around speed-wise and quite comfortable to work with. The conception of working with translations was to cache templates with language strings inlined. However, when you just need to do a translation and don't want to make a whole engine it's a bit of overkill I guess.

On 5 Aug 2011 13:42, "benjamin Dubois" <[hidden email]> wrote:

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj


On Fri, Aug 5, 2011 at 12:34 PM, Yanis Benson <[hidden email]> wrote:

It's rarely needed to use more than one language at once. So you usually do one language per file to speed up things.

Yep, that's what I'll do if the file becomes too big.
I've also be adviced to split localisation by theme (error messages, menus...) sections files to keep them even smaller.

And the simpliest possible format would be just one name and string per line. If you want to use Excel(or whatever else) I'd recommend to write it in the format you want and then just to convert it to this simple format.

I understand but I really like the pipeline involving the less possible code. I'll use that minimalist format for a time I need performance (I hope not that soon) 

Also, I plan to write fast template engine. Yaht like with all the cashing and possible speedups out of the box. I had an experience of writing such engine for PHP in the past and it was beating everything around speed-wise and quite comfortable to work with. The conception of working with translations was to cache templates with language strings inlined. However, when you just need to do a translation and don't want to make a whole engine it's a bit of overkill I guess.

That makes me wonder if is possible to create pre-localised (for non dynamic data) template files with templo. But I don't need it at the moment.

Ben

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
I'm testing the xml.Proxy class which seems to be what I need.
However I'm stuck with a little problem, see :

  • my xml file :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<text id="1">
<en>blop</en>
<fr>blopfr</fr>
</text>
<text id="2">
<en>blop2</en>
<fr>blop2fr</fr>
</text>
</data>

  • my implementation :
typedef Item = {
en : String,
fr : String
}

class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml", Item> {
}

class CLocals {
    public static var all = new Hash<Item>();
    public static var text = new CLocalsProxy(all.get);
public function test() {
text.resolve( "1" ).en; // both compilation & completion works
text.1; // both compilation & completion works
text.1.en; // neither compilation nor completion works (missing ";" after 1)
}
}


So finally I can't access my datas (in Items.fr or .en), there must be something I missed.

Any idea ?

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

George McBay
Have you considered using macros?

If you're going to include the localizations as data in haxe resources
anyway, why not include them as code data that has nearly zero runtime
cost?

I don't actually do this with localization strings currently but the
gui/reactive programming framework I created for AS2/AS3 apps running
on our low-CPU chumby targets is all XML based with the XML parsed at
compile time using macros and converted into haxe object creation code
so we get the nice human readable XML for layout design but completely
avoid the cost of xml parsing at runtime.




On Fri, Aug 5, 2011 at 11:14 AM, benjamin Dubois <[hidden email]> wrote:

> I'm testing the xml.Proxy class which seems to be what I need.
> However I'm stuck with a little problem, see :
>
> my xml file :
>
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
> <text id="1">
> <en>blop</en>
> <fr>blopfr</fr>
> </text>
> <text id="2">
> <en>blop2</en>
> <fr>blop2fr</fr>
> </text>
> </data>
>
> my implementation :
>
> typedef Item = {
> en : String,
> fr : String
> }
> class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml", Item> {
> }
> class CLocals {
>     public static var all = new Hash<Item>();
>     public static var text = new CLocalsProxy(all.get);
> public function test() {
> text.resolve( "1" ).en; // both compilation & completion works
> text.1; // both compilation & completion works
> text.1.en; // neither compilation nor completion works (missing ";" after 1)
> }
> }
>
> So finally I can't access my datas (in Items.fr or .en), there must be
> something I missed.
> Any idea ?
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Yanis Benson
In reply to this post by bubblebenj

Try using [a-zA-Z][0-9a-zA-Z]* as id.

On 5 Aug 2011 22:16, "benjamin Dubois" <[hidden email]> wrote:
> I'm testing the xml.Proxy class which seems to be what I need.
> However I'm stuck with a little problem, see :
>
>
> - my xml file :
>
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
> <text id="1">
> <en>blop</en>
> <fr>blopfr</fr>
> </text>
> <text id="2">
> <en>blop2</en>
> <fr>blop2fr</fr>
> </text>

> </data>
>
>
> - my implementation :
>
> typedef Item = {
> en : String,
> fr : String
> }
>
> class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml", Item> {
> }
>
> class CLocals {
> public static var all = new Hash<Item>();
> public static var text = new CLocalsProxy(all.get);
> public function test() {
> text.resolve( "1" ).en; // both compilation & completion works
> text.1; // both compilation & completion works
> text.1.en; // neither compilation nor completion works (missing ";" after
> 1)
> }
> }
>
>
> So finally I can't access my datas (in Items.fr or .en), there must be
> something I missed.
>
> Any idea ?

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Baluta Cristian
i noticed that you can't use uppercases in ids, you don't get any error but you don't get any result either, so maybe there's a problem with numbers too.

On Fri, Aug 5, 2011 at 9:44 PM, Yanis Benson <[hidden email]> wrote:

Try using [a-zA-Z][0-9a-zA-Z]* as id.

On 5 Aug 2011 22:16, "benjamin Dubois" <[hidden email]> wrote:
> I'm testing the xml.Proxy class which seems to be what I need.
> However I'm stuck with a little problem, see :
>
>
> - my xml file :
>
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
> <text id="1">
> <en>blop</en>
> <fr>blopfr</fr>
> </text>
> <text id="2">
> <en>blop2</en>
> <fr>blop2fr</fr>
> </text>

> </data>
>
>
> - my implementation :
>
> typedef Item = {
> en : String,
> fr : String
> }
>
> class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml", Item> {
> }
>
> class CLocals {
> public static var all = new Hash<Item>();
> public static var text = new CLocalsProxy(all.get);
> public function test() {
> text.resolve( "1" ).en; // both compilation & completion works
> text.1; // both compilation & completion works
> text.1.en; // neither compilation nor completion works (missing ";" after
> 1)
> }
> }
>
>
> So finally I can't access my datas (in Items.fr or .en), there must be
> something I missed.
>
> Any idea ?

--
haXe - an open source web programming language
http://haxe.org



--
Băluță Cristian
http://ralcr.com
http://imagin.ro

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Yanis Benson

I never looked into this topic for haXe specifically, but identifiers aren't usually allowed to start from digits at all, so they never collide with numbers. Actually I'm very surprised that object.1 worked. Maybe it should even be considered a bug. We should look into formal syntax definition for haXe (I never examined such document actually.) or just ask Nicolas.

On 5 Aug 2011 22:52, "Baluta Cristian" <[hidden email]> wrote:
> i noticed that you can't use uppercases in ids, you don't get any error but
> you don't get any result either, so maybe there's a problem with numbers
> too.
>
> On Fri, Aug 5, 2011 at 9:44 PM, Yanis Benson <[hidden email]> wrote:
>
>> Try using [a-zA-Z][0-9a-zA-Z]* as id.
>> On 5 Aug 2011 22:16, "benjamin Dubois" <[hidden email]> wrote:
>> > I'm testing the xml.Proxy class which seems to be what I need.
>> > However I'm stuck with a little problem, see :
>> >
>> >
>> > - my xml file :
>> >
>> > <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> > <data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
>> > <text id="1">
>> > <en>blop</en>
>> > <fr>blopfr</fr>
>> > </text>
>> > <text id="2">
>> > <en>blop2</en>
>> > <fr>blop2fr</fr>
>> > </text>
>> > </data>
>> >
>> >
>> > - my implementation :
>> >
>> > typedef Item = {
>> > en : String,
>> > fr : String
>> > }
>> >
>> > class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml",
>> Item> {
>> > }
>> >
>> > class CLocals {
>> > public static var all = new Hash<Item>();
>> > public static var text = new CLocalsProxy(all.get);
>> > public function test() {
>> > text.resolve( "1" ).en; // both compilation & completion works
>> > text.1; // both compilation & completion works
>> > text.1.en; // neither compilation nor completion works (missing ";" after
>> > 1)
>> > }
>> > }
>> >
>> >
>> > So finally I can't access my datas (in Items.fr or .en), there must be
>> > something I missed.
>> >
>> > Any idea ?
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>>
>
>
>
> --
> Băluță Cristian
> http://ralcr.com
> http://imagin.ro

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
In reply to this post by Baluta Cristian
Yes the problem was the numeral ids.
So I made some test and it seems that the problem appears only when the idea begins with a number :
  • 7
  • 7est
When you think about it's normal that pure number doesn't work since that can't be variable names

I didn't have the problem mentioned with uppercase ids and ids with numbers "not at the beginning" (I get compilation completion working) :
  • Uppercase
  • UPPERCASE
  • uPpercase
  • tes7
Albeit compilation and completion work, the vars are not filled : my hash is empty ( {} ).
my code, closer to Nicolas's example :
  • localisation.xml :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<t id="aa">blop</t>
<t id="bb">blop2</t>
</data>
  • CLocals.hx :
class CLocalsProxy extends haxe.xml.Proxy<"assets/localisation.xml", String> { }

class CLocals {
    public static var all = new Hash<String>();
    public static var text = new CLocalsProxy(all.get);
}



--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
Ok so the problem was that I had to fill the hash myself (I found several posts about that, I feel a little ashamed).
It works well now.

I've produced an example for the community.

Thx for your help all

Ben

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

Baluta Cristian
Strange usage, i don't think apply in real life situations. The purpose of a localization is to change the texts inside an app without editing it, just pointing it to a new xml, and you can't do that. You should access data only with: CLocals.text.film01

Here is my implementation, like in Nicholas sample: https://ralcr.googlecode.com/svn/trunk/Foundation/RCLocalization.hx (haha, and i think i just found the problem with uppercases i was talking about :P , i don't remember why i've applied toLowercases)


On Sat, Aug 6, 2011 at 2:52 PM, benjamin Dubois <[hidden email]> wrote:
Ok so the problem was that I had to fill the hash myself (I found several posts about that, I feel a little ashamed).
It works well now.

I've produced an example for the community.

Thx for your help all

Ben

--
haXe - an open source web programming language
http://haxe.org



--
Băluță Cristian
http://ralcr.com
http://imagin.ro

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: localisation on Xml

bubblebenj
On Sat, Aug 6, 2011 at 2:20 PM, Baluta Cristian <[hidden email]> wrote:
Strange usage, i don't think apply in real life situations. The purpose of a localization is to change the texts inside an app without editing it, just pointing it to a new xml, and you can't do that. You should access data only with: CLocals.text.film01
Yep, I realized that when I read this old message from Nicolas :
"After that, you're free to use any XML file at runtime by filling the 
hashtable as you wish, but if you use another file than fr.xml, then the 
compile-time check will not check against this file." 
However I can still have a single XML file and make my hashtable filling function do the switch.
One reason for a single XML file is the capability to choose another (default) language if a word is missing from a language.

Ben

--
haXe - an open source web programming language
http://haxe.org