CPP / NME Charset

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

CPP / NME Charset

Julien CASTETS
Hi,

I'm trying to port my flash app to cpp with nme and I'm facing a charset encoding problem.

My server-side application (haxe/PHP) is retrieving some UTF-8 strings from a MySQL DB and sending it back to the client via haxe.remoting.

I've managed to compile a debug version of nme.ndll to better understand what is going wrong.
There is an assertion that fails in Textfield::Layout

Here is the stack trace

> nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  Ligne 56 + 0x2a octets C++
  nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
  nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})  Ligne 1367 + 0xc octets C++
  nme.ndll!nme::TextField::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForBitmap=false)  Ligne 1169 C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
  nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)  Ligne 1180 + 0x28 octets C++
  Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne 78 + 0xc octets C++
  Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj> _Data={...}, String _Theme={...})  Ligne 193 C++
  Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
  00000001()


I'm compiling NME with hxcpp from a visual studio command prompt

Is there some compilation flag I should set to avoid this ?
Or should I find a way to convert/sanitize my strings on the server-side application ?

Cheers
Thank you
Julien

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

Julien CASTETS
I tried to enable HX_UTF8_STRINGS in msvc-toolchain.xml and in the nme Build.xml file with no luck.


2011/3/2 Julien CASTETS <[hidden email]>
Hi,

I'm trying to port my flash app to cpp with nme and I'm facing a charset encoding problem.

My server-side application (haxe/PHP) is retrieving some UTF-8 strings from a MySQL DB and sending it back to the client via haxe.remoting.

I've managed to compile a debug version of nme.ndll to better understand what is going wrong.
There is an assertion that fails in Textfield::Layout

Here is the stack trace

> nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  Ligne 56 + 0x2a octets C++
  nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
  nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})  Ligne 1367 + 0xc octets C++
  nme.ndll!nme::TextField::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForBitmap=false)  Ligne 1169 C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
  nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)  Ligne 1180 + 0x28 octets C++
  Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne 78 + 0xc octets C++
  Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj> _Data={...}, String _Theme={...})  Ligne 193 C++
  Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
  00000001()


I'm compiling NME with hxcpp from a visual studio command prompt

Is there some compilation flag I should set to avoid this ?
Or should I find a way to convert/sanitize my strings on the server-side application ?

Cheers
Thank you
Julien


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

Julien CASTETS
Ok found it.

The problem was because if this character http://www.unicodemap.org/details/0x2019/index.html

Now I've got a HEAP CORRUPTION =s


Cheers
Julien

2011/3/2 Julien CASTETS <[hidden email]>
I tried to enable HX_UTF8_STRINGS in msvc-toolchain.xml and in the nme Build.xml file with no luck.


2011/3/2 Julien CASTETS <[hidden email]>

Hi,

I'm trying to port my flash app to cpp with nme and I'm facing a charset encoding problem.

My server-side application (haxe/PHP) is retrieving some UTF-8 strings from a MySQL DB and sending it back to the client via haxe.remoting.

I've managed to compile a debug version of nme.ndll to better understand what is going wrong.
There is an assertion that fails in Textfield::Layout

Here is the stack trace

> nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  Ligne 56 + 0x2a octets C++
  nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
  nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})  Ligne 1367 + 0xc octets C++
  nme.ndll!nme::TextField::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForBitmap=false)  Ligne 1169 C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform & inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)  Ligne 1088 + 0x1a octets C++
  nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
  nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)  Ligne 1180 + 0x28 octets C++
  Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne 78 + 0xc octets C++
  Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj> _Data={...}, String _Theme={...})  Ligne 193 C++
  Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
  00000001()


I'm compiling NME with hxcpp from a visual studio command prompt

Is there some compilation flag I should set to avoid this ?
Or should I find a way to convert/sanitize my strings on the server-side application ?

Cheers
Thank you
Julien



--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

Gamehaxe
Hi,
Just quickly looking, there seems to be a bug in:

Tile Font::GetGlyph(int inCharacter,int &outAdvance)

where it does not use "mExtendedGlyph" when setting a missing glyph.

I have added a change to common/Font.cpp that may do the trick (untested).

Hugh


> Ok found it.
>
> The problem was because if this character
> http://www.unicodemap.org/details/0x2019/index.html
>
> Now I've got a HEAP CORRUPTION =s
>
>
> Cheers
> Julien
>
> 2011/3/2 Julien CASTETS <[hidden email]>
>
>> I tried to enable HX_UTF8_STRINGS in msvc-toolchain.xml and in the nme
>> Build.xml file with no luck.
>>
>>
>> 2011/3/2 Julien CASTETS <[hidden email]>
>>
>> Hi,
>>>
>>> I'm trying to port my flash app to cpp with nme and I'm facing a  
>>> charset
>>> encoding problem.
>>>
>>> My server-side application (haxe/PHP) is retrieving some UTF-8 strings
>>> from a MySQL DB and sending it back to the client via haxe.remoting.
>>>
>>> I've managed to compile a debug version of nme.ndll to better  
>>> understand
>>> what is going wrong.
>>> There is an assertion that fails in Textfield::Layout
>>>
>>> Here is the stack trace
>>>
>>> > nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  Ligne  
>>> 56
>>> + 0x2a octets C++
>>>   nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
>>>   nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})
>>>  Ligne 1367 + 0xc octets C++
>>>   nme.ndll!nme::TextField::GetExtent(const nme::Transform &
>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool  
>>> inForBitmap=false)
>>>  Ligne 1169 C++
>>>   nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform  
>>> &
>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool  
>>> inForScreen=false)
>>>  Ligne 1088 + 0x1a octets C++
>>>   nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform  
>>> &
>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool  
>>> inForScreen=false)
>>>  Ligne 1088 + 0x1a octets C++
>>>   nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
>>>   nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)
>>>  Ligne 1180 + 0x28 octets C++
>>>   Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne 78  
>>> +
>>> 0xc octets C++
>>>   Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj>
>>> _Data={...}, String _Theme={...})  Ligne 193 C++
>>>   Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const
>>> Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
>>>   00000001()
>>>
>>>
>>> I'm compiling NME with hxcpp from a visual studio command prompt
>>>
>>> Is there some compilation flag I should set to avoid this ?
>>> Or should I find a way to convert/sanitize my strings on the  
>>> server-side
>>> application ?
>>>
>>> Cheers
>>> Thank you
>>> Julien
>>>
>>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

Julien CASTETS
Hi Hugh,

Thank you for answering.

The problem was in isalpha and isspace functions :
for each there is an ASSERTE that checks that the char is lower than 256 (mine was 0x2019)

I made a filter that sanitize the strings before storing them in the database. Text rendering is now fine. Then no need for extended glyphs atm, maybe I'll need some when I'll be in the localization process =)

The heap corruption was caused because of a code like this

for( o in list )
{
  if(comeCondition)
  {
    o.shut();
    list.remove(o);
    o = null;
  }
}

I've never encounter any problem with this kind of code on other platforms (removing an iterable while iterating)
I assume it's a bit dangerous, I'll try to not doing it anymore :)

I'll post for another problem targeting iphone simulator sdk 4.2 when I'll get more info on the reason why =)

Cheers
Many thanks
Julien

2011/3/2 Hugh Sanderson <[hidden email]>
Hi,
Just quickly looking, there seems to be a bug in:

Tile Font::GetGlyph(int inCharacter,int &outAdvance)

where it does not use "mExtendedGlyph" when setting a missing glyph.

I have added a change to common/Font.cpp that may do the trick (untested).

Hugh



Ok found it.

The problem was because if this character
http://www.unicodemap.org/details/0x2019/index.html

Now I've got a HEAP CORRUPTION =s


Cheers
Julien

2011/3/2 Julien CASTETS <[hidden email]>

I tried to enable HX_UTF8_STRINGS in msvc-toolchain.xml and in the nme
Build.xml file with no luck.


2011/3/2 Julien CASTETS <[hidden email]>

Hi,

I'm trying to port my flash app to cpp with nme and I'm facing a charset
encoding problem.

My server-side application (haxe/PHP) is retrieving some UTF-8 strings
from a MySQL DB and sending it back to the client via haxe.remoting.

I've managed to compile a debug version of nme.ndll to better understand
what is going wrong.
There is an assertion that fails in Textfield::Layout

Here is the stack trace

> nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  Ligne 56
+ 0x2a octets C++
 nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
 nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})
 Ligne 1367 + 0xc octets C++
 nme.ndll!nme::TextField::GetExtent(const nme::Transform &
inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForBitmap=false)
 Ligne 1169 C++
 nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform &
inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)
 Ligne 1088 + 0x1a octets C++
 nme.ndll!nme::DisplayObjectContainer::GetExtent(const nme::Transform &
inTrans={...}, nme::Extent2D<float> & outExt={...}, bool inForScreen=false)
 Ligne 1088 + 0x1a octets C++
 nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
 nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)
 Ligne 1180 + 0x28 octets C++
 Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne 78 +
0xc octets C++
 Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj>
_Data={...}, String _Theme={...})  Ligne 193 C++
 Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const
Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
 00000001()


I'm compiling NME with hxcpp from a visual studio command prompt

Is there some compilation flag I should set to avoid this ?
Or should I find a way to convert/sanitize my strings on the server-side
application ?

Cheers
Thank you
Julien



--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

Gamehaxe
Hi,
I guess the question is, "what is the best way to determine word breaks  
for codes>256".

A simple fix would be to break if code>128, so the isalpha never gets run.

I will look into this - but I wonder why it didn't reproduce when I tried  
it?
Maybe it is a debug switch, or maybe have to test with word-breaking on.

Hugh

> Hi Hugh,
>
> Thank you for answering.
>
> The problem was in isalpha and isspace functions :
> for each there is an ASSERTE that checks that the char is lower than 256
> (mine was 0x2019)
>
> I made a filter that sanitize the strings before storing them in the
> database. Text rendering is now fine. Then no need for extended glyphs  
> atm,
> maybe I'll need some when I'll be in the localization process =)
>
> The heap corruption was caused because of a code like this
>
> for( o in list )
> {
>   if(comeCondition)
>   {
>     o.shut();
>     list.remove(o);
>     o = null;
>   }
> }
>
> I've never encounter any problem with this kind of code on other  
> platforms
> (removing an iterable while iterating)
> I assume it's a bit dangerous, I'll try to not doing it anymore :)
>
> I'll post for another problem targeting iphone simulator sdk 4.2 when  
> I'll
> get more info on the reason why =)
>
> Cheers
> Many thanks
> Julien
>
> 2011/3/2 Hugh Sanderson <[hidden email]>
>
>> Hi,
>> Just quickly looking, there seems to be a bug in:
>>
>> Tile Font::GetGlyph(int inCharacter,int &outAdvance)
>>
>> where it does not use "mExtendedGlyph" when setting a missing glyph.
>>
>> I have added a change to common/Font.cpp that may do the trick  
>> (untested).
>>
>> Hugh
>>
>>
>>
>>  Ok found it.
>>>
>>> The problem was because if this character
>>> http://www.unicodemap.org/details/0x2019/index.html
>>>
>>> Now I've got a HEAP CORRUPTION =s
>>>
>>>
>>> Cheers
>>> Julien
>>>
>>> 2011/3/2 Julien CASTETS <[hidden email]>
>>>
>>>  I tried to enable HX_UTF8_STRINGS in msvc-toolchain.xml and in the nme
>>>> Build.xml file with no luck.
>>>>
>>>>
>>>> 2011/3/2 Julien CASTETS <[hidden email]>
>>>>
>>>> Hi,
>>>>
>>>>>
>>>>> I'm trying to port my flash app to cpp with nme and I'm facing a  
>>>>> charset
>>>>> encoding problem.
>>>>>
>>>>> My server-side application (haxe/PHP) is retrieving some UTF-8  
>>>>> strings
>>>>> from a MySQL DB and sending it back to the client via haxe.remoting.
>>>>>
>>>>> I've managed to compile a debug version of nme.ndll to better  
>>>>> understand
>>>>> what is going wrong.
>>>>> There is an assertion that fails in Textfield::Layout
>>>>>
>>>>> Here is the stack trace
>>>>>
>>>>> > nme.ndll!_chvalidator(int c=0x00009c61, int mask=0x00000103)  
>>>>> Ligne 56
>>>>> + 0x2a octets C++
>>>>>  nme.ndll!isalpha(int c=0x00009c61)  Ligne 69 + 0xe octets C++
>>>>>  nme.ndll!nme::TextField::Layout(const nme::Matrix & inMatrix={...})
>>>>>  Ligne 1367 + 0xc octets C++
>>>>>  nme.ndll!nme::TextField::GetExtent(const nme::Transform &
>>>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool
>>>>> inForBitmap=false)
>>>>>  Ligne 1169 C++
>>>>>  nme.ndll!nme::DisplayObjectContainer::GetExtent(const  
>>>>> nme::Transform &
>>>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool
>>>>> inForScreen=false)
>>>>>  Ligne 1088 + 0x1a octets C++
>>>>>  nme.ndll!nme::DisplayObjectContainer::GetExtent(const  
>>>>> nme::Transform &
>>>>> inTrans={...}, nme::Extent2D<float> & outExt={...}, bool
>>>>> inForScreen=false)
>>>>>  Ligne 1088 + 0x1a octets C++
>>>>>  nme.ndll!nme::DisplayObject::getHeight()  Ligne 467 C++
>>>>>  nme.ndll!nme_display_object_get_height(_value * inObj=0x008937fc)
>>>>>  Ligne 1180 + 0x28 octets C++
>>>>>  Main.exe!ExternalPrimitive::__run(const Dynamic & a={...})  Ligne  
>>>>> 78 +
>>>>> 0xc octets C++
>>>>>  Main.exe!QuizWnd_obj::__new(hx::ObjectPtr<data::QuizData_obj>
>>>>> _Data={...}, String _Theme={...})  Ligne 193 C++
>>>>>
>>>>>  Main.exe!`haxe::remoting::HttpAsyncConnection_obj::call'::`3'::_Function_1_1::__run(const
>>>>> Dynamic & inArg0={...})  Ligne 134 + 0x1d octets C++
>>>>>  00000001()
>>>>>
>>>>>
>>>>> I'm compiling NME with hxcpp from a visual studio command prompt
>>>>>
>>>>> Is there some compilation flag I should set to avoid this ?
>>>>> Or should I find a way to convert/sanitize my strings on the  
>>>>> server-side
>>>>> application ?
>>>>>
>>>>> Cheers
>>>>> Thank you
>>>>> Julien
>>>>>
>>>>>
>>>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: CPP / NME Charset

leona313
This post has NOT been accepted by the mailing list yet.
In reply to this post by Julien CASTETS