hxcpp command-line application

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

hxcpp command-line application

Cauê W.
Guys,

I've just had an application that was taking several days to process 10GB of data on neko. I've had squeezed every performance I could get with it, but the calculations were very heavy, with some complex algorithms.

I've decided then to take the time to run it over hxcpp. Guys, IT WAS WORTH IT. : )

But this port wasn't completely painless:

  • I had to change all sqlite code and make it run directly in memory. I had already done most part of it because SPOD doesn't support multi-threading, and mutexes every time a db call is made isn't exactly performant, so I only used the db to load all values into memory
  • I've ran into some problems when compiling with the HXCPP_MULTI_THREADED flag, there were some other reports of it on the forum, and I got away by using a post-April 3rd svn version of the GCInternal.cpp file. The weird thing is that the latest version wouldn't compile as well
  • try ...catch doesn't seem to work very well, maybe not at all... I couldn't catch any errors, even though I really knew where they were.
  • fortunately, on windows it will open the visual studio debugger, which shows the offending source code, which was very helpful.
  • there seems to be some problems with empty strings. A Std.parseFloat (Int?) call with an empty string would return an exception, as would also file.writeString()
  • debugging on errors inside a CFFI call is just impossible. It was a trial and error processes
  • some clashes with cpp's keywords, like namespace.
  • on neko, I did some traces on multiple threads and accessed the same file on multiple threads for writing. It worked very well, but it crashed on cpp. Maybe we should make a distinction on the api between synchronized api and not-synchronized. Maybe a compiler flag?

The result? What was taking several days to run, is running in less than AN HOUR!!!!!!!!!!!!!!!!!!!!!!

so.. I owe a big, huge thanks to Huge! :)

I'm really amazed to see it working... It really paid off!

Thanks again!
Cheers!

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: hxcpp command-line application

David Bergman
Please create a blog post about this, as I am sure it would catch a few Google search words and could drive people to haXe in general.

Typed on an iPhone

On Aug 11, 2010, at 5:26 PM, Cauê a <[hidden email]> wrote:

Guys,

I've just had an application that was taking several days to process 10GB of data on neko. I've had squeezed every performance I could get with it, but the calculations were very heavy, with some complex algorithms.

I've decided then to take the time to run it over hxcpp. Guys, IT WAS WORTH IT. : )

But this port wasn't completely painless:

  • I had to change all sqlite code and make it run directly in memory. I had already done most part of it because SPOD doesn't support multi-threading, and mutexes every time a db call is made isn't exactly performant, so I only used the db to load all values into memory
  • I've ran into some problems when compiling with the HXCPP_MULTI_THREADED flag, there were some other reports of it on the forum, and I got away by using a post-April 3rd svn version of the GCInternal.cpp file. The weird thing is that the latest version wouldn't compile as well
  • try ...catch doesn't seem to work very well, maybe not at all... I couldn't catch any errors, even though I really knew where they were.
  • fortunately, on windows it will open the visual studio debugger, which shows the offending source code, which was very helpful.
  • there seems to be some problems with empty strings. A Std.parseFloat (Int?) call with an empty string would return an exception, as would also file.writeString()
  • debugging on errors inside a CFFI call is just impossible. It was a trial and error processes
  • some clashes with cpp's keywords, like namespace.
  • on neko, I did some traces on multiple threads and accessed the same file on multiple threads for writing. It worked very well, but it crashed on cpp. Maybe we should make a distinction on the api between synchronized api and not-synchronized. Maybe a compiler flag?

The result? What was taking several days to run, is running in less than AN HOUR!!!!!!!!!!!!!!!!!!!!!!

so.. I owe a big, huge thanks to Huge! :)

I'm really amazed to see it working... It really paid off!

Thanks again!
Cheers!
--
haXe - an open source web programming language
http://haxe.org

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: hxcpp command-line application

Heinz Hölzer-2
In reply to this post by Cauê W.
I just posted a issue for keyword clashes.

you should create Testcases when possible and post them on http://code.google.com/p/hxcpp/issues/list
hugh does a fantastic job, but he really needs simple examples to isolate the problems.

best,
h

Am 11.08.2010 23:26, schrieb Cauê Waneck:
Guys,

I've just had an application that was taking several days to process 10GB of data on neko. I've had squeezed every performance I could get with it, but the calculations were very heavy, with some complex algorithms.

I've decided then to take the time to run it over hxcpp. Guys, IT WAS WORTH IT. : )

But this port wasn't completely painless:

  • I had to change all sqlite code and make it run directly in memory. I had already done most part of it because SPOD doesn't support multi-threading, and mutexes every time a db call is made isn't exactly performant, so I only used the db to load all values into memory
  • I've ran into some problems when compiling with the HXCPP_MULTI_THREADED flag, there were some other reports of it on the forum, and I got away by using a post-April 3rd svn version of the GCInternal.cpp file. The weird thing is that the latest version wouldn't compile as well
  • try ...catch doesn't seem to work very well, maybe not at all... I couldn't catch any errors, even though I really knew where they were.
  • fortunately, on windows it will open the visual studio debugger, which shows the offending source code, which was very helpful.
  • there seems to be some problems with empty strings. A Std.parseFloat (Int?) call with an empty string would return an exception, as would also file.writeString()
  • debugging on errors inside a CFFI call is just impossible. It was a trial and error processes
  • some clashes with cpp's keywords, like namespace.
  • on neko, I did some traces on multiple threads and accessed the same file on multiple threads for writing. It worked very well, but it crashed on cpp. Maybe we should make a distinction on the api between synchronized api and not-synchronized. Maybe a compiler flag?

The result? What was taking several days to run, is running in less than AN HOUR!!!!!!!!!!!!!!!!!!!!!!

so.. I owe a big, huge thanks to Huge! :)

I'm really amazed to see it working... It really paid off!

Thanks again!
Cheers!


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: hxcpp command-line application

Cauê W.
You're right! I haven't done that yet since I'm not using the latest svn version, I don't really know what has been already fixed.
But will do ; )

2010/8/11 Heinz Hölzer <[hidden email]>
I just posted a issue for keyword clashes.

you should create Testcases when possible and post them on http://code.google.com/p/hxcpp/issues/list
hugh does a fantastic job, but he really needs simple examples to isolate the problems.

best,
h

Am 11.08.2010 23:26, schrieb Cauê Waneck:
Guys,

I've just had an application that was taking several days to process 10GB of data on neko. I've had squeezed every performance I could get with it, but the calculations were very heavy, with some complex algorithms.

I've decided then to take the time to run it over hxcpp. Guys, IT WAS WORTH IT. : )

But this port wasn't completely painless:

  • I had to change all sqlite code and make it run directly in memory. I had already done most part of it because SPOD doesn't support multi-threading, and mutexes every time a db call is made isn't exactly performant, so I only used the db to load all values into memory
  • I've ran into some problems when compiling with the HXCPP_MULTI_THREADED flag, there were some other reports of it on the forum, and I got away by using a post-April 3rd svn version of the GCInternal.cpp file. The weird thing is that the latest version wouldn't compile as well
  • try ...catch doesn't seem to work very well, maybe not at all... I couldn't catch any errors, even though I really knew where they were.
  • fortunately, on windows it will open the visual studio debugger, which shows the offending source code, which was very helpful.
  • there seems to be some problems with empty strings. A Std.parseFloat (Int?) call with an empty string would return an exception, as would also file.writeString()
  • debugging on errors inside a CFFI call is just impossible. It was a trial and error processes
  • some clashes with cpp's keywords, like namespace.
  • on neko, I did some traces on multiple threads and accessed the same file on multiple threads for writing. It worked very well, but it crashed on cpp. Maybe we should make a distinction on the api between synchronized api and not-synchronized. Maybe a compiler flag?

The result? What was taking several days to run, is running in less than AN HOUR!!!!!!!!!!!!!!!!!!!!!!

so.. I owe a big, huge thanks to Huge! :)

I'm really amazed to see it working... It really paid off!

Thanks again!
Cheers!


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org