Coroutines in haXe

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Coroutines in haXe

Cauê W.
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

blackdog-2
+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

bd 

On Wed, Jun 30, 2010 at 11:42 PM, Cauê Waneck <[hidden email]> wrote:
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

Mark de Bruijn | Dykam
I actually built a cross platform microthreading API. It performs really well, even for small tasks. The only downside is that you have to break up the task yourself and have to keep note of state, or use some coroutine builder. but you need that anyway. I actually think I am able to make this a bit more nice to use by building an coroutine emulation API.
--
Mark


On Thu, Jul 1, 2010 at 6:05 AM, blackdog <[hidden email]> wrote:
+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

bd 

On Wed, Jun 30, 2010 at 11:42 PM, Cauê Waneck <[hidden email]> wrote:
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--

haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

Cauê W.
In reply to this post by blackdog-2
hey blackdog!

I haven't looked very much into node.js internals. Does it implement concurrency on top of system threads? The cool thing about coroutines is that it can achieve a far better server-side robustness by not using system threads all the time (i.e. only when needed). This is how erlang works, and this is how google go also works.
But I agree an asynchronous io api would be very great! I was toying with the asynchronous .net api's, and I realised that it wouldn't still be very trivial to do, since to guarantee that an api call end up in the same thread where the call was made would implicate in some kind of global lock (wouldn't it? I don't really know for sure).


Mark,
That's very cool!

Have you open-sourced it? I think the custom break definition would still exist, but normally on a listen() function call (if we are thinking about programming with channels)
A compiler-based support for this wouldn't be too difficult, would it?

cheers
cauê

2010/7/1 blackdog <[hidden email]>
+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

bd 

On Wed, Jun 30, 2010 at 11:42 PM, Cauê Waneck <[hidden email]> wrote:
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

David Bergman
In reply to this post by Cauê W.
Coroutines are not the same as "green threads." The latter does not even *have* to be non-preemptive, although green thread schedulers *tend* to want specific types of instructions yielding (pun intended? maybe...) into the scheduler, not to put too much overhead on the runtime (such as the abstract machine's execution.)

Also, what Erlang has is LWPs, that are "green" in that they are scheduled (and allocated to the proper CPU in multi-CPU/core cases) by the Erlang runtime (inside of that sacred 'beam' container.)

So, what do you want: coroutines or "threads scheduled by the language runtime, instead of by the underlying (operating) system"?

I don't mean to be a grumpy old theorist, yet I am :-)

/David

On Jun 30, 2010, at 11:42 PM, Cauê Waneck wrote:

Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê
--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

blackdog-2
In reply to this post by Cauê W.
node.js is in the nature of twisted and tornado, it's single threaded. When you have long running calculations you can exec child process and read/write to them getting your replies via async calls. Simple and works well (not to mention the kernel can schedule your child process on different cpus as it needs). For example, bdog-gtk2 project opens a gtk-server as a child process and then can read and write command strings to it to provide a gtk ui - that's all async and does not block the node process.

I am now (after watching Rich Hickey's clojure videos) of the opinion that threads are, for the most part, a serious distraction and pitfall for programmers, analogous to manual v gc'd memory handling, i.e. remove them and you remove all sorts of errors. Thus single threaded systems are the future :) Less is more!

bd

On Thu, Jul 1, 2010 at 8:10 AM, Cauê Waneck <[hidden email]> wrote:
hey blackdog!

I haven't looked very much into node.js internals. Does it implement concurrency on top of system threads? The cool thing about coroutines is that it can achieve a far better server-side robustness by not using system threads all the time (i.e. only when needed). This is how erlang works, and this is how google go also works.
But I agree an asynchronous io api would be very great! I was toying with the asynchronous .net api's, and I realised that it wouldn't still be very trivial to do, since to guarantee that an api call end up in the same thread where the call was made would implicate in some kind of global lock (wouldn't it? I don't really know for sure).


Mark,
That's very cool!

Have you open-sourced it? I think the custom break definition would still exist, but normally on a listen() function call (if we are thinking about programming with channels)
A compiler-based support for this wouldn't be too difficult, would it?

cheers
cauê

2010/7/1 blackdog <[hidden email]>

+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

bd 

On Wed, Jun 30, 2010 at 11:42 PM, Cauê Waneck <[hidden email]> wrote:
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

Mark de Bruijn | Dykam
@Cauê, oops, forgot to add the link: http://gitorious.org/more/more/trees/master/haxe/more/threading

A bit about the default scheduler. With this one you give each thread a timeshare, giving two threads the same amount of timeshare results in them being allowed to use the same amount of time. Each task is a function which get's called untill he's done, or untill he used all of his time. When he ran too long, the time is subtracted from the next run. It's a bit confusing to explain, but just try it.
--
Mark


On Thu, Jul 1, 2010 at 2:41 PM, blackdog <[hidden email]> wrote:
node.js is in the nature of twisted and tornado, it's single threaded. When you have long running calculations you can exec child process and read/write to them getting your replies via async calls. Simple and works well (not to mention the kernel can schedule your child process on different cpus as it needs). For example, bdog-gtk2 project opens a gtk-server as a child process and then can read and write command strings to it to provide a gtk ui - that's all async and does not block the node process.

I am now (after watching Rich Hickey's clojure videos) of the opinion that threads are, for the most part, a serious distraction and pitfall for programmers, analogous to manual v gc'd memory handling, i.e. remove them and you remove all sorts of errors. Thus single threaded systems are the future :) Less is more!

bd

On Thu, Jul 1, 2010 at 8:10 AM, Cauê Waneck <[hidden email]> wrote:
hey blackdog!

I haven't looked very much into node.js internals. Does it implement concurrency on top of system threads? The cool thing about coroutines is that it can achieve a far better server-side robustness by not using system threads all the time (i.e. only when needed). This is how erlang works, and this is how google go also works.
But I agree an asynchronous io api would be very great! I was toying with the asynchronous .net api's, and I realised that it wouldn't still be very trivial to do, since to guarantee that an api call end up in the same thread where the call was made would implicate in some kind of global lock (wouldn't it? I don't really know for sure).


Mark,
That's very cool!

Have you open-sourced it? I think the custom break definition would still exist, but normally on a listen() function call (if we are thinking about programming with channels)
A compiler-based support for this wouldn't be too difficult, would it?

cheers
cauê

2010/7/1 blackdog <[hidden email]>

+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

bd 

On Wed, Jun 30, 2010 at 11:42 PM, Cauê Waneck <[hidden email]> wrote:
Is there any plan to support coroutines in haXe? After reading several articles about this, I strongly believe that supporting coroutines would make haXe an even stronger programming language/framework/tool.

That is because while threads are not implementable in all platforms, green threads only rely on how the compiler compiles the code. And while a thread is expensive to create and has well defined limits, a green thread is very lightweight and can give an astonishing improvement in performance, specially on server applications (as we can see by comparing erlang-powered yaws vs apache : http://www.sics.se/~joe/apachevsyaws.html ).

This possibility can give a new meaning to neko, for example, if it supports well a coroutine model. Implementing an erlang-like way to deal with hot-swapping of code can make neko -the- vm for working with highly concurrent applications.
Of course, we would need to build up an entirely asynchronous api for this to work well, but I really, really think it would pay off.

What do you guys think?

Cheers
Cauê

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

Nicolas Cannasse
In reply to this post by blackdog-2
blackdog a écrit :
> +1 async api
>
> A Nicholas blessed async io api would be good for my haxe-node.js
> project ( which will probably provide you with astonishing server side
> performance right now :)

If someone want to contribute a C based async IO API that works on all
platforms supported by Neko (Windows, Linux, OSX), I'ld be happy to
integrate it into the official distribution ;)

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

blackdog-2

Pegging the interface to a neko implementation will probably result in nothing been done, as users of neko don't have an incentive to create an implementation as there are existing methods and approaches to IO. Similarly, those of us who don't use neko have no incentive to create a neko implementation. 

Thus, an API interface is the best way to go here for the sake of innovation;  maintainers of various platforms can implement it as they require, and those of us experimenting with other platforms can experiment and target said API.

bd

On Sun, Jul 4, 2010 at 6:09 AM, Nicolas Cannasse <[hidden email]> wrote:
blackdog a écrit :

+1 async api

A Nicholas blessed async io api would be good for my haxe-node.js project ( which will probably provide you with astonishing server side performance right now :)

If someone want to contribute a C based async IO API that works on all platforms supported by Neko (Windows, Linux, OSX), I'ld be happy to integrate it into the official distribution ;)

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

laurence taylor
Would an arrows library suffice? It is asynchronous, and
crossplatform, and most of the various combinators are implemented
already. Doesn´t work on php in practise, but i´m sure thats a crincle
that could be ironed out. Just needs some rewriting in the scheduler
for clarity, and to figure out the choice arrow, and it´s complete.
Let me now if your interested.
laurence

On Mon, Jul 5, 2010 at 1:09 AM, blackdog <[hidden email]> wrote:

>
> Pegging the interface to a neko implementation will probably result in
> nothing been done, as users of neko don't have an incentive to create an
> implementation as there are existing methods and approaches to IO.
> Similarly, those of us who don't use neko have no incentive to create a neko
> implementation.
> Thus, an API interface is the best way to go here for the sake of
> innovation;  maintainers of various platforms can implement it as they
> require, and those of us experimenting with other platforms can experiment
> and target said API.
> bd
> On Sun, Jul 4, 2010 at 6:09 AM, Nicolas Cannasse <[hidden email]>
> wrote:
>>
>> blackdog a écrit :
>>>
>>> +1 async api
>>>
>>> A Nicholas blessed async io api would be good for my haxe-node.js project
>>> ( which will probably provide you with astonishing server side performance
>>> right now :)
>>
>> If someone want to contribute a C based async IO API that works on all
>> platforms supported by Neko (Windows, Linux, OSX), I'ld be happy to
>> integrate it into the official distribution ;)
>>
>> Best,
>> Nicolas
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines in haXe

Cauê W.
laurence, an arrows library would be great, I think for all haXe community!
I'd be surely interested. 
Why does it not work in php? Is it too slow?

Cheers
Cauê

2010/7/4 laurence taylor <[hidden email]>
Would an arrows library suffice? It is asynchronous, and
crossplatform, and most of the various combinators are implemented
already. Doesn´t work on php in practise, but i´m sure thats a crincle
that could be ironed out. Just needs some rewriting in the scheduler
for clarity, and to figure out the choice arrow, and it´s complete.
Let me now if your interested.
laurence

On Mon, Jul 5, 2010 at 1:09 AM, blackdog <[hidden email]> wrote:
>
> Pegging the interface to a neko implementation will probably result in
> nothing been done, as users of neko don't have an incentive to create an
> implementation as there are existing methods and approaches to IO.
> Similarly, those of us who don't use neko have no incentive to create a neko
> implementation.
> Thus, an API interface is the best way to go here for the sake of
> innovation;  maintainers of various platforms can implement it as they
> require, and those of us experimenting with other platforms can experiment
> and target said API.
> bd
> On Sun, Jul 4, 2010 at 6:09 AM, Nicolas Cannasse <[hidden email]>
> wrote:
>>
>> blackdog a écrit :
>>>
>>> +1 async api
>>>
>>> A Nicholas blessed async io api would be good for my haxe-node.js project
>>> ( which will probably provide you with astonishing server side performance
>>> right now :)
>>
>> If someone want to contribute a C based async IO API that works on all
>> platforms supported by Neko (Windows, Linux, OSX), I'ld be happy to
>> integrate it into the official distribution ;)
>>
>> Best,
>> Nicolas
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org