Node.JS, async and scaling revisited

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Node.JS, async and scaling revisited

Cauê W.
Hey guys!

I might be a little OT here...

Anyway, I haven't been very happy with apache performance for some time now, and I've decided to look for better alternatives. My #1 need by now is an haxe-based application server that also serves static files, but I'm also dying to be able to use comet with haxe as well, and maybe expand it to websockets later so it could also be a general-purpose push server as well.

After studying the alternatives, most of the mainstream solutions didn't make me very enthusiastic. What is really missing, and it seems that really "the right way" to do it is how Node JS is done - a lightweight server built on top of epoll and asynchronous i/o. There is more information about the i/o strategies in here http://www.kegel.com/c10k.html (very recommended reading). It seems it's the best way to scale, and it makes sense, since the cpu will be mostly busy processing the message (your code), and not delivering it (or related tasks, like thread context switching).
When I was looking for solutions, I stumbled at G-WAN ( http://www.gwan.ch/ ). Their benchmarks are really impressive, and it has some principles I admire, like a small and simple code base, despite all its pseudo-political statements. It advertises that you code in c, and it intercepts blocking calls and turns it to epoll instead. Seems magical. It's a pity that it doesn't work, their support is terrible, and it isn't open-source.
Anyway, when I first saw it I started coding a tora backend for it. Just then to know that their "magical" transform of sockets into asynchronous wasn't working.

I remember some time ago someone posted a server that a guy on haxe's irc channel made. His name is sality. Now that's a great start, I think, it's based on epoll, zero-copy, has already a tora backend, it's great!
But after studying about scaling network applications nowadays, it really seems that tora would easily become the bottleneck on a comet-server context, because for a comet server to be effective we would need to support something like.. 1k clients to be connected simultaneously.
On tora this would mean we would have to have 1k threads. And then we're back again to the problem of context switching between 1k threads (not to mention memory footprint, etc).

So it seems that really the best solution would be to have a kind of a tora server that spawns the quantity of threads with the exact number of cores the system has, and works totally asynchronous from then on.

I've read what Nicolas has written about node.js, and it's realy node.js' major downside how it deals with starvation, even though it seems to me that in cases where node.js clients would starve, tora would only raise the limit of non-starving clients, at the cost of overall performance.

So... I'd really like to know about what do you guys think? What would be the best solution? Maybe it would be to have two separate solutions, one for an application server (where an epoll-based server + tora seems to be a clear winner), and other for a comet server (where a totally async api would really make a difference).
Anyway it seems to me that asynchronous i/o api's are having more and more importance. It would be great to leverage this on haxe, and maybe create an async std api, based on neko?

Thanks!
Caue

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Justin Donaldson-3
Hi Cauê,

Great thoughts and information, I was wondering how tora, etc. could fit as a high availability comet server. 

I really can't come up with a preference between tora+async, vs. a comet focused approach.  I think I would probably find a use for either one.

In any event, I am interested in the development of the async api, however it ends up being thought out. 

Thanks again,
-Justin



--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
In reply to this post by Cauê W.

I'm working on this http://cloudshift.cl.

It's a haxe stack for node.js. currently i'm documenting and looking at it from a higher level, tweaking and filling out apis, as I go. For example, I have http, but I haven't added https yet. I don't expect it to settle down for another couple of months but it's on the way.

Http http://cloudshift.cl/external/manual/http.html

Re Nicolas's node bashing, one of my components is seamless Workers which runs possibly heavy weight processing in a sub process with the haxe remote protocol.

http://cloudshift.cl/external/manual/workers.html

Re cores, have a look at the new node.js clustering api. I'll be incorporating that in my http module by end of week.

http://nodejs.org/docs/v0.6.1/api/cluster.html

There's also a push based message system which is haxe oriented in that you pass haxe objects around and a haxe object store to sqlite.

Code on https://github.com/cloudshift, I'll publish my messaging code today too since you've mentioned comet.

I hope Nicolas decides to support an async api so that Node and other async solutions can be supported in the haxe std lib, and that he includes the node.js api in the next release of haxe. These two additions would be awesome for haxe3 and would catch a lot of attention. I've been banging on about haxe/node.js for about 3yrs, I feel we're almost at a tipping point where folks wake up to the benefits.

However, the node.js haxelib is available and up to date so you don't need to wait to get the benefits of node.js now.

R

On 11/15/2011 11:27 PM, Cauê Waneck wrote:
Hey guys!

I might be a little OT here...

Anyway, I haven't been very happy with apache performance for some time now, and I've decided to look for better alternatives. My #1 need by now is an haxe-based application server that also serves static files, but I'm also dying to be able to use comet with haxe as well, and maybe expand it to websockets later so it could also be a general-purpose push server as well.

After studying the alternatives, most of the mainstream solutions didn't make me very enthusiastic. What is really missing, and it seems that really "the right way" to do it is how Node JS is done - a lightweight server built on top of epoll and asynchronous i/o. There is more information about the i/o strategies in here http://www.kegel.com/c10k.html (very recommended reading). It seems it's the best way to scale, and it makes sense, since the cpu will be mostly busy processing the message (your code), and not delivering it (or related tasks, like thread context switching).
When I was looking for solutions, I stumbled at G-WAN ( http://www.gwan.ch/ ). Their benchmarks are really impressive, and it has some principles I admire, like a small and simple code base, despite all its pseudo-political statements. It advertises that you code in c, and it intercepts blocking calls and turns it to epoll instead. Seems magical. It's a pity that it doesn't work, their support is terrible, and it isn't open-source.
Anyway, when I first saw it I started coding a tora backend for it. Just then to know that their "magical" transform of sockets into asynchronous wasn't working.

I remember some time ago someone posted a server that a guy on haxe's irc channel made. His name is sality. Now that's a great start, I think, it's based on epoll, zero-copy, has already a tora backend, it's great!
But after studying about scaling network applications nowadays, it really seems that tora would easily become the bottleneck on a comet-server context, because for a comet server to be effective we would need to support something like.. 1k clients to be connected simultaneously.
On tora this would mean we would have to have 1k threads. And then we're back again to the problem of context switching between 1k threads (not to mention memory footprint, etc).

So it seems that really the best solution would be to have a kind of a tora server that spawns the quantity of threads with the exact number of cores the system has, and works totally asynchronous from then on.

I've read what Nicolas has written about node.js, and it's realy node.js' major downside how it deals with starvation, even though it seems to me that in cases where node.js clients would starve, tora would only raise the limit of non-starving clients, at the cost of overall performance.

So... I'd really like to know about what do you guys think? What would be the best solution? Maybe it would be to have two separate solutions, one for an application server (where an epoll-based server + tora seems to be a clear winner), and other for a comet server (where a totally async api would really make a difference).
Anyway it seems to me that asynchronous i/o api's are having more and more importance. It would be great to leverage this on haxe, and maybe create an async std api, based on neko?

Thanks!
Caue




-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Tarwin Stroh-Spijer
Yeah, if you're running a large site you always need background processing. For the stuff I'm doing this is being done in Java ATM. I know you could probably do this with Neko but it would be hard to get people to trust it, so Node is a great way to go (good press).


Tarwin Stroh-Spijer
_______________________

Touch My Pixel
http://www.touchmypixel.com/
phone: +61 3 8060 5321
_______________________


On Wed, Nov 16, 2011 at 1:26 AM, blackdog <[hidden email]> wrote:

I'm working on this http://cloudshift.cl.

It's a haxe stack for node.js. currently i'm documenting and looking at it from a higher level, tweaking and filling out apis, as I go. For example, I have http, but I haven't added https yet. I don't expect it to settle down for another couple of months but it's on the way.

Http http://cloudshift.cl/external/manual/http.html

Re Nicolas's node bashing, one of my components is seamless Workers which runs possibly heavy weight processing in a sub process with the haxe remote protocol.

http://cloudshift.cl/external/manual/workers.html

Re cores, have a look at the new node.js clustering api. I'll be incorporating that in my http module by end of week.

http://nodejs.org/docs/v0.6.1/api/cluster.html

There's also a push based message system which is haxe oriented in that you pass haxe objects around and a haxe object store to sqlite.

Code on https://github.com/cloudshift, I'll publish my messaging code today too since you've mentioned comet.

I hope Nicolas decides to support an async api so that Node and other async solutions can be supported in the haxe std lib, and that he includes the node.js api in the next release of haxe. These two additions would be awesome for haxe3 and would catch a lot of attention. I've been banging on about haxe/node.js for about 3yrs, I feel we're almost at a tipping point where folks wake up to the benefits.

However, the node.js haxelib is available and up to date so you don't need to wait to get the benefits of node.js now.

R


On 11/15/2011 11:27 PM, Cauê Waneck wrote:
Hey guys!

I might be a little OT here...

Anyway, I haven't been very happy with apache performance for some time now, and I've decided to look for better alternatives. My #1 need by now is an haxe-based application server that also serves static files, but I'm also dying to be able to use comet with haxe as well, and maybe expand it to websockets later so it could also be a general-purpose push server as well.

After studying the alternatives, most of the mainstream solutions didn't make me very enthusiastic. What is really missing, and it seems that really "the right way" to do it is how Node JS is done - a lightweight server built on top of epoll and asynchronous i/o. There is more information about the i/o strategies in here http://www.kegel.com/c10k.html (very recommended reading). It seems it's the best way to scale, and it makes sense, since the cpu will be mostly busy processing the message (your code), and not delivering it (or related tasks, like thread context switching).
When I was looking for solutions, I stumbled at G-WAN ( http://www.gwan.ch/ ). Their benchmarks are really impressive, and it has some principles I admire, like a small and simple code base, despite all its pseudo-political statements. It advertises that you code in c, and it intercepts blocking calls and turns it to epoll instead. Seems magical. It's a pity that it doesn't work, their support is terrible, and it isn't open-source.
Anyway, when I first saw it I started coding a tora backend for it. Just then to know that their "magical" transform of sockets into asynchronous wasn't working.

I remember some time ago someone posted a server that a guy on haxe's irc channel made. His name is sality. Now that's a great start, I think, it's based on epoll, zero-copy, has already a tora backend, it's great!
But after studying about scaling network applications nowadays, it really seems that tora would easily become the bottleneck on a comet-server context, because for a comet server to be effective we would need to support something like.. 1k clients to be connected simultaneously.
On tora this would mean we would have to have 1k threads. And then we're back again to the problem of context switching between 1k threads (not to mention memory footprint, etc).

So it seems that really the best solution would be to have a kind of a tora server that spawns the quantity of threads with the exact number of cores the system has, and works totally asynchronous from then on.

I've read what Nicolas has written about node.js, and it's realy node.js' major downside how it deals with starvation, even though it seems to me that in cases where node.js clients would starve, tora would only raise the limit of non-starving clients, at the cost of overall performance.

So... I'd really like to know about what do you guys think? What would be the best solution? Maybe it would be to have two separate solutions, one for an application server (where an epoll-based server + tora seems to be a clear winner), and other for a comet server (where a totally async api would really make a difference).
Anyway it seems to me that asynchronous i/o api's are having more and more importance. It would be great to leverage this on haxe, and maybe create an async std api, based on neko?

Thanks!
Caue




-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
On 11/16/2011 06:54 AM, Tarwin Stroh-Spijer wrote:
Yeah, if you're running a large site you always need background processing. For the stuff I'm doing this is being done in Java ATM. I know you could probably do this with Neko but it would be hard to get people to trust it, so Node is a great way to go (good press).


Tarwin Stroh-Spijer
_______________________

Touch My Pixel
http://www.touchmypixel.com/
phone: +61 3 8060 5321
_______________________


On Wed, Nov 16, 2011 at 1:26 AM, blackdog <[hidden email]> wrote:

I'm working on this http://cloudshift.cl.

It's a haxe stack for node.js. currently i'm documenting and looking at it from a higher level, tweaking and filling out apis, as I go. For example, I have http, but I haven't added https yet. I don't expect it to settle down for another couple of months but it's on the way.

Http http://cloudshift.cl/external/manual/http.html

Re Nicolas's node bashing, one of my components is seamless Workers which runs possibly heavy weight processing in a sub process with the haxe remote protocol.

http://cloudshift.cl/external/manual/workers.html

Re cores, have a look at the new node.js clustering api. I'll be incorporating that in my http module by end of week.

http://nodejs.org/docs/v0.6.1/api/cluster.html

There's also a push based message system which is haxe oriented in that you pass haxe objects around and a haxe object store to sqlite.

Code on https://github.com/cloudshift, I'll publish my messaging code today too since you've mentioned comet.

I hope Nicolas decides to support an async api so that Node and other async solutions can be supported in the haxe std lib, and that he includes the node.js api in the next release of haxe. These two additions would be awesome for haxe3 and would catch a lot of attention. I've been banging on about haxe/node.js for about 3yrs, I feel we're almost at a tipping point where folks wake up to the benefits.

However, the node.js haxelib is available and up to date so you don't need to wait to get the benefits of node.js now.

R


On 11/15/2011 11:27 PM, Cauê Waneck wrote:
Hey guys!

I might be a little OT here...

Anyway, I haven't been very happy with apache performance for some time now, and I've decided to look for better alternatives. My #1 need by now is an haxe-based application server that also serves static files, but I'm also dying to be able to use comet with haxe as well, and maybe expand it to websockets later so it could also be a general-purpose push server as well.

After studying the alternatives, most of the mainstream solutions didn't make me very enthusiastic. What is really missing, and it seems that really "the right way" to do it is how Node JS is done - a lightweight server built on top of epoll and asynchronous i/o. There is more information about the i/o strategies in here http://www.kegel.com/c10k.html (very recommended reading). It seems it's the best way to scale, and it makes sense, since the cpu will be mostly busy processing the message (your code), and not delivering it (or related tasks, like thread context switching).
When I was looking for solutions, I stumbled at G-WAN ( http://www.gwan.ch/ ). Their benchmarks are really impressive, and it has some principles I admire, like a small and simple code base, despite all its pseudo-political statements. It advertises that you code in c, and it intercepts blocking calls and turns it to epoll instead. Seems magical. It's a pity that it doesn't work, their support is terrible, and it isn't open-source.
Anyway, when I first saw it I started coding a tora backend for it. Just then to know that their "magical" transform of sockets into asynchronous wasn't working.

I remember some time ago someone posted a server that a guy on haxe's irc channel made. His name is sality. Now that's a great start, I think, it's based on epoll, zero-copy, has already a tora backend, it's great!
But after studying about scaling network applications nowadays, it really seems that tora would easily become the bottleneck on a comet-server context, because for a comet server to be effective we would need to support something like.. 1k clients to be connected simultaneously.
On tora this would mean we would have to have 1k threads. And then we're back again to the problem of context switching between 1k threads (not to mention memory footprint, etc).

So it seems that really the best solution would be to have a kind of a tora server that spawns the quantity of threads with the exact number of cores the system has, and works totally asynchronous from then on.

I've read what Nicolas has written about node.js, and it's realy node.js' major downside how it deals with starvation, even though it seems to me that in cases where node.js clients would starve, tora would only raise the limit of non-starving clients, at the cost of overall performance.

So... I'd really like to know about what do you guys think? What would be the best solution? Maybe it would be to have two separate solutions, one for an application server (where an epoll-based server + tora seems to be a clear winner), and other for a comet server (where a totally async api would really make a difference).
Anyway it seems to me that asynchronous i/o api's are having more and more importance. It would be great to leverage this on haxe, and maybe create an async std api, based on neko?

Thanks!
Caue




-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org



Although my worker module uses node.js as a subprocess, you can easily fire off other backend processes and listen on the stdin/stdout from the main node event loop. So, you could have something similar to my worker module that does the same for java (some exe/neko sub process), that is, attaches to the stdin/stdout of java and controls it's processing.

-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Cauê W.
@Justin, if it interests you to see more detail on i/o strategies, there is a great benchmark and presentation here http://bulk.fefe.de/scalable-networking.pdf

@blackdog,
I haven't looked too much into your site right now, as I'm at work. I'll do it when I get home. It seems great, specially the messaging part!

But anyway, I don't mean to say that Nicolas was bashing node.js, quite the contrary, really. I think he made a very valid criticism on it, which really makes sense. It doesn't matter that node.js is faster and in the long run it can handle the requests more quickly than other alternatives, if it has to completely finish a request to take another - leading really to a scenario of starvation. That's why it seems to me that for normal http requests, tora is a better option still, as you'd have e.g. 50 tora threads that can take requests simultaneously. On a steady connection (e.g. comet) scenario, though, tora wouldn't be as effective, it seems.

Maybe there is a better solution? What would be really great would be to be able to pause the execution of the neko interpreter, so we can still code like we would with blocking calls ,and maybe have a hybrid solution developed there. But I don't think this is very easy (or doable at all) - so maybe a better language support to asynchronous calls would be a great feature!

2011/11/16 blackdog <[hidden email]>
On 11/16/2011 06:54 AM, Tarwin Stroh-Spijer wrote:
Yeah, if you're running a large site you always need background processing. For the stuff I'm doing this is being done in Java ATM. I know you could probably do this with Neko but it would be hard to get people to trust it, so Node is a great way to go (good press).


Tarwin Stroh-Spijer
_______________________

Touch My Pixel
http://www.touchmypixel.com/
phone: <a href="tel:%2B61%203%208060%205321" value="+61380605321" target="_blank">+61 3 8060 5321
_______________________


On Wed, Nov 16, 2011 at 1:26 AM, blackdog <[hidden email]> wrote:

I'm working on this http://cloudshift.cl.

It's a haxe stack for node.js. currently i'm documenting and looking at it from a higher level, tweaking and filling out apis, as I go. For example, I have http, but I haven't added https yet. I don't expect it to settle down for another couple of months but it's on the way.

Http http://cloudshift.cl/external/manual/http.html

Re Nicolas's node bashing, one of my components is seamless Workers which runs possibly heavy weight processing in a sub process with the haxe remote protocol.

http://cloudshift.cl/external/manual/workers.html

Re cores, have a look at the new node.js clustering api. I'll be incorporating that in my http module by end of week.

http://nodejs.org/docs/v0.6.1/api/cluster.html

There's also a push based message system which is haxe oriented in that you pass haxe objects around and a haxe object store to sqlite.

Code on https://github.com/cloudshift, I'll publish my messaging code today too since you've mentioned comet.

I hope Nicolas decides to support an async api so that Node and other async solutions can be supported in the haxe std lib, and that he includes the node.js api in the next release of haxe. These two additions would be awesome for haxe3 and would catch a lot of attention. I've been banging on about haxe/node.js for about 3yrs, I feel we're almost at a tipping point where folks wake up to the benefits.

However, the node.js haxelib is available and up to date so you don't need to wait to get the benefits of node.js now.

R


On 11/15/2011 11:27 PM, Cauê Waneck wrote:
Hey guys!

I might be a little OT here...

Anyway, I haven't been very happy with apache performance for some time now, and I've decided to look for better alternatives. My #1 need by now is an haxe-based application server that also serves static files, but I'm also dying to be able to use comet with haxe as well, and maybe expand it to websockets later so it could also be a general-purpose push server as well.

After studying the alternatives, most of the mainstream solutions didn't make me very enthusiastic. What is really missing, and it seems that really "the right way" to do it is how Node JS is done - a lightweight server built on top of epoll and asynchronous i/o. There is more information about the i/o strategies in here http://www.kegel.com/c10k.html (very recommended reading). It seems it's the best way to scale, and it makes sense, since the cpu will be mostly busy processing the message (your code), and not delivering it (or related tasks, like thread context switching).
When I was looking for solutions, I stumbled at G-WAN ( http://www.gwan.ch/ ). Their benchmarks are really impressive, and it has some principles I admire, like a small and simple code base, despite all its pseudo-political statements. It advertises that you code in c, and it intercepts blocking calls and turns it to epoll instead. Seems magical. It's a pity that it doesn't work, their support is terrible, and it isn't open-source.
Anyway, when I first saw it I started coding a tora backend for it. Just then to know that their "magical" transform of sockets into asynchronous wasn't working.

I remember some time ago someone posted a server that a guy on haxe's irc channel made. His name is sality. Now that's a great start, I think, it's based on epoll, zero-copy, has already a tora backend, it's great!
But after studying about scaling network applications nowadays, it really seems that tora would easily become the bottleneck on a comet-server context, because for a comet server to be effective we would need to support something like.. 1k clients to be connected simultaneously.
On tora this would mean we would have to have 1k threads. And then we're back again to the problem of context switching between 1k threads (not to mention memory footprint, etc).

So it seems that really the best solution would be to have a kind of a tora server that spawns the quantity of threads with the exact number of cores the system has, and works totally asynchronous from then on.

I've read what Nicolas has written about node.js, and it's realy node.js' major downside how it deals with starvation, even though it seems to me that in cases where node.js clients would starve, tora would only raise the limit of non-starving clients, at the cost of overall performance.

So... I'd really like to know about what do you guys think? What would be the best solution? Maybe it would be to have two separate solutions, one for an application server (where an epoll-based server + tora seems to be a clear winner), and other for a comet server (where a totally async api would really make a difference).
Anyway it seems to me that asynchronous i/o api's are having more and more importance. It would be great to leverage this on haxe, and maybe create an async std api, based on neko?

Thanks!
Caue




-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org



Although my worker module uses node.js as a subprocess, you can easily fire off other backend processes and listen on the stdin/stdout from the main node event loop. So, you could have something similar to my worker module that does the same for java (some exe/neko sub process), that is, attaches to the stdin/stdout of java and controls it's processing.


-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Nicolas Cannasse
Le 16/11/2011 11:36, Cauê Waneck a écrit :
> have e.g. 50 tora threads that can take requests simultaneously. On a
> steady connection (e.g. comet) scenario, though, tora wouldn't be as
> effective, it seems.

We have been using Tora for permanent connections for several of our
services at motion-twin, which are chats and multiplayer games (such as
http://kube.en.muxxu.com).

This is done by opening another port on Tora on which Flash can connect
directly with a Socket. You will then be able to send requests (sames as
HTTP but using Tora protocol) and get answers back.

It can also listen on a tora.Queue on which some messages can be sent,
which will broadcast them to all listening clients.

So far it works very well and we didn't have scaling issues. For
instance Kube has a Ping of ~80ms which includes every step (sending
message,user authentication, message parsing, handling,response,and wait
time).

You also get the benefits of Tora, which are :

- isolated threads : you can handle several messages in parallel, which
is nice for multicore CPUs (NodeJS only use one of your cores :D ), and
without having to deal with multithreading issues

- code hot-swapping : you can deploy an updated version of your .n
bytecode without having to restart the server, actually clients don't
even get disconnected !

- same model as http : you get an URL request with parameters for each
message sent on your server, you don't have to write twice your
authentication layer, your database connection and all the error
handling : everything goes through the same code path

- interoperability with http : you can make standard Http requests as
well on the same Tora server, which means you can broadcast messages to
connected clients in a Http action.

You can read more about it there :
http://ncannasse.fr/blog/tora_comet

My main interest is first to be able to easily write code for
multiplayer games, which is often very complicated. Performances comes
second even if I'm as always very interested in the topic as well ;)

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
In reply to this post by Cauê W.
On 11/16/2011 07:36 AM, Cauê Waneck wrote:
> But anyway, I don't mean to say that Nicolas was bashing node.js,
> quite the contrary, really. I think he made a very valid criticism on
> it, which really makes sense. It doesn't matter that node.js is faster
> and in the long run it can handle the requests more quickly than other
> alternatives, if it has to completely finish a request to take another
I know you didn't say Nicolas was node bashing, I did :), the blog post
was a hit piece promoting neko/tora over node.js. Fair enough, his baby,
but the argument about starvation is a complete red herring by omitting
to point out that you can do sub process background processing async in
node.js.

In node, long running requests are farmed out to a subprocess - async,
which allows the main event loop to continue unabated until it receives
io back from the subprocess it then pushes it to the client. The main
event loop does not block, if someone can tell me how a threaded server
could do better let me know.  Further if one round robins to the
multiple subprocess each of those can be working on separate cores.

One valid criticism is the granularity of a subprocess to thread,
obviously one could subprocess to, e.g. a neko subprocess with multiple
threads or java for that matter, the point is there are solutions (and
I'm working on them), e.g. a neko threaded server running subprocess
could be quite nice.

Have a look at my worker module it doesn't get any easier or more
seamless to get worker subprocess using haxe remoting. Don't forget the
massive benefit of not having to debug threads/synchronisation.

R

--
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
In reply to this post by Nicolas Cannasse
On 11/16/2011 09:53 AM, Nicolas Cannasse wrote:
> isolated threads : you can handle several messages in parallel, which
> is nice for multicore CPUs (NodeJS only use one of your cores :D ),
> and without having to deal with multithreading issues

Nicolas can you please keep it real.

Thanks

R

--
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Nicolas Cannasse
Le 16/11/2011 14:40, blackdog a écrit :
> On 11/16/2011 09:53 AM, Nicolas Cannasse wrote:
>> isolated threads : you can handle several messages in parallel, which
>> is nice for multicore CPUs (NodeJS only use one of your cores :D ),
>> and without having to deal with multithreading issues
>
> Nicolas can you please keep it real.

It seems quite real to me, it's even part of the NodeJS FAQ : unless you
manually fork child processes, you will only use a single core. Creating
a process is a very expensive OS operation BTW.

I didn't mean to bash NodeJS, I'm just saying that I don't agree with
its overall architecture. Bashing is gratuitous, I'm making arguments
here based on my own experience writing similar piece of software.

I'm perfectly fine with async programming btw, it's just the way NodeJS
is written, in particular the fact that you have to move any CPU usage
task into a subprocess, as if one could easily anticipate which task in
particular will take the most CPU...

And it seems I'm not the only one...
http://teddziuba.com/2011/10/straight-talk-on-event-loops.html

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog

You're not telling the entire story. Sure you need subprocess, but it's possible, and can be done very seamlessly from haxe. It's like me saying, neko can't serve web pages, it's got this nekotools server that can handle one connection at a time - when in fact you sit it all behind apache and make it a very viable solution - but i didn't mention that bit. The fact is you need a ton of extra plumbing to serve pages from neko effectively, and you won't give node a subprocess :) Throw me a bone.

Ry's response to Dziuba

http://blog.nodejs.org/2011/10/04/an-easy-way-to-build-scalable-network-programs/

R





On 11/16/2011 11:03 AM, Nicolas Cannasse wrote:
Le 16/11/2011 14:40, blackdog a écrit :
On 11/16/2011 09:53 AM, Nicolas Cannasse wrote:
isolated threads : you can handle several messages in parallel, which
is nice for multicore CPUs (NodeJS only use one of your cores :D ),
and without having to deal with multithreading issues

Nicolas can you please keep it real.

It seems quite real to me, it's even part of the NodeJS FAQ : unless you manually fork child processes, you will only use a single core. Creating a process is a very expensive OS operation BTW.

I didn't mean to bash NodeJS, I'm just saying that I don't agree with its overall architecture. Bashing is gratuitous, I'm making arguments here based on my own experience writing similar piece of software.

I'm perfectly fine with async programming btw, it's just the way NodeJS is written, in particular the fact that you have to move any CPU usage task into a subprocess, as if one could easily anticipate which task in particular will take the most CPU...

And it seems I'm not the only one...
http://teddziuba.com/2011/10/straight-talk-on-event-loops.html

Best,
Nicolas



-- 
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Cauê W.
Hey guys, I don't mean to start a flame war, but just ask - how can we improve it?
Both systems have their merit and their weaknesses, I'd love to see how we can work these out with haxe. The answer could be (and I think it might be) as simple as tora + aync i/o.

blackdog, correct me if I'm wrong, but it seems to me that the main risk with node.js is when you have a certain area in your code that makes processing take more than the usual time. If you are running a node.js server and have some kind of vulnerability like that, one to four requests (one for each core?) can take the whole server down for some time, because of how it works. It seems that if things changed, specially about how it distributes tasks based on the event loop, it would be a more robust solution.
Great link, but it seems that he is assuming an infinite-core cpu, isn't he?

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
On 11/16/2011 11:56 AM, Cauê Waneck wrote:
> blackdog, correct me if I'm wrong, but it seems to me that the main
> risk with node.js is when you have a certain area in your code that
> makes processing take more than the usual time. If you are running a
> node.js server and have some kind of vulnerability like that, one to
> four requests (one for each core?) can take the whole server down for
> some time, because of how it works. It seems that if things changed,
> specially about how it distributes tasks based on the event loop, it
> would be a more robust solution.

You could have the subprocess do it's own thing on it's own core forever
it doesnt intefere with the main event loop, it just has no data coming
in from that io handle, therefore it's ignored by the main process event
loop.

Now, you could easily set a timeout Node.setTimeout(), usual javascript,
and kill a subprocess by signal if you have something hanging badly. For
example, one might implement erlang like restart semantics if you really
got into it.  But I fail to see how a threaded solution would be less
brittle than this - let's be clear that that kind of failure would be
problematic for a threaded system too (but probably harder to debug).

R



--
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Juraj Kirchheim
In reply to this post by Nicolas Cannasse
On Wed, Nov 16, 2011 at 3:03 PM, Nicolas Cannasse
<[hidden email]> wrote:

> Le 16/11/2011 14:40, blackdog a écrit :
>>
>> On 11/16/2011 09:53 AM, Nicolas Cannasse wrote:
>>>
>>> isolated threads : you can handle several messages in parallel, which
>>> is nice for multicore CPUs (NodeJS only use one of your cores :D ),
>>> and without having to deal with multithreading issues
>>
>> Nicolas can you please keep it real.
>
> It seems quite real to me, it's even part of the NodeJS FAQ : unless you
> manually fork child processes, you will only use a single core. Creating a
> process is a very expensive OS operation BTW.

Yes, but by pooling them, this is easily amortized.

> I didn't mean to bash NodeJS, I'm just saying that I don't agree with its
> overall architecture. Bashing is gratuitous, I'm making arguments here based
> on my own experience writing similar piece of software.
>
> I'm perfectly fine with async programming btw, it's just the way NodeJS is
> written, in particular the fact that you have to move any CPU usage task
> into a subprocess, as if one could easily anticipate which task in
> particular will take the most CPU...

This is a bit of a blanket statement. It is much like saying neko is a
poor tool, because I/O-intense tasks will block a whole worker.

Using MongoDB/CouchDB you can often push a lot of heavylifting right
to the DB. I think it has yet to be conclusively proven, that CPU is
likely to become an actual bottleneck. Nothing is to keep me from
running one node.js instance per core (thus in isolation - perhaps
with some interprocess communication similar to the tora queue) behind
an nginx and thus max out all cores. In the neko execution model
however, you need more than one thread per core to max it out, because
workers are largely stalled by blocking I/O.

As an alternative to subprocesses, you can also single out CPU-intense
tasks into asynchronous calls to C++ extensions (as far as mutability
permits this).

In any case, I think your assessment of the programming model is a bit
oversimplified.

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Nicolas Cannasse
Le 16/11/2011 16:20, Juraj Kirchheim a écrit :
> In any case, I think your assessment of the programming model is a bit
> oversimplified.

Again, I'm not criticizing async programming model in any of my posts.
But I think that NodeJS is making async programming a ideology instead
of a good practice.

I think that either NodeJS + threads or Tora + async calls would both be
great, and will actually look much more like Erlang actors.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

blackdog
On 11/16/2011 12:48 PM, Nicolas Cannasse wrote:
> Again, I'm not criticizing async programming model in any of my posts.
> But I think that NodeJS is making async programming a ideology instead
> of a good practice.

Making aync programming easier and more accessible is hardly an
ideology, it is however a viable alternative to thread base solutions
and accepted practice.

R

--
Simplicity is the ultimate sophistication. ~ Leonardo da Vinci


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

sledorze
In reply to this post by Cauê W.
Let's face it; you want async APIs? would take years to reimplement all that's available to nodeJs.
We have a big advantage with haXe; no reason to prevent us to surf on it!

Be it on nodeJs or neko; it's the natural complexity of scalability that arise.
Perhaps nodeJs may fool the newcomer by it's apparent simplicity.

That's said; I'm happy with the ecosystems that support them.
And if we want to push haXe more; it has to be present in those places.. we better invest in that!

Stephane
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Cauê W.
2011/11/16 sledorze <[hidden email]>
Let's face it; you want async APIs? would take years to reimplement all
that's available to nodeJs.
 
It might, but it would not take much to implement neko's std library in async, and I think it covers 90% of the use cases. That's what I really like in haxe/neko, it's simplicity.
 
Be it on nodeJs or neko; it's the natural complexity of scalability that
arise.
Perhaps nodeJs may fool the newcomer by it's apparent simplicity.

Sorry, I don't understand well what you're talking about? 

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

Cauê W.
In reply to this post by Nicolas Cannasse
Again, I'm not criticizing async programming model in any of my posts. But I think that NodeJS is making async programming a ideology instead of a good practice.

I think that either NodeJS + threads or Tora + async calls would both be great, and will actually look much more like Erlang actors.

This just gave me an idea. Will try to come up with a nice proof of concept!

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Node.JS, async and scaling revisited

laurence taylor
One thing you can do with Arrows, and I will keep herping on about this, is to make a synchronous api asynchronous, so that you could monkey patch that in and then build out proper asynchronous services as and when necessary, while keeping the api stable.

What api's might need to be made this way?

On Thu, Nov 17, 2011 at 1:42 PM, Cauê Waneck <[hidden email]> wrote:
Again, I'm not criticizing async programming model in any of my posts. But I think that NodeJS is making async programming a ideology instead of a good practice.

I think that either NodeJS + threads or Tora + async calls would both be great, and will actually look much more like Erlang actors.

This just gave me an idea. Will try to come up with a nice proof of concept!

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org