A very strange problem about neko ThreadServer

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

A very strange problem about neko ThreadServer

Fei Yin
This is very strange that I 'm testing a simple multi thread server under nekovm . The code and runtime file is in attach .

But the server always give me a error like this :

Called from /usr/lib/haxe/std/neko/net/Poll.hx line 75
Called from /usr/lib/haxe/std/neko/net/ThreadServer.hx line 113
Called from /usr/lib/haxe/std/neko/net/ThreadServer.hx line 74^CToo many sockets in poll

I just make 600 empty socket connection. My computer is AMD Athlon Dual Core 5200+ , 2G ram, Ubuntu 10.10 32-bit . with neko 1.82.

I try to ask this questions in IRC , blue112 tell me it maybe ulimit problem , Then I try to install a Ubuntu Server in VirtualBox , and set the ulimit to 65535 , but still have this peoblem.

Anybody can help me ?
--
Yin Fei
From Icebirds.net


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: A very strange problem about neko ThreadServer

John Plsek
ThreadServer is not the greatest bit of programming I've seen. Under linux, it's default maximums are 10 threads, each with 64 sockets. So you think it should handle 640 sockets.

The problem is, when a new socket is connected, it gets "assigned" to one of the 10 random worker threads. The worker thread blindly adds the socket to it's array of sockets without consideration to the maximum sockets per thread (64). The 65th concurrent open socket on a thread will cause that error. This can happen as early as the 65th socket connection, if the random number returns the same thread 65 times - unlikely, but possible

Changing addClient in ThreadServer - or overriding it in a class extending ThreadServer - to the following, ensures ThreadServer can handle up to nthreads * maxSocketPerThread sockets before greacefully refusing the connection

    function addClient( sock : neko.net.Socket )
    {
        var tid = Std.random(nthreads);
        var firstTid = tid;
        // start at a random thread, find the first thread with less than maxSockPerThread clients
        while (true)
        {
       
            if (threads[tid].socks.length < maxSockPerThread)
            {
           
                var infos : ClientInfos<Client> = {
                    thread : threads[tid],
                    client : clientConnected(sock),
                    sock : sock,
                    buf : haxe.io.Bytes.alloc(initialBufferSize),
                    bufpos : 0,
                };
                sock.custom = infos;
                infos.thread.t.sendMessage({ s : sock, cnx : true });
                break;
               
            }
           
            tid = (tid + 1) % nthreads; // try the next thread
           
            if (tid == firstTid)
            {
                // we've searched all threads, all are full, shut this new socket down gracefully
                // you could send some response here if you want to be really graceful for HTTP connections, eg:
                // s.write("HTTP/1.0 503 Service Unavailable\r\n\r\n");
                s.shutdown(true, true);
                break;
               
            }
        }
    }


On 24 September 2011 05:41, Fei Yin <[hidden email]> wrote:
This is very strange that I 'm testing a simple multi thread server under nekovm . The code and runtime file is in attach .

But the server always give me a error like this :

Called from /usr/lib/haxe/std/neko/net/Poll.hx line 75
Called from /usr/lib/haxe/std/neko/net/ThreadServer.hx line 113
Called from /usr/lib/haxe/std/neko/net/ThreadServer.hx line 74^CToo many sockets in poll

I just make 600 empty socket connection. My computer is AMD Athlon Dual Core 5200+ , 2G ram, Ubuntu 10.10 32-bit . with neko 1.82.

I try to ask this questions in IRC , blue112 tell me it maybe ulimit problem , Then I try to install a Ubuntu Server in VirtualBox , and set the ulimit to 65535 , but still have this peoblem.

Anybody can help me ?
--
Yin Fei
From Icebirds.net


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org