String + socket + Neko = question

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

String + socket + Neko = question

Viktor Hesselbom
Hello again everyone,

I'm here with another 'interesting' question.

Is there any limit to how long a string can be when sending it through sockets in Neko?

Like if I were trying to send a million byte long string through a socket, should I send it as one long string or split it up in parts?

The thing I'm trying to do is load a local file with a local Neko application and then send the file to a server running a ThreadServer application with Neko. I thought I'd load the file and send the content over sockets to the server and the server would recreate the file with the received bytes. Perhaps there's a better way for this?

Wow, so many questions. I would be really grateful if someone took the time to give me help and tips.

Thanks,
Viktor Hesselbom

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
I think this questions has gone missed. I'm sure someone must know something about it. Anything helps. :)

2008/11/10 Viktor Hesselbom <[hidden email]>
Hello again everyone,

I'm here with another 'interesting' question.

Is there any limit to how long a string can be when sending it through sockets in Neko?

Like if I were trying to send a million byte long string through a socket, should I send it as one long string or split it up in parts?

The thing I'm trying to do is load a local file with a local Neko application and then send the file to a server running a ThreadServer application with Neko. I thought I'd load the file and send the content over sockets to the server and the server would recreate the file with the received bytes. Perhaps there's a better way for this?

Wow, so many questions. I would be really grateful if someone took the time to give me help and tips.

Thanks,
Viktor Hesselbom


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Nicolas Cannasse
In reply to this post by Viktor Hesselbom
Viktor Hesselbom a écrit :
> Hello again everyone,
>
> I'm here with another 'interesting' question.
>
> Is there any limit to how long a string can be when sending it through
> sockets in Neko?

No. But the client will block until the whole string is sent. Also, in
order to no block the server, it's better to read the data little by little.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
Hello Nicolas,

Do you have a recommendation or estimation on about how much one should send? 1 MB, 100 MB, 1 GB? Not any exact value but an approximate.

Thanks,
Viktor

2008/11/12 Nicolas Cannasse <[hidden email]>
Viktor Hesselbom a écrit :

Hello again everyone,

I'm here with another 'interesting' question.

Is there any limit to how long a string can be when sending it through sockets in Neko?

No. But the client will block until the whole string is sent. Also, in order to no block the server, it's better to read the data little by little.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Nicolas Cannasse
Viktor Hesselbom a écrit :
> Hello Nicolas,
>
> Do you have a recommendation or estimation on about how much one should
> send? 1 MB, 100 MB, 1 GB? Not any exact value but an approximate.

It's appropriate to send what you have to send. No more, and no less :)

Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Gamehaxe
And you get a Zen lesson as well.

> Viktor Hesselbom a écrit :
>> Hello Nicolas,
>>  Do you have a recommendation or estimation on about how much one  
>> should send? 1 MB, 100 MB, 1 GB? Not any exact value but an approximate.
>
> It's appropriate to send what you have to send. No more, and no less :)
>
> Nicolas
>



--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
In reply to this post by Nicolas Cannasse
Ok, but if I wanted to send, say 1GB, should I send it all at once or in parts? I figure parts but not sure how big parts should be to not lock the server.

2008/11/18 Nicolas Cannasse <[hidden email]>
Viktor Hesselbom a écrit :
Hello Nicolas,


Do you have a recommendation or estimation on about how much one should send? 1 MB, 100 MB, 1 GB? Not any exact value but an approximate.

It's appropriate to send what you have to send. No more, and no less :)


Nicolas

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Nicolas Cannasse
Viktor Hesselbom a écrit :
> Ok, but if I wanted to send, say 1GB, should I send it all at once or in
> parts? I figure parts but not sure how big parts should be to not lock
> the server.

For huge files, sending by parts is good since you might not want first
to read your whole file into memory... So read little by little, with
something like a 256KB buffer, and send little by little to the server.

Same for the server : he can read little by little, and write little by
little into a file.

Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
Hello Nicolas,

Thank you very much for all your help, it is greatly appreciated.

Now, how can I read parts of a file with Neko? With neko.io.File.read() I get the whole file in memory, right? I see no function that reads only certain parts of a file.

2008/11/18 Nicolas Cannasse <[hidden email]>
Viktor Hesselbom a écrit :

Ok, but if I wanted to send, say 1GB, should I send it all at once or in parts? I figure parts but not sure how big parts should be to not lock the server.

For huge files, sending by parts is good since you might not want first to read your whole file into memory... So read little by little, with something like a 256KB buffer, and send little by little to the server.

Same for the server : he can read little by little, and write little by little into a file.


Nicolas

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

clemos
On Tue, Nov 18, 2008 at 3:53 PM, Viktor Hesselbom
<[hidden email]> wrote:
> Hello Nicolas,
> Thank you very much for all your help, it is greatly appreciated.
> Now, how can I read parts of a file with Neko? With neko.io.File.read() I
> get the whole file in memory, right?

No I don't think so.
neko.io.File.read() only returns a neko.io.FileInput, which extends
haxe.io.Input.
It doesn't load the whole file in memory, it only provides access to the data.
(I guess the memory is only used when actually reading the Input)

> I see no function that reads only
> certain parts of a file.

Use readBytes, in haxe.io.Input, for example

++++++
Clément

> 2008/11/18 Nicolas Cannasse <[hidden email]>
>>
>> Viktor Hesselbom a écrit :
>>>
>>> Ok, but if I wanted to send, say 1GB, should I send it all at once or in
>>> parts? I figure parts but not sure how big parts should be to not lock the
>>> server.
>>
>> For huge files, sending by parts is good since you might not want first to
>> read your whole file into memory... So read little by little, with something
>> like a 256KB buffer, and send little by little to the server.
>>
>> Same for the server : he can read little by little, and write little by
>> little into a file.
>>
>> Nicolas
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
Oh I see.

Thank you very much, that should be all.

Viktor

2008/11/18 clemos <[hidden email]>
On Tue, Nov 18, 2008 at 3:53 PM, Viktor Hesselbom
<[hidden email]> wrote:
> Hello Nicolas,
> Thank you very much for all your help, it is greatly appreciated.
> Now, how can I read parts of a file with Neko? With neko.io.File.read() I
> get the whole file in memory, right?

No I don't think so.
neko.io.File.read() only returns a neko.io.FileInput, which extends
haxe.io.Input.
It doesn't load the whole file in memory, it only provides access to the data.
(I guess the memory is only used when actually reading the Input)

> I see no function that reads only
> certain parts of a file.

Use readBytes, in haxe.io.Input, for example

++++++
Clément

> 2008/11/18 Nicolas Cannasse <[hidden email]>
>>
>> Viktor Hesselbom a écrit :
>>>
>>> Ok, but if I wanted to send, say 1GB, should I send it all at once or in
>>> parts? I figure parts but not sure how big parts should be to not lock the
>>> server.
>>
>> For huge files, sending by parts is good since you might not want first to
>> read your whole file into memory... So read little by little, with something
>> like a 256KB buffer, and send little by little to the server.
>>
>> Same for the server : he can read little by little, and write little by
>> little into a file.
>>
>> Nicolas
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
In reply to this post by clemos
Ok, so readBytes has three parameters (s : Bytes, pos : Int, len : Int). I understand 'pos' must be the position to start reading and 'len' how far to read. Now 's' I assume is a reference to a Bytes instance where the read bytes will go, is that correct? Then why does the function return an Int? What does this Int represent?

Sorry, it just gets a little confusing with no description for some functions.

Thanks,
Viktor

2008/11/18 clemos <[hidden email]>
On Tue, Nov 18, 2008 at 3:53 PM, Viktor Hesselbom
<[hidden email]> wrote:
> Hello Nicolas,
> Thank you very much for all your help, it is greatly appreciated.
> Now, how can I read parts of a file with Neko? With neko.io.File.read() I
> get the whole file in memory, right?

No I don't think so.
neko.io.File.read() only returns a neko.io.FileInput, which extends
haxe.io.Input.
It doesn't load the whole file in memory, it only provides access to the data.
(I guess the memory is only used when actually reading the Input)

> I see no function that reads only
> certain parts of a file.

Use readBytes, in haxe.io.Input, for example

++++++
Clément

> 2008/11/18 Nicolas Cannasse <[hidden email]>
>>
>> Viktor Hesselbom a écrit :
>>>
>>> Ok, but if I wanted to send, say 1GB, should I send it all at once or in
>>> parts? I figure parts but not sure how big parts should be to not lock the
>>> server.
>>
>> For huge files, sending by parts is good since you might not want first to
>> read your whole file into memory... So read little by little, with something
>> like a 256KB buffer, and send little by little to the server.
>>
>> Same for the server : he can read little by little, and write little by
>> little into a file.
>>
>> Nicolas
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Franco Ponticelli

Ok, so readBytes has three parameters (s : Bytes, pos : Int, len : Int). I understand 'pos' must be the position to start reading and 'len' how far to read. Now 's' I assume is a reference to a Bytes instance where the read bytes will go, is that correct? Then why does the function return an Int? What does this Int represent?

You say to the function that you want to read "len" bytes but you probably don't know how many bytes will be read in practice. The return value gives back exactly that number. This is usually true for loops where the last chunk is shorter than the previous ones.

Franco

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
Oh, makes sense. That is great.

2008/11/19 Franco Ponticelli <[hidden email]>

Ok, so readBytes has three parameters (s : Bytes, pos : Int, len : Int). I understand 'pos' must be the position to start reading and 'len' how far to read. Now 's' I assume is a reference to a Bytes instance where the read bytes will go, is that correct? Then why does the function return an Int? What does this Int represent?

You say to the function that you want to read "len" bytes but you probably don't know how many bytes will be read in practice. The return value gives back exactly that number. This is usually true for loops where the last chunk is shorter than the previous ones.

Franco

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Donny Viszneki
The Linux kernel (and probably other kernels) now supports patching
data through from one readable socket to a writable socket without the
intervening copy of memory into program space. It sounds like you're
sending a *lot* of data, and although no one has suffered a great deal
before their favorite kernel implemented this sort of functionality,
this train of thought does have me asking: if you're transferring this
much data, *what* are you sending?- and would it be easier to send it
using existing tools?

On Wed, Nov 19, 2008 at 7:17 AM, Viktor Hesselbom
<[hidden email]> wrote:

> Oh, makes sense. That is great.
> 2008/11/19 Franco Ponticelli <[hidden email]>
>>
>>> Ok, so readBytes has three parameters (s : Bytes, pos : Int, len : Int).
>>> I understand 'pos' must be the position to start reading and 'len' how far
>>> to read. Now 's' I assume is a reference to a Bytes instance where the read
>>> bytes will go, is that correct? Then why does the function return an Int?
>>> What does this Int represent?
>>
>> You say to the function that you want to read "len" bytes but you probably
>> don't know how many bytes will be read in practice. The return value gives
>> back exactly that number. This is usually true for loops where the last
>> chunk is shorter than the previous ones.
>> Franco
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>



--
http://codebad.com/

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: String + socket + Neko = question

Viktor Hesselbom
Hi Donny,

I probably won't send this much, more like a couple of megs max per file, but I gave a large example because I still want it to be able to send this high amount without trouble, except that it will take longer to send of course.

2008/11/21 Donny Viszneki <[hidden email]>
The Linux kernel (and probably other kernels) now supports patching
data through from one readable socket to a writable socket without the
intervening copy of memory into program space. It sounds like you're
sending a *lot* of data, and although no one has suffered a great deal
before their favorite kernel implemented this sort of functionality,
this train of thought does have me asking: if you're transferring this
much data, *what* are you sending?- and would it be easier to send it
using existing tools?

On Wed, Nov 19, 2008 at 7:17 AM, Viktor Hesselbom
<[hidden email]> wrote:
> Oh, makes sense. That is great.
> 2008/11/19 Franco Ponticelli <[hidden email]>
>>
>>> Ok, so readBytes has three parameters (s : Bytes, pos : Int, len : Int).
>>> I understand 'pos' must be the position to start reading and 'len' how far
>>> to read. Now 's' I assume is a reference to a Bytes instance where the read
>>> bytes will go, is that correct? Then why does the function return an Int?
>>> What does this Int represent?
>>
>> You say to the function that you want to read "len" bytes but you probably
>> don't know how many bytes will be read in practice. The return value gives
>> back exactly that number. This is usually true for loops where the last
>> chunk is shorter than the previous ones.
>> Franco
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>



--
http://codebad.com/

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org