Process, and File.read/write(myFIFO)

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Process, and File.read/write(myFIFO)

Berend van Berkum
hi all,


i'm at a complete loss after trying for several hours,
perhaps someone here can write a few words on this..

First, i was trying to get a write handle on a fifo.
But at every attempt the script locks at the line:

        File.write(fifo, true);

Where the fifo is used in something like:

        var fifo = "fin";
        Sys.command("mkfifo", [fifo]);
        var proc = new Process("sh", ["-c", "backend", "<", fifo]);

Then, to test some things first:

        var proc = new Process("echo.py", []);
        proc.stdin.writeString("data1");
        proc.stdin.flush();
        trace(proc.stdout.readByte());
        trace(proc.stdout.readAll());
        proc.close();

Where echo.py will do exactly as the name suggests, and also flush its stdout.
But here any read operation on stdout will block.
This is on linux, reading output from e.g. 'ls' works fine.
PHP or Neko, no difference.

I think I saw somewhere on the lists that working with stdin/stdout
synchronously won't work yet. But that may be quite old, and im not sure if it
applies. This can be worked around by using a socket connection to an wrapper
around "backend", but alas, hoped this could work..


regards, Berend


--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
    http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Randy Stauner
i don't entirely understand what you're trying to figure out,
but in your third example, the readAll is probably wating for an EOF.

You could:
  * Add 'proc.stdin.close()' before you start reading... then readAll will return and the program will finish
  * Or you could add a newline 'proc.stdin.writeString("data\n");' and then use 'proc.stdout.readLine()' instead of readAll(). This too will allow the data to print and the program to finish.

Maybe this will help?

On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum <[hidden email]> wrote:
hi all,


i'm at a complete loss after trying for several hours,
perhaps someone here can write a few words on this..

First, i was trying to get a write handle on a fifo.
But at every attempt the script locks at the line:

       File.write(fifo, true);

Where the fifo is used in something like:

       var fifo = "fin";
       Sys.command("mkfifo", [fifo]);
       var proc = new Process("sh", ["-c", "backend", "<", fifo]);

Then, to test some things first:

       var proc = new Process("echo.py", []);
       proc.stdin.writeString("data1");
       proc.stdin.flush();
       trace(proc.stdout.readByte());
       trace(proc.stdout.readAll());
       proc.close();

Where echo.py will do exactly as the name suggests, and also flush its stdout.
But here any read operation on stdout will block.
This is on linux, reading output from e.g. 'ls' works fine.
PHP or Neko, no difference.

I think I saw somewhere on the lists that working with stdin/stdout
synchronously won't work yet. But that may be quite old, and im not sure if it
applies. This can be worked around by using a socket connection to an wrapper
around "backend", but alas, hoped this could work..


regards, Berend


--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
   http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Berend van Berkum

Randy, thanks for the suggestions. It'll give me something for further
experimenting.

The backend in question is Udanax Green (udanax.com). But anyway, largely
irrelevant for now, and expected to be buggy also.
Nevertheless Python equivalent scripts do function better.

The issue though, my echo experiment (stdin->stdout) does not make sense.
Every other test showed the (~3 line) echo.py to perform as expected..

The block on stdout read i cannot understand, as the data has been written by the
python script.
As shown i tried several read variants. But even reading the expected number of
bytes will block.


Thank you again. I'll have a go at it later and get back here.
Perhaps this behaviour might be wiki-worthy.



regards, Berend


----

Read will block, why?

        var proc = new Process("echo.py", []);
        proc.stdin.writeString("data1");
        proc.stdin.flush();
        trace(proc.stdout.read(5));
        proc.close();


On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:

>    i don't entirely understand what you're trying to figure out,
>    but in your third example, the readAll is probably wating for an EOF.
>    You could:
>    �  * Add 'proc.stdin.close()' before you start reading... then readAll
>    will return and the program will finish
>    �  * Or you could add a newline 'proc.stdin.writeString("data\n");' and
>    then use 'proc.stdout.readLine()' instead of readAll(). This too will
>    allow the data to print and the program to finish.
>    Maybe this will help?
>
>    On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>    <[1][hidden email]> wrote:
>
>      hi all,
>      i'm at a complete loss after trying for several hours,
>      perhaps someone here can write a few words on this..
>      First, i was trying to get a write handle on a fifo.
>      But at every attempt the script locks at the line:
>      �  �  �  � File.write(fifo, true);
>      Where the fifo is used in something like:
>      �  �  �  � var fifo = "fin";
>      �  �  �  � Sys.command("mkfifo", [fifo]);
>      �  �  �  � var proc = new Process("sh", ["-c", "backend", "<",
>      fifo]);
>      Then, to test some things first:
>      �  �  �  � var proc = new Process("echo.py", []);
>      �  �  �  � proc.stdin.writeString("data1");
>      �  �  �  � proc.stdin.flush();
>      �  �  �  � trace(proc.stdout.readByte());
>      �  �  �  � trace(proc.stdout.readAll());
>      �  �  �  � proc.close();
>      Where echo.py will do exactly as the name suggests, and also flush
>      its stdout.
>      But here any read operation on stdout will block.
>      This is on linux, reading output from e.g. 'ls' works fine.
>      PHP or Neko, no difference.
>      I think I saw somewhere on the lists that working with stdin/stdout
>      synchronously won't work yet. But that may be quite old, and im not
>      sure if it
>      applies. This can be worked around by using a socket connection to
>      an wrapper
>      around "backend", but alas, hoped this could work..
>      regards, Berend
>      --
>      � E: [2][hidden email]
>      � T: +31(0)6 - 19160770
>      � W: [3]http://services.dotmpe.com/
>      �  � [4]http://www.linkedin.com/pub/13/261/286
>      --
>      haXe - an open source web programming language
>      [5]http://haxe.org
>
> References
>
>    1. mailto:[hidden email]
>    2. mailto:[hidden email]
>    3. http://services.dotmpe.com/
>    4. http://www.linkedin.com/pub/13/261/286
>    5. http://haxe.org/

> --
> haXe - an open source web programming language
> http://haxe.org

--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
    http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Juan Delgado
Have you tried reading line by line till EOF? That's how I'm finally
doing in XAPI. Something like:

s = '';
while(true)
{
   try{ s += stream.readLine() + '\n'; } catch(e : haxe.io.Eof){ break; }
}

Actual code here:

http://github.com/zarate/xapi/blob/master/src/xa/Process.hx

As per working with Process, I've finally come to work:

* Call the Process
* Wait for exit code (this will block)
* Read output and error.

It seems to work consistently, although SWFMill was giving me some
problems I never got to the real reason why.

Not sure helping too much :/

Juan


On Wed, Mar 17, 2010 at 2:58 PM, Berend van Berkum <[hidden email]> wrote:

>
> Randy, thanks for the suggestions. It'll give me something for further
> experimenting.
>
> The backend in question is Udanax Green (udanax.com). But anyway, largely
> irrelevant for now, and expected to be buggy also.
> Nevertheless Python equivalent scripts do function better.
>
> The issue though, my echo experiment (stdin->stdout) does not make sense.
> Every other test showed the (~3 line) echo.py to perform as expected..
>
> The block on stdout read i cannot understand, as the data has been written by the
> python script.
> As shown i tried several read variants. But even reading the expected number of
> bytes will block.
>
>
> Thank you again. I'll have a go at it later and get back here.
> Perhaps this behaviour might be wiki-worthy.
>
>
>
> regards, Berend
>
>
> ----
>
> Read will block, why?
>
>        var proc = new Process("echo.py", []);
>        proc.stdin.writeString("data1");
>        proc.stdin.flush();
>        trace(proc.stdout.read(5));
>        proc.close();
>
>
> On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:
>>    i don't entirely understand what you're trying to figure out,
>>    but in your third example, the readAll is probably wating for an EOF.
>>    You could:
>>    Â  * Add 'proc.stdin.close()' before you start reading... then readAll
>>    will return and the program will finish
>>    Â  * Or you could add a newline 'proc.stdin.writeString("data\n");' and
>>    then use 'proc.stdout.readLine()' instead of readAll(). This too will
>>    allow the data to print and the program to finish.
>>    Maybe this will help?
>>
>>    On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>>    <[1][hidden email]> wrote:
>>
>>      hi all,
>>      i'm at a complete loss after trying for several hours,
>>      perhaps someone here can write a few words on this..
>>      First, i was trying to get a write handle on a fifo.
>>      But at every attempt the script locks at the line:
>>      Â  Â  Â  Â File.write(fifo, true);
>>      Where the fifo is used in something like:
>>      Â  Â  Â  Â var fifo = "fin";
>>      Â  Â  Â  Â Sys.command("mkfifo", [fifo]);
>>      Â  Â  Â  Â var proc = new Process("sh", ["-c", "backend", "<",
>>      fifo]);
>>      Then, to test some things first:
>>      Â  Â  Â  Â var proc = new Process("echo.py", []);
>>      Â  Â  Â  Â proc.stdin.writeString("data1");
>>      Â  Â  Â  Â proc.stdin.flush();
>>      Â  Â  Â  Â trace(proc.stdout.readByte());
>>      Â  Â  Â  Â trace(proc.stdout.readAll());
>>      Â  Â  Â  Â proc.close();
>>      Where echo.py will do exactly as the name suggests, and also flush
>>      its stdout.
>>      But here any read operation on stdout will block.
>>      This is on linux, reading output from e.g. 'ls' works fine.
>>      PHP or Neko, no difference.
>>      I think I saw somewhere on the lists that working with stdin/stdout
>>      synchronously won't work yet. But that may be quite old, and im not
>>      sure if it
>>      applies. This can be worked around by using a socket connection to
>>      an wrapper
>>      around "backend", but alas, hoped this could work..
>>      regards, Berend
>>      --
>>      Â E: [2][hidden email]
>>      Â T: +31(0)6 - 19160770
>>      Â W: [3]http://services.dotmpe.com/
>>      Â  Â [4]http://www.linkedin.com/pub/13/261/286
>>      --
>>      haXe - an open source web programming language
>>      [5]http://haxe.org
>>
>> References
>>
>>    1. mailto:[hidden email]
>>    2. mailto:[hidden email]
>>    3. http://services.dotmpe.com/
>>    4. http://www.linkedin.com/pub/13/261/286
>>    5. http://haxe.org/
>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>
> --
>  E: [hidden email]
>  T: +31(0)6 - 19160770
>  W: http://services.dotmpe.com/
>    http://www.linkedin.com/pub/13/261/286
>
>
> --
> haXe - an open source web programming language
> http://haxe.org
>



--
Juan Delgado - Zárate
http://zarate.tv
http://blog.zarate.tv

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Berend van Berkum
In reply to this post by Randy Stauner

If Neko writes to stdin, and then reads stdout, the echo script somehow never
finishes its own read.

E.g.

        proc = new Process("echo.py", []);
        proc.stdin.write("data");
        proc.close();

echo.py receives the data as expected. However:

        proc = new Process("echo.py", []);
        proc.stdin.write("data");
        proc.stdout.read(4);
        proc.close();

echo.py does not receive the data at its stdin, and the given
script hangs on the stdout.read, as I told before.

Writing EOT, LF, or using readLine: no effect.


I think I'll conclude here that std streams are unsuitable for communicating
with running processes. (The socket variant works fine).


regards, Berend


On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:

>    i don't entirely understand what you're trying to figure out,
>    but in your third example, the readAll is probably wating for an EOF.
>    You could:
>    �  * Add 'proc.stdin.close()' before you start reading... then readAll
>    will return and the program will finish
>    �  * Or you could add a newline 'proc.stdin.writeString("data\n");' and
>    then use 'proc.stdout.readLine()' instead of readAll(). This too will
>    allow the data to print and the program to finish.
>    Maybe this will help?
>
>    On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>    <[1][hidden email]> wrote:
>
>      hi all,
>      i'm at a complete loss after trying for several hours,
>      perhaps someone here can write a few words on this..
>      First, i was trying to get a write handle on a fifo.
>      But at every attempt the script locks at the line:
>      �  �  �  � File.write(fifo, true);
>      Where the fifo is used in something like:
>      �  �  �  � var fifo = "fin";
>      �  �  �  � Sys.command("mkfifo", [fifo]);
>      �  �  �  � var proc = new Process("sh", ["-c", "backend", "<",
>      fifo]);
>      Then, to test some things first:
>      �  �  �  � var proc = new Process("echo.py", []);
>      �  �  �  � proc.stdin.writeString("data1");
>      �  �  �  � proc.stdin.flush();
>      �  �  �  � trace(proc.stdout.readByte());
>      �  �  �  � trace(proc.stdout.readAll());
>      �  �  �  � proc.close();
>      Where echo.py will do exactly as the name suggests, and also flush
>      its stdout.
>      But here any read operation on stdout will block.
>      This is on linux, reading output from e.g. 'ls' works fine.
>      PHP or Neko, no difference.
>      I think I saw somewhere on the lists that working with stdin/stdout
>      synchronously won't work yet. But that may be quite old, and im not
>      sure if it
>      applies. This can be worked around by using a socket connection to
>      an wrapper
>      around "backend", but alas, hoped this could work..
>      regards, Berend
>      --
>      � E: [2][hidden email]
>      � T: +31(0)6 - 19160770
>      � W: [3]http://services.dotmpe.com/
>      �  � [4]http://www.linkedin.com/pub/13/261/286
>      --
>      haXe - an open source web programming language
>      [5]http://haxe.org
>
> References
>
>    1. mailto:[hidden email]
>    2. mailto:[hidden email]
>    3. http://services.dotmpe.com/
>    4. http://www.linkedin.com/pub/13/261/286
>    5. http://haxe.org/

> --
> haXe - an open source web programming language
> http://haxe.org

--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
    http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Justin Donaldson
Here's a quick demo of handling stdin/stdout between processes.  Looks like you're not closing your stdin requests in your message, and maybe a few other details.  The example below is a small script that will grep for itself in the current directory listing.

import neko.io.Process;
class ProcessDemo
{
    public static function main(): Void
    {

        var proc = new Process("ls", ['-l']); // set up a new process that operates on arguments
        proc.exitCode(); // wait for it to finish...


        var proc2 = new Process("grep",['ProcessDemo.n']); // set up another process
        proc2.stdin.writeInput(proc.stdout); // you can just write the stdout of the first process to the stdin here
/*        var result = proc.stdout.readString(); // or save the result as a string*/
/*        proc2.stdin.writeString(result);  // and write it in as a string*/
        proc2.stdin.close(); // close stdin (which will usually make it start running)
        proc2.exitCode(); // wait for it to finish.
        trace('result of grep');
        var result2 =proc2.stdout.readAll();
        trace(result2); // print the result
   
    }
}



#build file
-cp src
-main ProcessDemo
-x ProcessDemo




On Wed, Mar 17, 2010 at 4:02 PM, Berend van Berkum <[hidden email]> wrote:

If Neko writes to stdin, and then reads stdout, the echo script somehow never
finishes its own read.

E.g.

       proc = new Process("echo.py", []);
       proc.stdin.write("data");
       proc.close();

echo.py receives the data as expected. However:

       proc = new Process("echo.py", []);
       proc.stdin.write("data");
       proc.stdout.read(4);
       proc.close();

echo.py does not receive the data at its stdin, and the given
script hangs on the stdout.read, as I told before.

Writing EOT, LF, or using readLine: no effect.


I think I'll conclude here that std streams are unsuitable for communicating
with running processes. (The socket variant works fine).


regards, Berend


On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:
>    i don't entirely understand what you're trying to figure out,
>    but in your third example, the readAll is probably wating for an EOF.
>    You could:
>    Â  * Add 'proc.stdin.close()' before you start reading... then readAll
>    will return and the program will finish
>    Â  * Or you could add a newline 'proc.stdin.writeString("data\n");' and
>    then use 'proc.stdout.readLine()' instead of readAll(). This too will
>    allow the data to print and the program to finish.
>    Maybe this will help?
>
>    On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>    <[1][hidden email]> wrote:
>
>      hi all,
>      i'm at a complete loss after trying for several hours,
>      perhaps someone here can write a few words on this..
>      First, i was trying to get a write handle on a fifo.
>      But at every attempt the script locks at the line:
>      Â  Â  Â  Â File.write(fifo, true);
>      Where the fifo is used in something like:
>      Â  Â  Â  Â var fifo = "fin";
>      Â  Â  Â  Â Sys.command("mkfifo", [fifo]);
>      Â  Â  Â  Â var proc = new Process("sh", ["-c", "backend", "<",
>      fifo]);
>      Then, to test some things first:
>      Â  Â  Â  Â var proc = new Process("echo.py", []);
>      Â  Â  Â  Â proc.stdin.writeString("data1");
>      Â  Â  Â  Â proc.stdin.flush();
>      Â  Â  Â  Â trace(proc.stdout.readByte());
>      Â  Â  Â  Â trace(proc.stdout.readAll());
>      Â  Â  Â  Â proc.close();
>      Where echo.py will do exactly as the name suggests, and also flush
>      its stdout.
>      But here any read operation on stdout will block.
>      This is on linux, reading output from e.g. 'ls' works fine.
>      PHP or Neko, no difference.
>      I think I saw somewhere on the lists that working with stdin/stdout
>      synchronously won't work yet. But that may be quite old, and im not
>      sure if it
>      applies. This can be worked around by using a socket connection to
>      an wrapper
>      around "backend", but alas, hoped this could work..
>      regards, Berend
>      --
>      Â E: [2][hidden email]
>      Â T: +31(0)6 - 19160770
>      Â W: [3]http://services.dotmpe.com/
>      Â  Â [4]http://www.linkedin.com/pub/13/261/286
>      --
>      haXe - an open source web programming language
>      [5]http://haxe.org
>
> References
>
>    1. mailto:[hidden email]
>    2. mailto:[hidden email]
>    3. http://services.dotmpe.com/
>    4. http://www.linkedin.com/pub/13/261/286
>    5. http://haxe.org/

> --
> haXe - an open source web programming language
> http://haxe.org

--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
   http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org



--
Justin Donaldson
PhD Candidate, Informatics
Indiana University
http://www.scwn.net
aim: iujjd
twitter: jjdonald

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Randy Stauner
In reply to this post by Berend van Berkum
can you attach your echo.py script?
I've been using 'cat' assuming it operated the same way.
Using 'cat' your second example

var proc = new Process("cat", []);
proc.stdin.writeString("data");
proc.stdout.read(4);
proc.close();

works for me.
Notice I changed 'stdin.write' to 'stdin.writeString' b/c compilation failed with a type error.
If you trace the return from stdout.read(4) it will be 'data'.

So I'm wondering if the echo.py script is what's causing your neko program to hang.
It might be waiting for a newline or EOF (when you've only written "data" to it) before producing its first output (which the script is waiting to read).

On Wed, Mar 17, 2010 at 1:02 PM, Berend van Berkum <[hidden email]> wrote:

If Neko writes to stdin, and then reads stdout, the echo script somehow never
finishes its own read.

E.g.

       proc = new Process("echo.py", []);
       proc.stdin.write("data");
       proc.close();

echo.py receives the data as expected. However:

       proc = new Process("echo.py", []);
       proc.stdin.write("data");
       proc.stdout.read(4);
       proc.close();

echo.py does not receive the data at its stdin, and the given
script hangs on the stdout.read, as I told before.

Writing EOT, LF, or using readLine: no effect.


I think I'll conclude here that std streams are unsuitable for communicating
with running processes. (The socket variant works fine).


regards, Berend


On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:
>    i don't entirely understand what you're trying to figure out,
>    but in your third example, the readAll is probably wating for an EOF.
>    You could:
>    Â  * Add 'proc.stdin.close()' before you start reading... then readAll
>    will return and the program will finish
>    Â  * Or you could add a newline 'proc.stdin.writeString("data\n");' and
>    then use 'proc.stdout.readLine()' instead of readAll(). This too will
>    allow the data to print and the program to finish.
>    Maybe this will help?
>
>    On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>    <[1][hidden email]> wrote:
>
>      hi all,
>      i'm at a complete loss after trying for several hours,
>      perhaps someone here can write a few words on this..
>      First, i was trying to get a write handle on a fifo.
>      But at every attempt the script locks at the line:
>      Â  Â  Â  Â File.write(fifo, true);
>      Where the fifo is used in something like:
>      Â  Â  Â  Â var fifo = "fin";
>      Â  Â  Â  Â Sys.command("mkfifo", [fifo]);
>      Â  Â  Â  Â var proc = new Process("sh", ["-c", "backend", "<",
>      fifo]);
>      Then, to test some things first:
>      Â  Â  Â  Â var proc = new Process("echo.py", []);
>      Â  Â  Â  Â proc.stdin.writeString("data1");
>      Â  Â  Â  Â proc.stdin.flush();
>      Â  Â  Â  Â trace(proc.stdout.readByte());
>      Â  Â  Â  Â trace(proc.stdout.readAll());
>      Â  Â  Â  Â proc.close();
>      Where echo.py will do exactly as the name suggests, and also flush
>      its stdout.
>      But here any read operation on stdout will block.
>      This is on linux, reading output from e.g. 'ls' works fine.
>      PHP or Neko, no difference.
>      I think I saw somewhere on the lists that working with stdin/stdout
>      synchronously won't work yet. But that may be quite old, and im not
>      sure if it
>      applies. This can be worked around by using a socket connection to
>      an wrapper
>      around "backend", but alas, hoped this could work..
>      regards, Berend
>      --
>      Â E: [2][hidden email]
>      Â T: +31(0)6 - 19160770
>      Â W: [3]http://services.dotmpe.com/
>      Â  Â [4]http://www.linkedin.com/pub/13/261/286
>      --
>      haXe - an open source web programming language
>      [5]http://haxe.org
>
> References
>
>    1. mailto:[hidden email]
>    2. mailto:[hidden email]
>    3. http://services.dotmpe.com/
>    4. http://www.linkedin.com/pub/13/261/286
>    5. http://haxe.org/

> --
> haXe - an open source web programming language
> http://haxe.org

--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
   http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: Process, and File.read/write(myFIFO)

Berend van Berkum


On Wed, Mar 17, 2010 at 04:12:51PM -0700, Randy Stauner wrote:
>    can you attach your echo.py script?
>    I've been using 'cat' assuming it operated the same way.

Hah! Ofcourse, keep it simple, and yes, works nicely.
Multiple r/w too as it seems.

So this boils down to me not completely grasping Python stdio.
(Script below. Even with python -u its not what I'd expect).
Okay, now i can get a step forward at testing the Pipe and Proc stream wrappers.

To the other comments, thanks, but the process is LONG running. Like I
explicitly wrote. I do not want to shut it down, but communicate with it
through stdio. Even though I have a Python wrapper that sets up a TCP
server around it, been looking to discard that..


Anyway hope i didn't spoil to much good HaXe time of the good people here.
I'm sure theres more important stuff to do than wrap an buggy 20yr old backend :+


Randy, much appreciated bearing with me.


kind regards, Berend


---

#!/usr/bin/env python
import sys

while 1:
    d = sys.stdin.read()
    if d:
        sys.stdout.write(d)
        sys.stdout.flush()
        d = None


>    Using 'cat' your second example
>    var proc = new Process("cat", []);
>    proc.stdin.writeString("data");
>    proc.stdout.read(4);
>    proc.close();
>
>    works for me.
>    Notice I changed 'stdin.write' to 'stdin.writeString' b/c compilation
>    failed with a type error.
>    If you trace the return from stdout.read(4) it will be 'data'.
>    So I'm wondering if the echo.py script is what's causing your neko
>    program to hang.
>    It might be waiting for a newline or EOF (when you've only written
>    "data" to it) before producing its first output (which the script is
>    waiting to read).
>
>    On Wed, Mar 17, 2010 at 1:02 PM, Berend van Berkum
>    <[1][hidden email]> wrote:
>
>      If Neko writes to stdin, and then reads stdout, the echo script
>      somehow never
>      finishes its own read.
>      E.g.
>
>    �  �  �  � proc = new Process("echo.py", []);
>
>      �  �  �  � proc.stdin.write("data");
>      �  �  �  � proc.close();
>      echo.py receives the data as expected. However:
>
>    �  �  �  � proc = new Process("echo.py", []);
>
>      �  �  �  � proc.stdin.write("data");
>      �  �  �  � proc.stdout.read(4);
>      �  �  �  � proc.close();
>      echo.py does not receive the data at its stdin, and the given
>      script hangs on the stdout.read, as I told before.
>      Writing EOT, LF, or using readLine: no effect.
>      I think I'll conclude here that std streams are unsuitable for
>      communicating
>      with running processes. (The socket variant works fine).
>      regards, Berend
>
>    On Tue, Mar 16, 2010 at 11:08:12PM -0700, Randy Stauner wrote:
>
>    > �  � i don't entirely understand what you're trying to figure out,
>    > �  � but in your third example, the readAll is probably wating for an
>    EOF.
>    > �  � You could:
>
>      > �  � � � * Add 'proc.stdin.close()' before you start reading...
>      then readAll
>
>    > �  � will return and the program will finish
>
>      > �  � � � * Or you could add a newline
>      'proc.stdin.writeString("data\n");' and
>
>    > �  � then use 'proc.stdout.readLine()' instead of readAll(). This too
>    will
>    > �  � allow the data to print and the program to finish.
>    > �  � Maybe this will help?
>    >
>    > �  � On Tue, Mar 16, 2010 at 9:15 AM, Berend van Berkum
>
>    > �  � <[1][2][hidden email]> wrote:
>    >
>    > �  �  � hi all,
>    > �  �  � i'm at a complete loss after trying for several hours,
>    > �  �  � perhaps someone here can write a few words on this..
>    > �  �  � First, i was trying to get a write handle on a fifo.
>    > �  �  � But at every attempt the script locks at the line:
>
>      > �  �  � � � � � � � � File.write(fifo, true);
>
>    > �  �  � Where the fifo is used in something like:
>
>    > �  �  � � � � � � � � var fifo = "fin";
>    > �  �  � � � � � � � � Sys.command("mkfifo", [fifo]);
>
>      > �  �  � � � � � � � � var proc = new Process("sh", ["-c",
>      "backend", "<",
>
>    > �  �  � fifo]);
>    > �  �  � Then, to test some things first:
>
>    > �  �  � � � � � � � � var proc = new Process("echo.py", []);
>    > �  �  � � � � � � � � proc.stdin.writeString("data1");
>    > �  �  � � � � � � � � proc.stdin.flush();
>    > �  �  � � � � � � � � trace(proc.stdout.readByte());
>    > �  �  � � � � � � � � trace(proc.stdout.readAll());
>
>      > �  �  � � � � � � � � proc.close();
>
>    > �  �  � Where echo.py will do exactly as the name suggests, and also
>    flush
>    > �  �  � its stdout.
>    > �  �  � But here any read operation on stdout will block.
>    > �  �  � This is on linux, reading output from e.g. 'ls' works fine.
>    > �  �  � PHP or Neko, no difference.
>    > �  �  � I think I saw somewhere on the lists that working with
>    stdin/stdout
>    > �  �  � synchronously won't work yet. But that may be quite old, and
>    im not
>    > �  �  � sure if it
>    > �  �  � applies. This can be worked around by using a socket
>    connection to
>    > �  �  � an wrapper
>    > �  �  � around "backend", but alas, hoped this could work..
>    > �  �  � regards, Berend
>    > �  �  � --
>
>    > �  �  � � E: [2][3][hidden email]
>    > �  �  � � T: +31(0)6 - 19160770
>    > �  �  � � W: [3][4]http://services.dotmpe.com/
>
>      > �  �  � � � � [4][5]http://www.linkedin.com/pub/13/261/286
>
>    > �  �  � --
>    > �  �  � haXe - an open source web programming language
>
>    > �  �  � [5][6]http://haxe.org
>    >
>    > References
>    >
>    > �  � 1. mailto:[7][hidden email]
>    > �  � 2. mailto:[8][hidden email]
>    > �  � 3. [9]http://services.dotmpe.com/
>
>      > �  � 4. [10]http://www.linkedin.com/pub/13/261/286
>      > �  � 5. [11]http://haxe.org/
>      > --
>
>    > haXe - an open source web programming language
>    > [12]http://haxe.org
>
>      --
>
>    � E: [13][hidden email]
>    � T: +31(0)6 - 19160770
>    � W: [14]http://services.dotmpe.com/
>    �  � [15]http://www.linkedin.com/pub/13/261/286
>    --
>    haXe - an open source web programming language
>    [16]http://haxe.org
>
> References
>
>    1. mailto:[hidden email]
>    2. mailto:[hidden email]
>    3. mailto:[hidden email]
>    4. http://services.dotmpe.com/
>    5. http://www.linkedin.com/pub/13/261/286
>    6. http://haxe.org/
>    7. mailto:[hidden email]
>    8. mailto:[hidden email]
>    9. http://services.dotmpe.com/
>   10. http://www.linkedin.com/pub/13/261/286
>   11. http://haxe.org/
>   12. http://haxe.org/
>   13. mailto:[hidden email]
>   14. http://services.dotmpe.com/
>   15. http://www.linkedin.com/pub/13/261/286
>   16. http://haxe.org/

> --
> haXe - an open source web programming language
> http://haxe.org

--
 E: [hidden email]
 T: +31(0)6 - 19160770
 W: http://services.dotmpe.com/
    http://www.linkedin.com/pub/13/261/286


--
haXe - an open source web programming language
http://haxe.org