haxe/neko issues

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

haxe/neko issues

blackdog-2

Two things I think should be checked for the next release of haxe.

1. The addition of a haxe.Timer for neko, I found the attached on
the list and tweaked it, thanks to however did the first can't remember.

2. Sqlite hangs due to startTransaction being issued on lines 73/78
of Sqlite.hx. Removal of those lines and it functions fine. I note
they're there to match mysql but I believe it's wrong in this instance
as the sqlite default is autocommit.

bd



--
'Perfection is achieved not when there is nothing more to add, but
rather when there is nothing more to take away.'

-- Antoine de Saint-Exupéry

--
haXe - an open source web programming language
http://haxe.org

Timer.hx (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Thomas-8
On Sun, Sep 20, 2009 at 7:05 PM, black dog <[hidden email]> wrote:
>
> Two things I think should be checked for the next release of haxe.
>
> 1. The addition of a haxe.Timer for neko, I found the attached on
> the list and tweaked it, thanks to however did the first can't remember.
neko.Sys.sleep?

> 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> of Sqlite.hx.
I have no problem with sqlite at all, I use SPOD though.

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Don-Duong Quach
That Timer does look useful!

Btw, neko.Sys.sleep just affects the current thread, while the Timer class is spawning another thread.

Don Q.

On Sun, Sep 20, 2009 at 10:12 AM, Thomas <[hidden email]> wrote:
On Sun, Sep 20, 2009 at 7:05 PM, black dog <[hidden email]> wrote:
>
> Two things I think should be checked for the next release of haxe.
>
> 1. The addition of a haxe.Timer for neko, I found the attached on
> the list and tweaked it, thanks to however did the first can't remember.
neko.Sys.sleep?

> 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> of Sqlite.hx.
I have no problem with sqlite at all, I use SPOD though.

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

blackdog-2
In reply to this post by Thomas-8

Sqlite has a default of auto transaction, per statement, this works
fine, if however you want to use it by doing something with multiple
statements it hangs

static function
doTransaction(account:Account,by:User,
value:Float,newBalance:Float, details:Dynamic,trnType:String) {

                var entry = new Ledger(),
                                entryDate = Date.now().toString();

                begin();

                entry.accountID = account.id;
                entry.addedBy = by.id;
                entry.tvalue = value;
                entry.newBalance = newBalance;
                entry.summary = JSON.encode(details);
                entry.transType = trnType;
                entry.addedOn = entryDate;
                entry.insert();

                account.balance = newBalance;
                account.lastTransOn = entryDate;
                account.update();

                commit();


                return lastInsertId();
        }

unless the statements I've indicated are removed from sqlite.hx.

where begin() and commit() are ...

public static inline
        function begin() {
                if (inTrans++ == 0)
                        neko.db.Manager.cnx.startTransaction();
        }

        public static inline
        function commit() {
                if (--inTrans == 0) {
                        neko.db.Manager.cnx.commit();
                }
        }

> > 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> > of Sqlite.hx.
> I have no problem with sqlite at all, I use SPOD though.
>



--
'Perfection is achieved not when there is nothing more to add, but
rather when there is nothing more to take away.'

-- Antoine de Saint-Exupéry

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Nicolas Cannasse
In reply to this post by blackdog-2
black dog a écrit :
> Two things I think should be checked for the next release of haxe.
>
> 1. The addition of a haxe.Timer for neko, I found the attached on
> the list and tweaked it, thanks to however did the first can't remember.

Since Neko is not event-based, there is no other way than using threads
for timer. However, for people that don't know the issues that might
occur with threads, relying on haxe.Timer might create a lot of
bugs/crashes. I prefer to leave the class unimplemented in neko for this
reason.

> 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> of Sqlite.hx. Removal of those lines and it functions fine. I note
> they're there to match mysql but I believe it's wrong in this instance
> as the sqlite default is autocommit.

Could you detail why it hangs ?
The idea is that after each commit/rollback, a new transaction is created.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

blackdog-2


On Thu, 2009-12-24 at 16:34 +0100, Nicolas Cannasse wrote:

> black dog a écrit :
> > Two things I think should be checked for the next release of haxe.
> >
> > 1. The addition of a haxe.Timer for neko, I found the attached on
> > the list and tweaked it, thanks to however did the first can't remember.
>
> Since Neko is not event-based, there is no other way than using threads
> for timer. However, for people that don't know the issues that might
> occur with threads, relying on haxe.Timer might create a lot of
> bugs/crashes. I prefer to leave the class unimplemented in neko for this
> reason.
>

That's worrisome, i have a product based on a thread based neko timer :/
Luckily my node.js haxe wrappers can be used alternatively. For me, not
having a reliable timer is a nail in the coffin for neko.

> > 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> > of Sqlite.hx. Removal of those lines and it functions fine. I note
> > they're there to match mysql but I believe it's wrong in this instance
> > as the sqlite default is autocommit.
>

Basically, I've found that beginning a transaction and then doing
multiple inserts etc, hangs sqlite. When I get some time after xmas i'll
post some code.

> Could you detail why it hangs ?
> The idea is that after each commit/rollback, a new transaction is created.
>
> Best,
> Nicolas
>


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Nicolas Cannasse
blackdog a écrit :

>
> On Thu, 2009-12-24 at 16:34 +0100, Nicolas Cannasse wrote:
>> black dog a écrit :
>>> Two things I think should be checked for the next release of haxe.
>>>
>>> 1. The addition of a haxe.Timer for neko, I found the attached on
>>> the list and tweaked it, thanks to however did the first can't remember.
>> Since Neko is not event-based, there is no other way than using threads
>> for timer. However, for people that don't know the issues that might
>> occur with threads, relying on haxe.Timer might create a lot of
>> bugs/crashes. I prefer to leave the class unimplemented in neko for this
>> reason.
>>
>
> That's worrisome, i have a product based on a thread based neko timer :/
> Luckily my node.js haxe wrappers can be used alternatively. For me, not
> having a reliable timer is a nail in the coffin for neko.

You can have a reliable timer but it needs a different API :

For instance, one thread which measure time and dispatch timer events,
and the main thread which wait for these events in an event loop and run
the appropriate handlers (all in the same thread). You can use a
neko.vm.Deque for this.

Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Gamehaxe
In reply to this post by blackdog-2
Hi black dog,
I think the main problem is there is no "mainLoop" in neko.
One issue is, what is the thread context in which your timer
callbacks run?  Lots of things require that particular context
is associated with a particular thread (eg, thread you open your
db connection with, and the thread you make the query with should be the  
same).
If you are doing your own threads, then you are probably having
a control thread waiting for workers to complete, so it is
going to be in some kind of blocking wait routine.
If you use the "neko.vm.Lock" class, you can specify a timeout,
which can be use to emulate your timer, with 1 thread waiting for
the minimum time of all the current timers.
Does that make sense?  I guess what I'm saying is that you can
use the existing ndll-s and make some library code to do
exactly what you want.

So it would be something like this:

import neko.vm.Mutex;
import neko.vm.Lock;

class TimerThread
{
var mMutex:Mutex;
var mQueueLock:Lock;
var mQueue : Array<Dynamic>;
var mAlive : Bool;

public function new()
{
   mQueue = [];
   mQueueLock = new Lock();
   mMutex = new Mutex();
   mAlive = true;
   var me = this;
   neko.vm.Thread.create( me.mainLoop );
}

public function addTimer(inDelay:Float, inCallback:Void -> Void)
{
    mMutex.acquire();
    mQueue.push( { time:haxe.Timer.stamp()+inDelay, func:inCallback} );
    mMutex.release();
    // Make the main-loop recalculate the wake-up time...
    mQueueLock.release();
}

public function quit(?inWhenDone:Void->Void)
{
    var me = this;
    addTimer( 0, function() { me.mAlive = false; if (inWhenDone!=null)  
inWhenDone(); } );
}


function mainLoop()
{
    while(mAlive)
    {
       var wake:Null<Float> = null;
       var now = haxe.Timer.stamp();
       var to_run = new Array<Dynamic>();
       var not_yet = new Array<Dynamic>();

       mMutex.acquire();
       for(m in mQueue)
       {
          if (m.time<=now)
             to_run.push(m.func);
          else
          {
             not_yet.push(m);
             if (wake==null || wake>m.time)
                wake = m.time;
          }
       }
       mQueue = not_yet;
       mMutex.release();

       for(func in to_run)
       {
          func();
          if (!mAlive)
             break;
       }

       if (!mAlive)
          break;

       if (wake==null)
          mQueueLock.wait();
       else
       {
          var delay = wake - haxe.Timer.stamp();
          //trace("Delay : " + delay);
          if (delay>0)
             mQueueLock.wait(delay);
       }
    }
}


}



class Timer
{
public static function main()
{
    var timer = new TimerThread();

    for(i in 0...10)
       timer.addTimer( 10-i, function() { trace("Callback " + i); } );
    var t = neko.vm.Thread.current();
    timer.addTimer(12, function() { timer.quit( function()  
t.sendMessage("All done") ); } );

    var msg = neko.vm.Thread.readMessage(true);
    trace(msg);
}

}


You could optimise the code to not "release" (ie, wakeup) the waiting
thread if it knows the timer in later than the current wait timer.

Hugh



>
>
> On Thu, 2009-12-24 at 16:34 +0100, Nicolas Cannasse wrote:
>> black dog a écrit :
>> > Two things I think should be checked for the next release of haxe.
>> >
>> > 1. The addition of a haxe.Timer for neko, I found the attached on
>> > the list and tweaked it, thanks to however did the first can't  
>> remember.
>>
>> Since Neko is not event-based, there is no other way than using threads
>> for timer. However, for people that don't know the issues that might
>> occur with threads, relying on haxe.Timer might create a lot of
>> bugs/crashes. I prefer to leave the class unimplemented in neko for this
>> reason.
>>
>
> That's worrisome, i have a product based on a thread based neko timer :/
> Luckily my node.js haxe wrappers can be used alternatively. For me, not
> having a reliable timer is a nail in the coffin for neko.
>
>> > 2. Sqlite hangs due to startTransaction being issued on lines 73/78
>> > of Sqlite.hx. Removal of those lines and it functions fine. I note
>> > they're there to match mysql but I believe it's wrong in this instance
>> > as the sqlite default is autocommit.
>>
>
> Basically, I've found that beginning a transaction and then doing
> multiple inserts etc, hangs sqlite. When I get some time after xmas i'll
> post some code.
>
>> Could you detail why it hangs ?
>> The idea is that after each commit/rollback, a new transaction is  
>> created.
>>
>> Best,
>> Nicolas
>>
>
>


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Randy Stauner
just a suggestion, but is this example code the kind of thing that deserves a page on the wiki?

On Fri, Dec 25, 2009 at 8:37 PM, Hugh Sanderson <[hidden email]> wrote:
Hi black dog,
I think the main problem is there is no "mainLoop" in neko.
One issue is, what is the thread context in which your timer
callbacks run?  Lots of things require that particular context
is associated with a particular thread (eg, thread you open your
db connection with, and the thread you make the query with should be the same).
If you are doing your own threads, then you are probably having
a control thread waiting for workers to complete, so it is
going to be in some kind of blocking wait routine.
If you use the "neko.vm.Lock" class, you can specify a timeout,
which can be use to emulate your timer, with 1 thread waiting for
the minimum time of all the current timers.
Does that make sense?  I guess what I'm saying is that you can
use the existing ndll-s and make some library code to do
exactly what you want.

So it would be something like this:

import neko.vm.Mutex;
import neko.vm.Lock;

class TimerThread
{
var mMutex:Mutex;
var mQueueLock:Lock;
var mQueue : Array<Dynamic>;
var mAlive : Bool;

public function new()
{
 mQueue = [];
 mQueueLock = new Lock();
 mMutex = new Mutex();
 mAlive = true;
 var me = this;
 neko.vm.Thread.create( me.mainLoop );
}

public function addTimer(inDelay:Float, inCallback:Void -> Void)
{
  mMutex.acquire();
  mQueue.push( { time:haxe.Timer.stamp()+inDelay, func:inCallback} );
  mMutex.release();
  // Make the main-loop recalculate the wake-up time...
  mQueueLock.release();
}

public function quit(?inWhenDone:Void->Void)
{
  var me = this;
  addTimer( 0, function() { me.mAlive = false; if (inWhenDone!=null) inWhenDone(); } );
}


function mainLoop()
{
  while(mAlive)
  {
     var wake:Null<Float> = null;
     var now = haxe.Timer.stamp();
     var to_run = new Array<Dynamic>();
     var not_yet = new Array<Dynamic>();

     mMutex.acquire();
     for(m in mQueue)
     {
        if (m.time<=now)
           to_run.push(m.func);
        else
        {
           not_yet.push(m);
           if (wake==null || wake>m.time)
              wake = m.time;
        }
     }
     mQueue = not_yet;
     mMutex.release();

     for(func in to_run)
     {
        func();
        if (!mAlive)
           break;
     }

     if (!mAlive)
        break;

     if (wake==null)
        mQueueLock.wait();
     else
     {
        var delay = wake - haxe.Timer.stamp();
        //trace("Delay : " + delay);
        if (delay>0)
           mQueueLock.wait(delay);
     }
  }
}


}



class Timer
{
public static function main()
{
  var timer = new TimerThread();

  for(i in 0...10)
     timer.addTimer( 10-i, function() { trace("Callback " + i); } );
  var t = neko.vm.Thread.current();
  timer.addTimer(12, function() { timer.quit( function() t.sendMessage("All done") ); } );

  var msg = neko.vm.Thread.readMessage(true);
  trace(msg);
}

}


You could optimise the code to not "release" (ie, wakeup) the waiting
thread if it knows the timer in later than the current wait timer.

Hugh






On Thu, 2009-12-24 at 16:34 +0100, Nicolas Cannasse wrote:
black dog a écrit :
> Two things I think should be checked for the next release of haxe.
>
> 1. The addition of a haxe.Timer for neko, I found the attached on
> the list and tweaked it, thanks to however did the first can't remember.

Since Neko is not event-based, there is no other way than using threads
for timer. However, for people that don't know the issues that might
occur with threads, relying on haxe.Timer might create a lot of
bugs/crashes. I prefer to leave the class unimplemented in neko for this
reason.


That's worrisome, i have a product based on a thread based neko timer :/
Luckily my node.js haxe wrappers can be used alternatively. For me, not
having a reliable timer is a nail in the coffin for neko.

> 2. Sqlite hangs due to startTransaction being issued on lines 73/78
> of Sqlite.hx. Removal of those lines and it functions fine. I note
> they're there to match mysql but I believe it's wrong in this instance
> as the sqlite default is autocommit.


Basically, I've found that beginning a transaction and then doing
multiple inserts etc, hangs sqlite. When I get some time after xmas i'll
post some code.

Could you detail why it hangs ?
The idea is that after each commit/rollback, a new transaction is created.

Best,
Nicolas





--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: haxe/neko issues

Ron Wheeler
With a little cleaning of the discussion and an introductory paragraph,
this would make a nice tutorial.

Ron

Randy Stauner wrote:

> just a suggestion, but is this example code the kind of thing that
> deserves a page on the wiki?
>
> On Fri, Dec 25, 2009 at 8:37 PM, Hugh Sanderson <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi black dog,
>     I think the main problem is there is no "mainLoop" in neko.
>     One issue is, what is the thread context in which your timer
>     callbacks run?  Lots of things require that particular context
>     is associated with a particular thread (eg, thread you open your
>     db connection with, and the thread you make the query with should
>     be the same).
>     If you are doing your own threads, then you are probably having
>     a control thread waiting for workers to complete, so it is
>     going to be in some kind of blocking wait routine.
>     If you use the "neko.vm.Lock" class, you can specify a timeout,
>     which can be use to emulate your timer, with 1 thread waiting for
>     the minimum time of all the current timers.
>     Does that make sense?  I guess what I'm saying is that you can
>     use the existing ndll-s and make some library code to do
>     exactly what you want.
>
>     So it would be something like this:
>
>     import neko.vm.Mutex;
>     import neko.vm.Lock;
>
>     class TimerThread
>     {
>     var mMutex:Mutex;
>     var mQueueLock:Lock;
>     var mQueue : Array<Dynamic>;
>     var mAlive : Bool;
>
>     public function new()
>     {
>      mQueue = [];
>      mQueueLock = new Lock();
>      mMutex = new Mutex();
>      mAlive = true;
>      var me = this;
>      neko.vm.Thread.create( me.mainLoop );
>     }
>
>     public function addTimer(inDelay:Float, inCallback:Void -> Void)
>     {
>       mMutex.acquire();
>       mQueue.push( { time:haxe.Timer.stamp()+inDelay, func:inCallback} );
>       mMutex.release();
>       // Make the main-loop recalculate the wake-up time...
>       mQueueLock.release();
>     }
>
>     public function quit(?inWhenDone:Void->Void)
>     {
>       var me = this;
>       addTimer( 0, function() { me.mAlive = false; if
>     (inWhenDone!=null) inWhenDone(); } );
>     }
>
>
>     function mainLoop()
>     {
>       while(mAlive)
>       {
>          var wake:Null<Float> = null;
>          var now = haxe.Timer.stamp();
>          var to_run = new Array<Dynamic>();
>          var not_yet = new Array<Dynamic>();
>
>          mMutex.acquire();
>          for(m in mQueue)
>          {
>             if (m.time<=now)
>                to_run.push(m.func);
>             else
>             {
>                not_yet.push(m);
>                if (wake==null || wake>m.time)
>                   wake = m.time;
>             }
>          }
>          mQueue = not_yet;
>          mMutex.release();
>
>          for(func in to_run)
>          {
>             func();
>             if (!mAlive)
>                break;
>          }
>
>          if (!mAlive)
>             break;
>
>          if (wake==null)
>             mQueueLock.wait();
>          else
>          {
>             var delay = wake - haxe.Timer.stamp();
>             //trace("Delay : " + delay);
>             if (delay>0)
>                mQueueLock.wait(delay);
>          }
>       }
>     }
>
>
>     }
>
>
>
>     class Timer
>     {
>     public static function main()
>     {
>       var timer = new TimerThread();
>
>       for(i in 0...10)
>          timer.addTimer( 10-i, function() { trace("Callback " + i); } );
>       var t = neko.vm.Thread.current();
>       timer.addTimer(12, function() { timer.quit( function()
>     t.sendMessage("All done") ); } );
>
>       var msg = neko.vm.Thread.readMessage(true);
>       trace(msg);
>     }
>
>     }
>
>
>     You could optimise the code to not "release" (ie, wakeup) the waiting
>     thread if it knows the timer in later than the current wait timer.
>
>     Hugh
>
>
>
>
>
>
>         On Thu, 2009-12-24 at 16:34 +0100, Nicolas Cannasse wrote:
>
>             black dog a écrit :
>             > Two things I think should be checked for the next
>             release of haxe.
>             >
>             > 1. The addition of a haxe.Timer for neko, I found the
>             attached on
>             > the list and tweaked it, thanks to however did the first
>             can't remember.
>
>             Since Neko is not event-based, there is no other way than
>             using threads
>             for timer. However, for people that don't know the issues
>             that might
>             occur with threads, relying on haxe.Timer might create a
>             lot of
>             bugs/crashes. I prefer to leave the class unimplemented in
>             neko for this
>             reason.
>
>
>         That's worrisome, i have a product based on a thread based
>         neko timer :/
>         Luckily my node.js haxe wrappers can be used alternatively.
>         For me, not
>         having a reliable timer is a nail in the coffin for neko.
>
>             > 2. Sqlite hangs due to startTransaction being issued on
>             lines 73/78
>             > of Sqlite.hx. Removal of those lines and it functions
>             fine. I note
>             > they're there to match mysql but I believe it's wrong in
>             this instance
>             > as the sqlite default is autocommit.
>
>
>         Basically, I've found that beginning a transaction and then doing
>         multiple inserts etc, hangs sqlite. When I get some time after
>         xmas i'll
>         post some code.
>
>             Could you detail why it hangs ?
>             The idea is that after each commit/rollback, a new
>             transaction is created.
>
>             Best,
>             Nicolas
>
>
>
>
>
>     --
>     haXe - an open source web programming language
>     http://haxe.org
>
>


--
haXe - an open source web programming language
http://haxe.org