What's the status of Memory class in nme?

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

What's the status of Memory class in nme?

Alan Shaw-4
I've updated to the new releases of hxcpp and nme, and my program hasn't been crashing any more. Great!
Now I thought I'd do some optimization using flash.Memory for my presentation next week.

I notice nme does have the Memory class; but also in nme I cannot set the length of a ByteArray as I can in Flash.

Does nme even need the Memory class? Why is it there? I'd think the cpp target would use "fast" memory by default anyway!
I expect that I should conditionally compile the Memory stuff just for the Flash target...

-A

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Tarwin Stroh-Spijer
My understanding of why you want this fast Memory access is so you can use simply blobs of memory, as opposed to Ints or Floats or whatever.

For example, if you want to keep an RGB image in memory for manipulation, in CPP you could do something like a Struct which has an R, G and B of 8 bits each. In haxe you'd have to do something like having, at simplest, an Array of Ints, which you'd have to bitshift to get separate values from, or have 1 value for each which would be wasteful, other than them being 32 rather than 24 bits anyway.

So, here comes Memory! Or that's my understanding anyway. Probably good for things like sound manipulation as well?

It might even be cool to implement some kind of struct accessor now as well do you think? An object type that is kept in a given "fast memory" pool for example. Like StructPoint which might have an x and a y property which were Floats, but simply kept packed in fast memory? Anyway, don't really know what I'm talking about here - good luck!


Tarwin Stroh-Spijer
_______________________

Touch My Pixel
http://www.touchmypixel.com/
phone: +61 3 8060 5321
_______________________


On Tue, Aug 30, 2011 at 3:46 PM, Alan Shaw <[hidden email]> wrote:
I notice nme does have the Memory class; but also in nme I cannot set the length of a ByteArray as I can in Flash.


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Julien CASTETS
According to his tweets, you could use nme.Memory http://www.twaitter.com/gamehaxe.aspx
I remember Hugh talking about playing with fast Memory access

Hope this helps
Best
Julien

2011/8/30 Tarwin Stroh-Spijer <[hidden email]>
My understanding of why you want this fast Memory access is so you can use simply blobs of memory, as opposed to Ints or Floats or whatever.

For example, if you want to keep an RGB image in memory for manipulation, in CPP you could do something like a Struct which has an R, G and B of 8 bits each. In haxe you'd have to do something like having, at simplest, an Array of Ints, which you'd have to bitshift to get separate values from, or have 1 value for each which would be wasteful, other than them being 32 rather than 24 bits anyway.

So, here comes Memory! Or that's my understanding anyway. Probably good for things like sound manipulation as well?

It might even be cool to implement some kind of struct accessor now as well do you think? An object type that is kept in a given "fast memory" pool for example. Like StructPoint which might have an x and a y property which were Floats, but simply kept packed in fast memory? Anyway, don't really know what I'm talking about here - good luck!


Tarwin Stroh-Spijer
_______________________

Touch My Pixel
http://www.touchmypixel.com/
phone: <a href="tel:%2B61%203%208060%205321" value="+61380605321" target="_blank">+61 3 8060 5321
_______________________



On Tue, Aug 30, 2011 at 3:46 PM, Alan Shaw <[hidden email]> wrote:
I notice nme does have the Memory class; but also in nme I cannot set the length of a ByteArray as I can in Flash.


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Matthew Spencer-2
In reply to this post by Tarwin Stroh-Spijer
My understanding of why you want this fast Memory access is so you can use simply blobs of memory, as opposed to Ints or Floats or whatever.
Yes this is one of the reasons. The other huge reason is it gives you the ability to write cache friendly code, since you control the exact memory layout. Not to mention, you can write much faster allocation routines.

--Matthew Spencer

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Yanis Benson
In reply to this post by Alan Shaw-4

Just so you know, according to my data tje way this guy usus memory slows things down in flash. Using classes is fster for particles.

On 30 Aug 2011 09:48, "Alan Shaw" <[hidden email]> wrote:
> I've updated to the new releases of hxcpp and nme, and my program hasn't
> been crashing any more. Great!
> Now I thought I'd do some optimization using flash.Memory for my
> presentation next week.
> I'm trying to follow webr3's example at
> http://webr3.org/experiments/perlin-particles/light-cloud/PerlinParticleEffects.hx
>
> I notice nme does have the Memory class; but also in nme I cannot set the
> length of a ByteArray as I can in Flash.
>
> Does nme even need the Memory class? Why is it there? I'd think the cpp
> target would use "fast" memory by default anyway!
> I expect that I should conditionally compile the Memory stuff just for the
> Flash target...
>
> -A

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Matthew Spencer-2
I'd love to see the proof of these claims. I've worked with fast mem extensively, and never seen anything outperform it.
--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Gamehaxe
In reply to this post by Alan Shaw-4
Hi,

The class is there to mirror the flash API.
In cpp, an array of doubles is very fast - faster still if you use the  
"unsafe" get/set methods.

For length, you need to use "setLength" because I can't change the
read-onlyness of the "length" member, so
#if nme
   bytes.setLength(100);
#else
   bytes.length = 100;
#end

You may like to make a function for yourself, or "using" a setLength  
member into the flash class.


Hugh


> I've updated to the new releases of hxcpp and nme, and my program hasn't
> been crashing any more. Great!
> Now I thought I'd do some optimization using flash.Memory for my
> presentation next week.
> I'm trying to follow webr3's example at
> http://webr3.org/experiments/perlin-particles/light-cloud/PerlinParticleEffects.hx
>
> I notice nme does have the Memory class; but also in nme I cannot set the
> length of a ByteArray as I can in Flash.
>
> Does nme even need the Memory class? Why is it there? I'd think the cpp
> target would use "fast" memory by default anyway!
> I expect that I should conditionally compile the Memory stuff just for  
> the
> Flash target...
>
> -A

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Alan Shaw-4
Ah, thanks, Hugh!

-A


On 8/30/11, Gamehaxe <[hidden email]> wrote:

> Hi,
>
> The class is there to mirror the flash API.
> In cpp, an array of doubles is very fast - faster still if you use the
> "unsafe" get/set methods.
>
> For length, you need to use "setLength" because I can't change the
> read-onlyness of the "length" member, so
> #if nme
>    bytes.setLength(100);
> #else
>    bytes.length = 100;
> #end
>
> You may like to make a function for yourself, or "using" a setLength
> member into the flash class.
>
>
> Hugh
>
>
>> I've updated to the new releases of hxcpp and nme, and my program hasn't
>> been crashing any more. Great!
>> Now I thought I'd do some optimization using flash.Memory for my
>> presentation next week.
>> I'm trying to follow webr3's example at
>> http://webr3.org/experiments/perlin-particles/light-cloud/PerlinParticleEffects.hx
>>
>> I notice nme does have the Memory class; but also in nme I cannot set the
>> length of a ByteArray as I can in Flash.
>>
>> Does nme even need the Memory class? Why is it there? I'd think the cpp
>> target would use "fast" memory by default anyway!
>> I expect that I should conditionally compile the Memory stuff just for
>> the
>> Flash target...
>>
>> -A
>
> --
> haXe - an open source web programming language
> http://haxe.org
>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Alan Shaw-4
BTW the "unsafe" get/set methods? I cd use a little guidance on that.
What exactly do you menan? A few lines of code wd be appreciated !)

-A


On 8/30/11, Alan Shaw <[hidden email]> wrote:

> Ah, thanks, Hugh!
>
> -A
>
>
> On 8/30/11, Gamehaxe <[hidden email]> wrote:
>> Hi,
>>
>> The class is there to mirror the flash API.
>> In cpp, an array of doubles is very fast - faster still if you use the
>> "unsafe" get/set methods.
>>
>> For length, you need to use "setLength" because I can't change the
>> read-onlyness of the "length" member, so
>> #if nme
>>    bytes.setLength(100);
>> #else
>>    bytes.length = 100;
>> #end
>>
>> You may like to make a function for yourself, or "using" a setLength
>> member into the flash class.
>>
>>
>> Hugh
>>
>>
>>> I've updated to the new releases of hxcpp and nme, and my program hasn't
>>> been crashing any more. Great!
>>> Now I thought I'd do some optimization using flash.Memory for my
>>> presentation next week.
>>> I'm trying to follow webr3's example at
>>> http://webr3.org/experiments/perlin-particles/light-cloud/PerlinParticleEffects.hx
>>>
>>> I notice nme does have the Memory class; but also in nme I cannot set
>>> the
>>> length of a ByteArray as I can in Flash.
>>>
>>> Does nme even need the Memory class? Why is it there? I'd think the cpp
>>> target would use "fast" memory by default anyway!
>>> I expect that I should conditionally compile the Memory stuff just for
>>> the
>>> Flash target...
>>>
>>> -A
>>
>> --
>> haXe - an open source web programming language
>> http://haxe.org
>>
>

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Michael Baczynski-2
On 30.08.2011 14:38, Alan Shaw wrote:
> BTW the "unsafe" get/set methods? I cd use a little guidance on that.
> What exactly do you menan? A few lines of code wd be appreciated !)

afaik what Hugh means is that there is no boundary checking when accessing elements, so you have to
be extra careful not to read/write outside the defined range of the byte array or you app will explode.

best,
michael

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Gamehaxe
Hi,
Yes, no bounds checking is done, eg:

class Test
{
   public static function main()
   {
      var d = new Array<Float>();
      // Allocate 100 elements ...
      d[99] = 0;
      for(i in 0...100)
         untyped d.__unsafe_set(i,i);
      var sum = 0.0;
      for(i in 0...100)
         sum += untyped d.__unsafe_get(i);
      trace(sum);
   }
}

Also fast for other arrays, eg Int.  But you would also need to ask
yourself if this extra speed is worth the danger & syntax problems.

HUgh

> On 30.08.2011 14:38, Alan Shaw wrote:
>> BTW the "unsafe" get/set methods? I cd use a little guidance on that.
>> What exactly do you menan? A few lines of code wd be appreciated !)
>
> afaik what Hugh means is that there is no boundary checking when  
> accessing elements, so you have to be extra careful not to read/write  
> outside the defined range of the byte array or you app will explode.
>
> best,
> michael

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Alan Shaw-4

I'm asking myself, and I'm replying "Do it!"

So, are you saying, for greatest speed, for the Flash target, use Memory,
but for cpp, use code like below, without Memory? (btw I used to do C and I remember about malloc etc)

My app is all about BitmapData. I do operations on them to produce new ones.
Many are done on a per-pixel basis. I'm using Michael's RGB class, sometimes
Int, usually Float, to extract rgb, perform the operation on each, then rebuild the rgb(a) Int.
Want to show that it does it fast!

Presenting at FlashONGames next week: "Level Up with haXe!"
and the revolution WILL be televised...

On Tue, Aug 30, 2011 at 8:00 AM, Gamehaxe <[hidden email]> wrote:
Hi,
Yes, no bounds checking is done, eg:

class Test
{
 public static function main()
 {
    var d = new Array<Float>();
    // Allocate 100 elements ...
    d[99] = 0;
    for(i in 0...100)
       untyped d.__unsafe_set(i,i);
    var sum = 0.0;
    for(i in 0...100)
       sum += untyped d.__unsafe_get(i);
    trace(sum);
 }
}

Also fast for other arrays, eg Int.  But you would also need to ask
yourself if this extra speed is worth the danger & syntax problems.

HUgh


On 30.08.2011 14:38, Alan Shaw wrote:
BTW the "unsafe" get/set methods? I cd use a little guidance on that.
What exactly do you menan? A few lines of code wd be appreciated !)

afaik what Hugh means is that there is no boundary checking when accessing elements, so you have to be extra careful not to read/write outside the defined range of the byte array or you app will explode.

best,
michael

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

bubblebenj
ho, another revolution ! :)

On Tue, Aug 30, 2011 at 7:51 PM, Alan Shaw <[hidden email]> wrote:

I'm asking myself, and I'm replying "Do it!"

So, are you saying, for greatest speed, for the Flash target, use Memory,
but for cpp, use code like below, without Memory? (btw I used to do C and I remember about malloc etc)

My app is all about BitmapData. I do operations on them to produce new ones.
Many are done on a per-pixel basis. I'm using Michael's RGB class, sometimes
Int, usually Float, to extract rgb, perform the operation on each, then rebuild the rgb(a) Int.
Want to show that it does it fast!

Presenting at FlashONGames next week: "Level Up with haXe!"
and the revolution WILL be televised...

On Tue, Aug 30, 2011 at 8:00 AM, Gamehaxe <[hidden email]> wrote:
Hi,
Yes, no bounds checking is done, eg:

class Test
{
 public static function main()
 {
    var d = new Array<Float>();
    // Allocate 100 elements ...
    d[99] = 0;
    for(i in 0...100)
       untyped d.__unsafe_set(i,i);
    var sum = 0.0;
    for(i in 0...100)
       sum += untyped d.__unsafe_get(i);
    trace(sum);
 }
}

Also fast for other arrays, eg Int.  But you would also need to ask
yourself if this extra speed is worth the danger & syntax problems.

HUgh


On 30.08.2011 14:38, Alan Shaw wrote:
BTW the "unsafe" get/set methods? I cd use a little guidance on that.
What exactly do you menan? A few lines of code wd be appreciated !)

afaik what Hugh means is that there is no boundary checking when accessing elements, so you have to be extra careful not to read/write outside the defined range of the byte array or you app will explode.

best,
michael

--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Yanis Benson
In reply to this post by Alan Shaw-4

Make a simple particle system, like position, speed and acceleration and update draw loop. From my expirience, of you use vector of particles it will usually be as fast as same particles in memory(could be a _little_ slower or a _little_ faster than memory). Try the same but using a list to store particles and it will be faster. Also, in such case you don't need to do all this direct memory accesses and pointer counting. Not that I think this stuff is any hard, but it doesn't make code clearer.

I did the fastest 3d particle renderer around and tried all the possible ways while doing it. My final version was made with memory, but it only worked out because of optimization on every operation, I was fighting for each int sum operation and looking at bytecode after every change(I half wrote avm2 dis/assembler while doing this, and read through half of tamarin sources.). Needless to say, the final code was so narrow targeted that it was only possible to use it for exactly that task and with a fixed number of particles.

However, it is very good to use memory for random access to single values(not structures). For example, the perlin noise Nicolass ported to haxe recently is twice faster with memory lookup table(I should also say that Nicolass did quite a good work on optimization, so except of big memory speedup, I was only able to squize out few more percents of speed.).

The trick is that value access is much faster than memory access anyhow. And when you have a simple xyz's for example you do one vector access+3 field accesses for a vector of structures vs 3 memory accesses for memory based aproach.

On 30 Aug 2011 14:23, "Matthew Spencer" <[hidden email]> wrote:
> I'd love to see the proof of these claims. I've worked with fast mem
> extensively, and never seen anything outperform it.

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Matthew Spencer-2

Make a simple particle system, like position, speed and acceleration and update draw loop. From my expirience, of you use vector of particles it will usually be as fast as same particles in memory(could be a _little_ slower or a _little_ faster than memory). Try the same but using a list to store particles and it will be faster. Also, in such case you don't need to do all this direct memory accesses and pointer counting. Not that I think this stuff is any hard, but it doesn't make code clearer.

I'll throw something together to test this, but last time I checked basic property access is slower than fastMem access (about half the speed, flash target). This is the primary reason there has been talk of a Structs<> library. I will definately look into this ASAP.

The trick is that value access is much faster than memory access anyhow. And when you have a simple xyz's for example you do one vector access+3 field accesses for a vector of structures vs 3 memory accesses for memory based aproach.

I tested this a long time ago, it's somewhat hard to get accurate results seeing that property set/access is tied to other opcodes (eg, a pop/dup/swap/etc..). In my results, flash.Memory was very very close to the performance of local registers/local properties, within 15%. Pretty hard to get accurate results, since it relies on other opcodes to load the necessary values though.


--Matthew Spencer

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Yanis Benson

Well, my results are a bit old(.5 year+). I'll recheck in a near time, maybe something has changed with last versions.

On 31 Aug 2011 00:14, "Matthew Spencer" <[hidden email]> wrote:
>>
>> Make a simple particle system, like position, speed and acceleration and
>> update draw loop. From my expirience, of you use vector of particles it will
>> usually be as fast as same particles in memory(could be a _little_ slower or
>> a _little_ faster than memory). Try the same but using a list to store
>> particles and it will be faster. Also, in such case you don't need to do all
>> this direct memory accesses and pointer counting. Not that I think this
>> stuff is any hard, but it doesn't make code clearer.
>>
> I'll throw something together to test this, but last time I checked basic
> property access is slower than fastMem access (about half the speed, flash
> target). This is the primary reason there has been talk of a Structs<>
> library. I will definately look into this ASAP.
>
> The trick is that value access is much faster than memory access anyhow. And
>> when you have a simple xyz's for example you do one vector access+3 field
>> accesses for a vector of structures vs 3 memory accesses for memory based
>> aproach.
>>
> I tested this a long time ago, it's somewhat hard to get accurate results
> seeing that property set/access is tied to other opcodes (eg, a
> pop/dup/swap/etc..). In my results, flash.Memory was very very close to the
> performance of local registers/local properties, within 15%. Pretty hard to
> get accurate results, since it relies on other opcodes to load the necessary
> values though.
>
>
> --Matthew Spencer

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Nicolas Cannasse
In reply to this post by Yanis Benson
Le 30/08/2011 21:55, Yanis Benson a écrit :
> Make a simple particle system, like position, speed and acceleration and
> update draw loop. From my expirience, of you use vector of particles it
> will usually be as fast as same particles in memory(could be a _little_
> slower or a _little_ faster than memory). Try the same but using a list
> to store particles and it will be faster. Also, in such case you don't
> need to do all this direct memory accesses and pointer counting. Not
> that I think this stuff is any hard, but it doesn't make code clearer.

In Flash, typed field access is the most fast but you need to have a
reference to the object, and not use Array/Vector.

For particles a haxe.FastList should be enough, or you can simply have a
"var next : Particle" in your Particle class to link them together.

As soon as you really need to index stuff or build some buffer (in
molehill for instance), flash.Memory is the best.

Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Matthew Spencer-2
In reply to this post by Yanis Benson

Well, my results are a bit old(.5 year+). I'll recheck in a near time, maybe something has changed with last versions.

My information is old too!  Alchemy ops are one of those things that seem to change with every release.  Taking speed differences out of the equation, alchemy ops are often times best avoided. Being able to specify only one buffer makes integrating multiple libraries together difficult. Of course, if your library works fairly standalone, you can always use flash.Memory on an external swf, and require your library be loaded as swf. This allows you multiple domainMems!

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Yanis Benson

For some complex manipulations it's frequently very cheap to save current memory in var, set yours memory, do what you want and set old memory back then. However, it isn't applicable when you need lots of singular accesses from all around the code.

On 31 Aug 2011 00:24, "Matthew Spencer" <[hidden email]> wrote:
>>
>> Well, my results are a bit old(.5 year+). I'll recheck in a near time,
>> maybe something has changed with last versions.
>>
> My information is old too! Alchemy ops are one of those things that seem to
> change with every release. Taking speed differences out of the equation,
> alchemy ops are often times best avoided. Being able to specify only one
> buffer makes integrating multiple libraries together difficult. Of course,
> if your library works fairly standalone, you can always use flash.Memory on
> an external swf, and require your library be loaded as swf. This allows you
> multiple domainMems!

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: What's the status of Memory class in nme?

Matthew Spencer-2

For some complex manipulations it's frequently very cheap to save current memory in var, set yours memory, do what you want and set old memory back then. However, it isn't applicable when you need lots of singular accesses from all around the code.

Depends on the size of the buffer. Most projects utilizing memory extensively will require 6+ms to swap buffers. 

--
haXe - an open source web programming language
http://haxe.org
12