macro problems

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

macro problems

Justin Donaldson-2
I have the macro examples working, and already I can see some neat opportunities for macro techniques.

I noticed that in several places it mentions that you can do file operations using the neko package in a macro.  However, I'm not having any luck doing something simple, like listing the current working directory:



#Demo.hx
import haxe.macro.Expr;
import haxe.macro.Context;

class Demo {   
    public static function main(){
        var x= CompileTime.cwd();
        trace(x);
    }
}

@:macro class CompileTime {
    public static function cwd() : Expr {
       var pos = haxe.macro.Context.currentPos();
       var loc = neko.Sys.getCwd();
       return {expr:EConst(CString(loc)), pos:pos}
    }
}


I get an error of:

/usr/local/haxe/std/neko/Sys.hx:64: characters 20-29 : Primitive not found std@get_cwd:0
src/Demo.hx:15: characters 14-31 : Called from
src/Demo.hx:6: characters 9-26 : Called from
Aborted

Other attempts at doing file i/o met with similar errors. 

Do I still have configuration issues, is the io module not working quite yet, or am I not understanding how to use the macros properly?
I thought I would post here before making an issue on google code.

Thanks!

-Justin



--
blog: http://www.scwn.net
aim: iujjd
twitter: jjdonald


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Nicolas Cannasse
Le 15/09/2010 02:54, Justin Donaldson a écrit :

> I have the macro examples working, and already I can see some neat
> opportunities for macro techniques.
>
> I noticed that in several places it mentions that you can do file
> operations using the neko package in a macro.  However, I'm not having
> any luck doing something simple, like listing the current working directory:
>
>
>
> #Demo.hx
> import haxe.macro.Expr;
> import haxe.macro.Context;
>
> class Demo {
>      public static function main(){
>          var x= CompileTime.cwd();
>          trace(x);
>      }
> }
>
> @:macro class CompileTime {
>      public static function cwd() : Expr {
>         var pos = haxe.macro.Context.currentPos();
>         var loc = neko.Sys.getCwd();
>         return {expr:EConst(CString(loc)), pos:pos}
>      }
> }
>
>
> I get an error of:
>
> /usr/local/haxe/std/neko/Sys.hx:64: characters 20-29 : Primitive not
> found std@get_cwd:0
> src/Demo.hx:15: characters 14-31 : Called from
> src/Demo.hx:6: characters 9-26 : Called from
> Aborted
>
> Other attempts at doing file i/o met with similar errors.
>
> Do I still have configuration issues, is the io module not working quite
> yet, or am I not understanding how to use the macros properly?
> I thought I would post here before making an issue on google code.

Your code is correct, but neko.Sys is not yet supported by macros ;)

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Adrian Veith
In reply to this post by Justin Donaldson-2

On 15.09.2010 02:54, Justin Donaldson wrote:
I have the macro examples working, and already I can see some neat opportunities for macro techniques.

I noticed that in several places it mentions that you can do file operations using the neko package in a macro.  However, I'm not having any luck doing something simple, like listing the current working directory:



#Demo.hx
import haxe.macro.Expr;
import haxe.macro.Context;

class Demo {   
    public static function main(){
        var x= CompileTime.cwd();
        trace(x);
    }
}

@:macro class CompileTime {
    public static function cwd() : Expr {
       var pos = haxe.macro.Context.currentPos();
       var loc = neko.Sys.getCwd();
       return {expr:EConst(CString(loc)), pos:pos}
    }
}


I get an error of:

/usr/local/haxe/std/neko/Sys.hx:64: characters 20-29 : Primitive not found std@get_cwd:0
src/Demo.hx:15: characters 14-31 : Called from
src/Demo.hx:6: characters 9-26 : Called from
Aborted


The problem is, that the macro interpreter is only neko compatible - but it is *not* neko ! as a consequence only code can be called which does not depend on neko .ndlls - which is a big show-stopper in my opinion, since there will be only a certain limited emulation library and not the full power of neko. If you want to access a database for schema validation for example - this will not be possible.

Cheers,

Adrian.




--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

blackdog-2

Before a call goes out for neko inclusion, imo it's also a big
showstopper if the compiler is tied to any runtime. If the neko api is
implemented in ocaml that's perfectly good, but as someone who never
installs neko i'd find it an imposition to have to do so.  

What this calls for is a completely platform agnostic api, which removes
all notion of #if neko #if php #if nodejs etc, (and is async too) which
the compiler could use internally. We need this anyway at a haxe std lib
level.

bd


On Wed, 2010-09-15 at 10:30 +0200, Adrian Veith wrote:

>
> On 15.09.2010 02:54, Justin Donaldson wrote:
> > I have the macro examples working, and already I can see some neat
> > opportunities for macro techniques.
> >
> > I noticed that in several places it mentions that you can do file
> > operations using the neko package in a macro.  However, I'm not
> > having any luck doing something simple, like listing the current
> > working directory:
> >
> >
> >
> > #Demo.hx
> > import haxe.macro.Expr;
> > import haxe.macro.Context;
> >
> > class Demo {    
> >     public static function main(){
> >         var x= CompileTime.cwd();
> >         trace(x);
> >     }
> > }
> >
> > @:macro class CompileTime {
> >     public static function cwd() : Expr {
> >        var pos = haxe.macro.Context.currentPos();
> >        var loc = neko.Sys.getCwd();
> >        return {expr:EConst(CString(loc)), pos:pos}
> >     }
> > }
> >
> >
> > I get an error of:
> >
> > /usr/local/haxe/std/neko/Sys.hx:64: characters 20-29 : Primitive not
> > found std@get_cwd:0
> > src/Demo.hx:15: characters 14-31 : Called from
> > src/Demo.hx:6: characters 9-26 : Called from
> > Aborted
> >
>
> The problem is, that the macro interpreter is only neko compatible -
> but it is *not* neko ! as a consequence only code can be called which
> does not depend on neko .ndlls - which is a big show-stopper in my
> opinion, since there will be only a certain limited emulation library
> and not the full power of neko. If you want to access a database for
> schema validation for example - this will not be possible.
>
> Cheers,
>
> Adrian.
>
>
>
> --
> haXe - an open source web programming language
> http://haxe.org



--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Nicolas Cannasse
In reply to this post by Adrian Veith
> The problem is, that the macro interpreter is only neko compatible - but
> it is *not* neko ! as a consequence only code can be called which does
> not depend on neko .ndlls - which is a big show-stopper in my opinion,
> since there will be only a certain limited emulation library and not the
> full power of neko. If you want to access a database for schema
> validation for example - this will not be possible.

This will be possible is you port the Mysql Client for Neko (written in
C) to haXe ;) Check neko/libs/mysql (in particular the my_proto
subdirectory) : there's actually very little code, and the only thing
you need are sockets which will be available in macros soon.

Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Adrian Veith
In reply to this post by blackdog-2

On 15.09.2010 13:07, blackdog wrote:
> Before a call goes out for neko inclusion, imo it's also a big
> showstopper if the compiler is tied to any runtime. If the neko api is
> implemented in ocaml that's perfectly good, but as someone who never
> installs neko i'd find it an imposition to have to do so.  
>
> What this calls for is a completely platform agnostic api, which removes
> all notion of #if neko #if php #if nodejs etc, (and is async too) which
> the compiler could use internally. We need this anyway at a haxe std lib
> level.

but at the moment we have one additional target (macro = neko emulated
in ocaml), which makes it even more difficult. If anyone wants to call
OS dependent code from inside a macro - it doesn't work. Now there are 2
possibilities:

1. You have to wait until this specific OS dependent code is emulated in
the macro ocaml layer.
2. You find a way to go around the problem (rewrite the code in pure
haXe - which is not always possible, because OS dependent code needs to
load libraries and there is no target independent way to do this). At
least you need a way to communicate with some proxy code.

Cheers,

Adrian.  

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Adrian Veith
In reply to this post by Nicolas Cannasse

On 15.09.2010 13:19, Nicolas Cannasse wrote:

>> The problem is, that the macro interpreter is only neko compatible - but
>> it is *not* neko ! as a consequence only code can be called which does
>> not depend on neko .ndlls - which is a big show-stopper in my opinion,
>> since there will be only a certain limited emulation library and not the
>> full power of neko. If you want to access a database for schema
>> validation for example - this will not be possible.
>
> This will be possible is you port the Mysql Client for Neko (written
> in C) to haXe ;) Check neko/libs/mysql (in particular the my_proto
> subdirectory) : there's actually very little code, and the only thing
> you need are sockets which will be available in macros soon.
>

I am not using mysql nor sqlite - we have our own database server. But
sockets will help anyway - if haXe remoting is possible. With haXe
remoting it is easy to call a proxy, which loads the database.

Cheers,

Adrian.

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Justin Donaldson-2
In reply to this post by Nicolas Cannasse
Ah ok, I did get a file operation to work using neko.io.File.getContent() now.

My other file i/o problem came from this earlier problem:

import haxe.macro.Expr;
import haxe.macro.Context;

class Demo {   
    public static function main(){
        var x= bar();
        trace(x);
        var y = mbar();
        trace(y);
    }
    public static function bar() {
            var fh = neko.io.File.read('/Users/jjdonald/Desktop/test.txt', false);
            var content = fh.readAll().toString();
            fh.close();
            return content;
    }
   
    @:macro public static function mbar() :Expr {
            var pos = haxe.macro.Context.currentPos();
            var fh = neko.io.File.read('/Users/jjdonald/Desktop/test.txt', false);
            var content = fh.readAll().toString();
            fh.close();
            return {expr:EConst(CString(content)), pos:pos}
    }
}


The normal bar() function will work fine, returning the contents of my test file.  The macro bar function will fail, throwing a "blocked" error:

/usr/local/haxe/std/haxe/io/Input.hx:84: characters 2-5 : Blocked
src/Demo.hx:23: characters 17-29 : Called from
src/Demo.hx:8: characters 10-16 : Called from
Aborted

I'm wondering if the macroneko vm is blocking itself from accessing the file, or if it is something else.  For instance, it looks like line 84 on Input.hx is a "try" statement, and perhaps macroneko cannot perform try(s). 

Thanks for the quick feedback on all of this, I realize macros are a moving target.

-Justin



(It looks like others are having the same db interface idea I was thinking of, great!)


On Wed, Sep 15, 2010 at 12:52 AM, Nicolas Cannasse <[hidden email]> wrote:
Le 15/09/2010 02:54, Justin Donaldson a écrit :

I have the macro examples working, and already I can see some neat
opportunities for macro techniques.

I noticed that in several places it mentions that you can do file
operations using the neko package in a macro.  However, I'm not having
any luck doing something simple, like listing the current working directory:



#Demo.hx
import haxe.macro.Expr;
import haxe.macro.Context;

class Demo {
    public static function main(){
        var x= CompileTime.cwd();
        trace(x);
    }
}

@:macro class CompileTime {
    public static function cwd() : Expr {
       var pos = haxe.macro.Context.currentPos();
       var loc = neko.Sys.getCwd();
       return {expr:EConst(CString(loc)), pos:pos}
    }
}


I get an error of:

/usr/local/haxe/std/neko/Sys.hx:64: characters 20-29 : Primitive not
found std@get_cwd:0
src/Demo.hx:15: characters 14-31 : Called from
src/Demo.hx:6: characters 9-26 : Called from
Aborted

Other attempts at doing file i/o met with similar errors.

Do I still have configuration issues, is the io module not working quite
yet, or am I not understanding how to use the macros properly?
I thought I would post here before making an issue on google code.

Your code is correct, but neko.Sys is not yet supported by macros ;)

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org



--
blog: http://www.scwn.net
aim: iujjd
twitter: jjdonald


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Nicolas Cannasse
[...]
> The normal bar() function will work fine, returning the contents of my
> test file.  The macro bar function will fail, throwing a "blocked" error:
>
> /usr/local/haxe/std/haxe/io/Input.hx:84: characters 2-5 : Blocked
> src/Demo.hx:23: characters 17-29 : Called from
> src/Demo.hx:8: characters 10-16 : Called from
> Aborted

Thanks for reporting, I'll try to get this fixed tomorrow. I'll also
commit partial Xml and Regexp support for macros.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Justin Donaldson-2
Regexp support would probably be useful ... Process too... although I can't come up with good ideas at the moment.  I'm sure they're out there.

I was looking at how you divided up the ExprDef's with EConst, EBlock, etc.  That looks really flexible already... it'll be interesting to reorganize expressions on the fly. 

I was thinking about macro completions, which I think will be very useful given how flexible they will become.  One possible approach may be an  ECompletionCIdent/ECompletionECall enum.  These enums would just be a sort of flag that gets assigned during partial expressions (fields, functions, etc). 

For example.... the compiler determines if a completion attempt is appropriate for haxe syntax, and then passes the new completion expression enum on to the macro.  The macro  could then detect that a completion was triggered, and where exactly in the expression structure it occurred.  If it's a valid completion state (for the macro), then the macro could create an appropriate CIdent or ECall expression on the fly to return back to the compiler.  The compiler would make sure that the completion request matches the returned value (for instance, EcompletionCIdent => CIdent), and then would create and return the xml completion data as normal.  The syntax would stay the same, and it would totally be up to the macro developer to handle completions.  If the macro developer chose not to handle completions, completion attempts would just generate compilation errors.

Great stuff so far, this is my favorite new feature since "using".

-Justin

On Wed, Sep 15, 2010 at 12:16 PM, Nicolas Cannasse <[hidden email]> wrote:
[...]

The normal bar() function will work fine, returning the contents of my
test file.  The macro bar function will fail, throwing a "blocked" error:

/usr/local/haxe/std/haxe/io/Input.hx:84: characters 2-5 : Blocked
src/Demo.hx:23: characters 17-29 : Called from
src/Demo.hx:8: characters 10-16 : Called from
Aborted

Thanks for reporting, I'll try to get this fixed tomorrow. I'll also commit partial Xml and Regexp support for macros.


Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org



--
blog: http://www.scwn.net
aim: iujjd
twitter: jjdonald


--
haXe - an open source web programming language
http://haxe.org
Reply | Threaded
Open this post in threaded view
|

Re: macro problems

Nicolas Cannasse
Le 16/09/2010 07:16, Justin Donaldson a écrit :
> Regexp support would probably be useful ... Process too... although I
> can't come up with good ideas at the moment.  I'm sure they're out there.

I've committed support for Regexp, ZLib, and a few File I/O fixes
(including your report).

There is also partial support for parse_xml but since it uses a very
specific Xml parser it does currently skip comments, doctype, prolog and
does not support CDATA. I'll have to implement at some time a
haXe-compatible Xml parser in ocaml but that's already a start.

The missing Neko apis are now the following :
   std : system, utf8, process, socket, serialize
   std : memory, module, thread, ui - not planned
   mysql, sqlite : not planned
   mod_neko : don't make sense
See http://nekovm.org/doc/libs for a complete API list

> I was thinking about macro completions, which I think will be very
> useful given how flexible they will become.  One possible approach may
> be an  ECompletionCIdent/ECompletionECall enum.  These enums would just
> be a sort of flag that gets assigned during partial expressions (fields,
> functions, etc).

This is already the way they are handled by the compiler. The parser
will return a EDisplay or EDisplayNew expression when hitting the place
completion is required.

I didn't added them to macros, but it's now done. Tell me if you manage
to get some smart completion working with it.

Best,
Nicolas

--
haXe - an open source web programming language
http://haxe.org