The grammar definitions in the article looked very similar to PEG at the first glance. And the article seems to confirm, “The parsing algorithm that RLMeta implements is based on parsing expression grammars (PEG), but is extended to match arbitrary objects, not just characters.”
I didn’t quite get the “match arbitrary objects” bit... Is this basically the same technique as PEG or what am I missing?
As far as I know, PEGs work only on streams of characters. RLMeta works on streams of arbitrary Python objects. The syntax currently has support for matching characters, strings, and lists. Does that clarify it?
Thanks, but it still seems to me that the first (“parser”) part is very much like a PEG.
It’s the “generator” part that matches on objects (AST), generated by the parser, but I’m not sure why that’s useful:
- in the toy example the evaluation could happen right in the parser,
- and if you decide to have an AST, why use a DSL that looks similar to the grammar definition to walk it?
Yes, you are right that the parser grammars are much like PEGs and it's the code generator grammars that match other objects than characters.
The reason I chose to implement it that way in the article is that I modeled RLMeta after OMeta. If I remember correctly, their reason for doing so was that they thought compilers would be easier to write if all passes (parsing, code generation, ...) could be written using the same language.
I think using the same DSL for code generators works quite well and they become quite readable. Perhaps there is a better DSL suited for code generation, and it would be interesting to experiment with that as well.
You are right that the toy examples could generate code directly in the parser. There is no need for a separate code generator step. But the RLMeta compiler would be much less readable without a separate code generator step I think. The reason I introduced an AST in the toy example was to introduce the concept before I showed it in the RLMeta implementation.
I didn’t quite get the “match arbitrary objects” bit... Is this basically the same technique as PEG or what am I missing?