pegtl is not actually a packrat parser. Not all PEGs and absolutely not all recursive descent parsers are packrats.
And actually, my experience with packrat parsers (mostly in ruby) in other languages has been that they actually slow things down on moderately or more complex grammars by massively exploding memory use and thus allocation pressures. Turning it off can make it faster, especially on complex grammars. It's a pretty good case study in how optimizing an O(n^2) worst case to O(n) does not always improve things.
That said, I'm not against the principle, but the shotgun approach to it can be brutally bad. You really only want to memoize the paths that are actually likely to backtrack. Or simple grammars where O(n^2) memory use is not going to balloon your memory use too much it's a clear win.
> Not all PEGs and absolutely not all recursive descent parsers are packrats.
Of course not, see the link I posted. PEGs are TDPL and recursive descent (not the other way around), and an alternative to CFGs. PEGs were coined by Bryan Ford, who then coined packrat parsing based on them.
You want to do smarter things than just memoising everything, yeah. Especially for complex grammars.
I've been working on a new PEG parsing algorithm, and though my first crack at it was way too slow and I still haven't finished debugging the second version, I was able to show that packrat parsing had about a 40% runtime hit over recursive descent for well-behaved inputs on some common grammars (and linear memory usage vs. constant).
And actually, my experience with packrat parsers (mostly in ruby) in other languages has been that they actually slow things down on moderately or more complex grammars by massively exploding memory use and thus allocation pressures. Turning it off can make it faster, especially on complex grammars. It's a pretty good case study in how optimizing an O(n^2) worst case to O(n) does not always improve things.
That said, I'm not against the principle, but the shotgun approach to it can be brutally bad. You really only want to memoize the paths that are actually likely to backtrack. Or simple grammars where O(n^2) memory use is not going to balloon your memory use too much it's a clear win.