Jitter is a very interesting project! It's a kind of generic interpreter that is—as the name suggests—almost a JIT compiler. It's something more or less "in between". Technically it's more an interpreter + VM. But it uses all kinds of (I think novel) tricks to remove the overhead of interpretation, which makes the code at runtime almost look like what a JIT compiler would have produced.
Some more internal info than on the GNU page on how this works can be found in the following slides:
I'm curious as to whether the techniques used here are safe against Spectre-like attacks. There appears to be a heavy reliance on indirect branching. The slides here are dated just before Spectre was published, so it would not have been a concern at the time.
Oh, this is one I hadn't heard of. There are several different takes on this idea, and I've had the notion to make my own for some years, too (not enough free time...)
I feel like a good "how powerful is this tool, really?" test would be to model the Adobe PSD format with it. That format is bananas, and is what got me into this topic in the first place.
But for those interested, here are some other tools in this vein:
Thanks for this compendium. I originally wrote and released binspector, but have since moved to HexFiend, because I like the power Tcl affords as a template language. I too have rewritten specs for PSD too many times to count; it’s one of my favorite nerd-sniping pastimes. I’ll have to explore some of these other links, to find a good editor for my Windows colleagues.
I am not affiliated Synalyze It!, but have been a customer for several years. Generally, I've been happy with it, though it crashes occasionally. The support has been responsive when I've had licensing issues.
This is why I keep reading these threads, looking for something I can wrap my head around.
I started looking at Kaitai Struct not too long ago. I purposely started with a pretty simple format, but I think I'm not smart enough to figure out the input format. Also, I'm pretty sure I hate YAML.
I tried building Poke just last week, having resolved to try it since 1.0, but hit problems with the Boehm GC on my old Mac, and then what looks like Readline problems on my new Mac. I haven't given up, but I'm off to a slow start.
Hachoir, mentioned in another comment, looks promising, although the input looks suspiciously similar to a program I'd write myself.
Haven't watched the full video, but this looks similar ImHex (https://imhex.werwolv.net/), which also includes a pattern language thing to describe the structure of data. I used it once for a project, and it was useful when it worked, although I ran into some limitations when trying to model container formats.
Maybe it could do that and I just couldn't figure it out at the time, but if you have say a zip file with different file formats, you couldn't tell the language to switch between different structures based on like an index or a header that tells you the format of a subsection. It was a limitation of the pattern language.
I wonder if GNU poke is more advanced in that regard? A tool like this would be super useful for debugging custom binary formats, but some formats can get pretty complex.
I always liked Winhex for this type of thing. It could dissect a structure with a c-like syntax. I found it as shareware as a kid and have never used a hex editor I liked more.
wxHexEditor has a similar look and feel to WinHex, and similarly supports editing of extremely large files or raw devices. It's nowhere near as featureful but still very useful, and doesn't have the proprietary license.
It may not be ideal to praise commercial software in a GNU software thread, but I really enjoy using 010 Editor. It's one of the few programs I pay for. Its C-like template syntax (looks similar to GNU Poke syntax) makes reverse engineering binary formats easy and fun!
I love/hate that "pickle" is defined through usage. Curious to know if that name has any meaning (e.g. python pickles are for long-term storage of ephemeral data), but not curious enough to watch a video just in case that's covered.
Poke is actually not only an editor, it's a domain specific language.
This language is built on top of something called GNU Jitter.
https://www.gnu.org/software/jitter/
Jitter is a very interesting project! It's a kind of generic interpreter that is—as the name suggests—almost a JIT compiler. It's something more or less "in between". Technically it's more an interpreter + VM. But it uses all kinds of (I think novel) tricks to remove the overhead of interpretation, which makes the code at runtime almost look like what a JIT compiler would have produced.
Some more internal info than on the GNU page on how this works can be found in the following slides:
http://ageinghacker.net/talks/jitter-slides--saiu--ghm2017--...