There was a piece here on HN about a week ago on Kolmagorov Arnold network (KANs) as an alternative to NNs. https://news.ycombinator.com/item?id=40219205 If I understood it correctly, a KAN would be a suitable network to work on this problem, as it's activation functions are in the links (and can be any function).

Page 15 of KAN paper deals with Feynman datasets.

If we can figure out symbolic regression in a universal fashion that would be AGI in a way, right?

Yes, I think so - especially if it was fast.

A very fast at scale universal symbolic regression machine would be formidable.

That's a really hard problem, I wonder how stuff like topology and combinatorial mathematics are related to this, do they apply to this kind of problems?

Yes. Topology of the function ASTs and combinatorics of the various ways to assign primitives (fn) and terminals (let) to the nodes of the function.

The answers to the challenge are words so obey a zipfian distribution