>Sorry, but NNs haven't made any significant progress in the realm of program synthesis. I can set up a model with Transformers and BERT that spews out thousands of lines of code that stylistically look correct, but does not compile. NNs aren't the right tool for the job here.

Di.. did you even bother reading what I wrote? You wouldn't use NNs to directly output code, you'd use it to guide a search process.