I've looked into using NN for some of my specific work, but making sure output is bounded ends up being such a big issue that the very code/checks required to make sure it's within acceptable specs, in a deterministic way, ends up being an acceptable solution, making the NN unnecessary.
How do you handle that sort of thing? Maybe main process then leave some relatively small residual to the NN?
Is your poking more like "fuzzing", where you just perturb all the input parameters in a relatively "complete" way to try to find if anything goes wild?
I'm very interested in the details behind "critical" type use cases of NN, which I've never been able to stomach in my work.
For us, the NN is used in a grey box model for MPC in chemical engineering. The factories we control have relatively long characteristic time, together with all the engineering bounds, we can use the NN to model parts of the equipment from raw DCS data. The NN modeled parts are usually not the most critical (we are 1st principles based for them) but this allows us to quickly fit/deploy a new MPC in production.
Faster time to market/production is for us the main reason/advantage of the approach.