- Releasing/acquiring the GIL isn't cheap. So you'd need to know in advance whether it's worth it or not.
- You need the GIL acquired to instantiate the resulting "match data", captures, etc.
Stuff like re.sub frequently calls into Python code as well, so it would seem to be rather hairy to get rid of the GIL for this. It could of course add the usual workarounds of cooperative multitasking, like yielding (releasing the GIL) from time to time, just like the main interpreter loop does (sys.setswitchinterval)
assuming it's legal to do so, GIL release seems like a reasonable flag to add to possibly-expensive functions like re.()
... or even better, have re. track its own performance and release the GIL if it forecasts that it statistically-might use a lot more computation... a trivial strawman algorithm: if len(input) > MAX then release GIL first.
> However, the regular expression module is a C module that doesn’t release the GIL before running.
I would love to know why.