I wonder if that holds up when you look at demographics within each country.

I don't know about the ones you mentioned, but I know that I the US, our national fertility rate is heavily increased by first generation immigrants, who are also more religious.