One trick that helped me to master the numbers in a foreign languages was to mentally make additions in that language. For example, in Japanese, I would ask myself "How much is Ni plus Go?"
At first I would mentally do "well, Ni is 2, Go is 5, 2+5=7, 7 is Nana, so that's the answer" and at some point I was able to do it directly without translating the numbers to my native language.
The most difficult problem I have with Japanese numbers (going on 13 years of study and ten years in-country) is STILL dealing with counting higher numbers based on 10,000s. I've drilled and practiced and it still doesn't come fluently; I always find myself converting the Japanese to English, doing the math, then reconverting to Japanese, usually using my fingers to tick off zeroes to figure out how many 10,000s the number has.
The times I count higher than 1,000 in Japanese is almost always currency. I think in USD and yen, so if I say 一万円 I think either "ichimanen" or $100. When I have to translate a big number like 54万円 I just tell them $5400. It's surely not as good as properly learning but it just takes so much longer when translating live.
At first I would mentally do "well, Ni is 2, Go is 5, 2+5=7, 7 is Nana, so that's the answer" and at some point I was able to do it directly without translating the numbers to my native language.