Tile tracker tags use a 3V 130mAh CR1632 battery, which lasts for a year... I wish I remembered the math to figure out what trickle wattage that works out to, for comparison.
3*0.130 = 0.39 Wh
Over 5000 years it would amount to about 8.9 nW (nanowatt), of course assuming the chemistry to be able to sustain such time span (which is not).
In that case, one of these diamond batteries would be able to substitute for the CR1632 and give a sealed tracker an effectively infinite lifetime (at least compared to, well, human civilization). I imagine it would be pricey, though.
It also wouldn't be able to do drive a speaker, another key purpose of a Tile. Its actual tracking is primarily passive, so it's hard to locate with just a phone. (You might be able to triangulate from multiple phones... I recall an HN article with somebody doing that a few weeks ago). The main use case is to get close enough to trigger it, then follow your ears. That's thousands of milliwatts.
I suppose it might be possible to add a capacitor that's being continually charged.