You'd have to design a capacitor, or the equivalent, that could last thousands of years for stuff that you can't service (like a space probe). That is a pretty big ask for many materials in general.
Right out of the gate aluminum electrolytic capacitors are out as getting 10-20 years out of consumer grade ones is pretty good. I have a single monitor I've replaced caps in 3 times now (and they weren't even a known bad batch) in the past 10 years. Electrolytic caps will go bad even faster if they aren't regularly receiving electricity, many have a 2 year shelf life. If you've ever plugged in a vintage electronic (say an Atari/Commodore computer) and you've had smoke-like stuff come out, that's usually the electrolytic capacitors going. If you open up an old device that has had one or more fail, you'll find this brownish-black stuff all over the insides after they go.
Most super-capacitors would die from use long before useful power output, they're usually good for a million cycles or less so even at one cycle a day you're only looking at 2700 years and change.
Polymer electrolytic capacitors are good in use for 10-15 years.
Pretty much every capacitor out there right now has a 5-15 year lifespan.
A data point from the Apollo Guidance Computer restoration: we found all the capacitors worked perfectly after 50 years. This included a bunch of electrolytic and tantalum capacitors.
They weren't in use that entire time though and I'm guessing they were kept somewhat climate controlled too which helps a ton unlike other vintage electronics that get shoved in attics/garages and exposed to wide temperature ranges year after year. First thing I do when I buy a new vintage computer or computer accessory (I'm an 8-bit Atari guy) is change the caps (period) and then remove anything socketed and clean the leads.
You actually worked on restoring them? Do public images of the innards exist anywhere from that? I'd love to see that, the innards of old electronics and machines are just so beautiful in their simplicity. You can look at them and actually understand what is going on.
Edit: found your blog, getting lost in the images.
Climate controlled? No, the AGC was in a junkyard in Houston and then years in a barn. The capacitors worked because they were aerospace quality and thoroughly tested.
Already subscribed and a few posts deep now, really cool stuff! If you see a spike in visitors I shared it on the AtariAge Facebook group as a lot of them will probably also dig your content.
Voltage is part of the problem for sure, which a variable transformer can definitely help keep voltage at a steady operation (you may not even need that since it's a Thermoelectric generator... I'm not familiar enough).
But it's primarily wattage - at some point the overall power dips too low to maintain voltage at that amperage to keep the system alive.
