Good point. The only advantage that occurs to me is that you could spend the journey (waves hands) converting the interstellar object into reaction mass to enable deceleration at the other end. And maybe also for course correction, because the object is very pretty unlikely to be going anywhere interesting just by chance
I typed slapdash things into Wolfram Alfa just to get a sense of the kinetic energy of something like a 100km metal sphere buzzing as at 40 km/s (it's a screaming chonker of an overestimate but hey) and got something like a millionth of total yearly solar output or 1/5th of lunar kinetic energy (orbiting earth). As D. Wooderson famously said, we're talkin' some fuckin' muscle, to ride such a thing. If you have it, you might even be better off just taking the moon instead.