You are reaping the benefits of having an active fitness regime in your 30s .
The average 50 year old who starts exercising today will never get to your fitness level .
I don’t think I’m convinced of that. Most fitness measures saturate as you approach a limit. For example, you can build muscle mass rapidly in the first couple of years but it slows dramatically as you approach your genetic limit. It only takes a few years of consistent training to get to that asymptotic level. So on that measure, you can basically “catch up” to your hypothetical “always trained” self.
While you might not be able to get to the exact same level of fitness as if you had trained for years, I think you can get fairly close. And without a doubt, you can drastically improve from wherever you’re starting.