I'm trying to find the episode, but on the All in Podcast ~6 months ago, they made comments about how the corporate structure of OpenAI may have been a secret way for Sam Altman to hold a large stake in the company. I don't think this is privacy related, but that there was a shell game with the equity and the non profit status. If they were training on data like that, the board/people at the company would have known.
"somebody has to own the residual value of the company, sam controls the non profit, and so the non profit after all equity gets paid out at lower valuations, owns the whole company. Sam altman controls all of open ai if its a trillion dollar valuation. Which if true would be a huge scandal"
EDIT:
episode is here: https://www.youtube.com/watch?v=4spNsmlxWVQ,
"somebody has to own the residual value of the company, sam controls the non profit, and so the non profit after all equity gets paid out at lower valuations, owns the whole company. Sam altman controls all of open ai if its a trillion dollar valuation. Which if true would be a huge scandal"