Uhhh, no they don't? They see the results of RAG but there's very little to distinguish what is context you gathered vs. what isn't. On top of that, there's nothing in the prompt that indicates what decision was made before inserting something into a prompt. Let's say my retrieval pipeline used a mix of semantic search and keyword-based search, and I used some mechanism to decide which results from which search to include for this specific request. Nothing in the prompt will say that, and so the differentiating behavior is still kept "secret" if you will.

But I think this line of thinking is a bit ridiculous. Yes, if you send your data to a database service, that provider has your data and theoretically use it to run you out of business. Except they kinda can't, both as a matter of practicality and legality. OpenAI, Anthropic, and others have SOC II compliance. The easiest way to ensure you run yourself out of the market is to start violating that stuff.