It looks like the frontend is a React SPA. What is the backend stack?
If your backend is is javascript and/or depending on how complicated it is, an easy idea might be to allow users to paste their own OpenAI API token in and have it use that. For various security/privacy reasons it would be ideal if the API calls came directly from the frontend in that case though, and given the caching implementation and other things I'm guessing decoupling that might be pretty challenging. Figured I'd throw out the idea anyway though.
If your backend is is javascript and/or depending on how complicated it is, an easy idea might be to allow users to paste their own OpenAI API token in and have it use that. For various security/privacy reasons it would be ideal if the API calls came directly from the frontend in that case though, and given the caching implementation and other things I'm guessing decoupling that might be pretty challenging. Figured I'd throw out the idea anyway though.