Isn’t it just like, send prompt, get response? Why is it necessary to implement chat if you don’t care about chat? Is it used for scheduling or something?
Its all to do with how the API is designed. Now the prompts need to be sent in the form of chat, and we need to worry about text being output from the model, and not just code. Basically, users need to comb through their codebase, change their API calls and see if nothing breaks. While gpt turbo is quite good, it was quite a headache to replace davinci calls with GPT Turbo, especially when I closely integrated it with a chain of prompts. I can only imagine it being more of a headache for codex users
You already had to deal with text being output from the Codex models. I regularly ran into text from Python mailing lists when using the Codex API. Sometimes it writes code, loses coherency and types out pages and pages of Python mailing list comments and signatures.
Any human text I get from the Chat completion API with the task of completing code is at least relevant to the code being written.
I tried a prompt like this to get it to spit code out:
“Pretend you are a SQL expert. You convert requests written in English to Postgres SQL. Your output is always SQL only. If you need to respond with something other than SQL code, add it to comments within the SQL code you generate and prepend it with ‘Note: ‘. Your output will automatically be executed on a read-only Postgres database called ‘my_db’ and the result of that will be given to you. If you are still working on getting an answer, add ‘Working’ at the end of your message, or ‘Done’ if you for the result you needed to answer the request.”
Request: How many API calls were made by clients last month?”
You can keep that at the top of every request, while running the SQL code iteratively until it says Done.
While I agree that some changes have to be done in your code, preventing the model from outputting text is relatively easy, you just have to have a strong system message to set the model to give you the output you want.
