- OpenAI APIs expose the extra features (like llama.cpp's grammar) anyway, as extra parameters.
- There's a huge productivity benefit to making your backend swappable, without having to redo all the API calls.
Prompt formatting in particular is a huge sticker with OpenAI endpoints though. Something does need to be done about that.
- OpenAI APIs expose the extra features (like llama.cpp's grammar) anyway, as extra parameters.
- There's a huge productivity benefit to making your backend swappable, without having to redo all the API calls.
Prompt formatting in particular is a huge sticker with OpenAI endpoints though. Something does need to be done about that.