And you still can't provide a custom grammar to the API...
The company I work for desperately need the LLM to consistently generate results in a subset of HTML. I was able to craft a small grammar file that does just that, in under 5 minutes, and use it successfully with llama.cpp. Yet, there are still no API offering this basic feature that could really benefit everyone.
Instead we have a thousand garbage medium articles with "tips & tricks" on how to prompt the AI to get better results. It's as if people don't care anymore about consistency and reliability.
The company I work for desperately need the LLM to consistently generate results in a subset of HTML. I was able to craft a small grammar file that does just that, in under 5 minutes, and use it successfully with llama.cpp. Yet, there are still no API offering this basic feature that could really benefit everyone.
Instead we have a thousand garbage medium articles with "tips & tricks" on how to prompt the AI to get better results. It's as if people don't care anymore about consistency and reliability.