Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yes, but llama.cpp already comes with a ready-made OpenAI-compatible inference server.


I think people are getting hung up on the "llama.cpp" name and thinking they need to write C++ code to use it.

llama.cpp isn't (just) a C++ library/codebase -- it's a CLI application, server application (llama-server), etc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: