Hacker News new | past | comments | ask | show | jobs | submit login

Llama.cpp is a lot of C SIMD. I think using C++ elsewhere in the project just made sense in its infancy, when it was CPU only.

Rust is really interesting for stuff besides running the actual llm though. Python can be a huge performance/debugging pain when your frontend and such get huge.




> Rust is really interesting for stuff besides running the actual llm though.

That's the reason why llama.cpp is so attractive though. It just runs the LLM.


Thats not giving the project enough credit though, it implements a ton of very useful features outside the llm itself.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: