Hacker News new | past | comments | ask | show | jobs | submit | instance's comments login

I tested on a serious use case and quality was subpar. For real use cases I had to either host the most powerful model you can get (e.g. LLaMA-65B or so) on a cloud machine, which again costs too much (you'll be paying like 500-1000 USD per month), or just go straight for GPT-3.5 on OpenAI. The latter economically makes most sense.


what real use case did you use it for?


For instance used it in conjunction with llama-index for knowledge management. Created an index for a whole confluence/jira of a mid-sized company, got good results with GPT, but for LLaMA of this size that use case was too much.


I'd argue 1k per month for mid-sized company is nothing, but I can understand where you are coming from.


Did you try instructor-xl? It ranks highest on huggingface.


Making demos to raise investment probably


What about turning the cloud vm off except when you're actually using it?


So modal.com is "turning-the-vm-off-when-unused-as-a-service" :-)

I ran research/open_llama_7b_preview_200bt on there, using they python example, with A10G gpu.

Cost 2-3c per run, taking ~20 seconds each time, on fairly small prompts. So about the same as GPT-4?

Now this is a non expert just playing, it probably can be optimized by trying different GPUs and optimizing the code somehow.

I don't think you are using these models to save money, but you might be using them for tunability, privacy, mobility [1], secrecy or fun/research.

[1] in other words you want to build a robot that can work disconnected from the internet.


A "serious use case" means it needs to be available around the clock.


I've put up something like this as a side project a few weeks ago: https://flamingoo.ai/

Love the term "PAF", I think I'll include this.


The website does not mention what the product is


yea, you're right. It's really pre-MVP. Basically an API in between your user facing input and OpenAI that detects prompt injections and flags them for you so you can abort sending to OpenAI.


7B is the number of parameters of the model.


I know an amazing IC at a software company I once worked at that I still keep in touch with. He has extremely deep technical knowledge, which you can simply deduct from basically everyone in the company (even from other teams) coming to him for advice. He's been at the company for >10 years.

He has strong opinions on current processes and just getting things done.

This post resonated deeply with me, since I've discussed before with him his role and how it could evolve. I know for a fact that he dislikes lots of meeting and really likes working on the core product, and so far hasn't really jumped on the opportunity to go into management - he doesn't really want to be manager. So he is kind of exactly the guy the post is describing. The company is growing though and he is very slowly getting pushed by the head of development into a more managerial role..

Let's see how it works out. I believe he is going to be a great manager though.


Some of the best I've seen are highly technical people who moved into management and then realized they can actually do more by leveraging their people. It's not the same hands-on, but they enjoy working with others to get things done.


Check it out: https://www.bmvi.de/EN/Home/home.html

"Federal Ministry for Digital and Transport"


It got its current name 2013. It was a merger 1998 of "Federal Ministry for Housing, Urban Development and Building" and the one responsible for transport. The urban development department was responsible for internet infrastructure not the transport one.


Shouldn't it be the ministry for infrastructure?


Doesn't confirm the "Because of this phrase" part of the claim.


To add to this, as a German, the future of Uber does not look great in Europe as well. The taxi lobby is strong here, and I’m sure regulations will come in countries like Germany and will affect negatively their business. And they're starting to face stronger competition from other companies like Bolt.


As a European I would like to say that using normal taxis is an absolutely terrible experience. It’s expensive, no way to get reviews about a driver/select for quality, you’re often not quoted a price up front/you don’t know how much you’ll end up paying/whether their meter is rigged, it’s hard to quickly get a taxi (many independent operators), and the list goes on. I sure hope Uber succeeds in Europe.


I've been collecting these for the past year - there is a large body of research going into solving optimization problems using quantum computing, and some interesting POC results coming out using the latest machines.


Yea, that title is quite something. I thought this was some long lost brother or sister of Mark that has now appeared.


I wouldn't say that's a YC-specific issue. Look at Nasdaq, this is a general issue with tech stocks right now, just this immense selling all across the board.


https://blog.xa0.de/list

My basic blog where I talk about Quantum Computing, Machine Learning, Finance, Business.


Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: