I feel like I'm living in a different universe sometimes. The consensus on HN seems to be that you can be pretty productive with LLMs as coding assistants, but every time I try I find it borderline impossible to get functional code even for pretty straightforward prompts.
I decided to fire up GPT-4o again today to see if maybe things have gotten better over the past few months.
I asked GPT to write code to render a triangle using Vulkan (a 3D graphics API). There are about 1000 tutorials on this that are almost certainly in GPT-4's training data. I gave GPT two small twists so it's not a simple case of copy/paste: I asked it 1) to apply a texture to the triangle and 2) to keep all the code in a single function. (Most tutorials break the code up into about a dozen functions, but each of these functions are called only once, so it should be trivial to inline them.)
Within the first ten lines, the code is already completely nonfunctional:
GPT-4o declares a pointer (VkPhysicalDevice) that is uninitialized. It queries the number of graphics devices on the host machine. A human being would allocate a buffer with that number of elements and store the reference in the pointer. GPT-4o just ignores the result. Completely ignores it. So the function call was just for fun, I guess? It then tries to copy an entire array of VkPhysicalDevice_T objects into this uninitialized pointer. So that's a guaranteed memory access violation right off the bat.
Sometime I think there's something wrong with me. I've used copilot, I'm paying for ChatGPT and we're also having the Jetbrains AI, and there's just something so off about all of this.
Some basic things are fine, but once you get into specialised things, everything gets just terribly wrong and weird. I can't even put it into words, I see people saying how they are 10x more productive (I'd like to see actual numbers and proof for this), but I just don't see how. Maybe we're working on very custom stuff, or very specific things, but all of these tools seem to give very deep or confident answers that are just plain wrong and shallow. Just yesterday I used GPT 4o for some basic help with Puppet, and the examples it printed, even though I would say it's quite basic, were just wrong, but in the sense of having to debug it for 2 hours just to figure out how ridiculous the error was.
I fear the fact that people will end up releasing unsafe, insecure and simply wrong code every day, code that they never debug and not even understand, that maybe works for a basic set of details, but once the real world hits it, it will fail like those self driving cars driving full speed into a trailer that has the same color as the road or sky.
I think there's a vast ocean of different software engineers and the type of code they write on a daily basis, which is why you get such differing views on AI's effectiveness.
For me, AI has only ever been useful for very basic tasks and scripts:
If I need a quick helper script for something in Python, a language that isn't my daily drier, AI usually gets me there in a couple of prompts. Or maybe I am writing some powershell/bash and forgot the syntax for something and AI is quicker (or has more context) than a web search.
However, my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base. AI is just completely out of its depth in such cases due to lack of context but also lack of source material to draw from.
Even unit tests only work with the most basic of cases, most of the time the setup is so complex it just produces garbage.
I've also had very little luck getting it to write performant code. I almost have to feed it the techniques or algorithms before it attempts to write such code, and even then it's usually wrong or not as efficient as it could be.
Indeed, if you need a lot of boilerplate that's pretty similar to existing commonly available code, you're set. However...
That code is probably buggy, slow, poorly architected, very verbose, and has logical issues where the examples and your needs dont exact match.
Generally, the longer the snippet you want your LLM to generate, the more likely its going to go off the rails.
I think for some positions this can get you 90% of the code done. For me this usually means I can get started very fast on a new problem, but the last remaining "10%" actually takes significantly longer and more effort to integrate because I dont understand the other 90% off the top :)
> my main job is trying to come up with "elegant" architectures for complex business logic that interacts with an existing large code base
Right, that's surely the main job of nearly every experienced developer. It's really cool that LLMs can generate code for isolated tasks, but they can barely even begin to do the hard work, and that seems very unlikely to change in the foreseeable future.
I have the same experience. I have my own benchmark, I take a relatively complex project like FreeSWITCH on github, which is part of the training set for all coding LLMs anyway so they should know it and I ask the AI to code small snippets, tests, suggestions and fixes to see how well it understands the codebase and the architecture.
I just tried the latest Cursor + Sonnet and it failed in every task. The problem is that there is no way to understand the code without either complete understanding of the domain and the intents or running it in some context.
Telecom and media domains in particular are well documented in specs and studied in forum discussions. I am sure they are part of the training data, because I can get most answers if asked directly. So far the LLMs fail to reason about anything useful for me.
Having spent a couple of hours with some llama models and others I've given up on them for code. Code generation from XML or grinding it out ad hoc with stuff like sed on top of a data or config file is faster and more convenient for me.
The Jetbrains thing is rather rudely incompetent, it consistently insists on suggestions that use variables and fragments that are supposed to be replaced by what I'm writing and also much more complex than what I actually need. I suffered through at least a hundred mistaken tabbed out shitty suggestions before I disconnected it.
I don’t use ChatGPT or chat AI for coding, at least not often. It does help me get unstuck on occasion, but most of the time the context switch is too much of a productivity sink.
However, I use copilot autocomplete consistently. That makes me much more productive. It’s just really good autocomplete.
As an experienced developer I’d put my productivity improvement at 2-3x. It’s huge but not 10x. I’m limited by my decision speed, I need to decide what I want the code to do, AI can’t help with that - it can only do the “how”.
Far from introducing more bugs, using Copilot frees some mental cycles for me to be more aware of the code I’m writing.
It's going to be the same issue as poorly-structured Excel workflows: non-programmers doing programming. Excel itself gets the blame for the buggy poorly-structured spreadsheets made by people who don't have the "programmer mindset" of thinking of the edge cases, exceptions, and extensibility.
So it will be with inexperienced people coding with LLMs.
Absolutely the same for me. Whenever I encounter a problem I'm pretty sure nobody else has encountered before (or has not written about it at least), ChatGPT writes complete nonsense.
On stuff that's a bit obscure, but not really (like your Vulkan example), ChatGPT tends to write 60-95% correct code, that's flawed in the exact ways a noob wouldn't be able to fix.
In this case, nicking code from Github seems to fix the issue, even if I need to adapt it a bit.
Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates.
> Then comes the licensing issue. Often, when searching for an obscure topic, the code ChatGPT generates is very close to what's found on Github, but said code often comes with a non-permissive license, unlike what the AI generates.
I think this is a feature for a lot of people. ChatGPT can launder code so they don't have to care about licenses.
I feel the same way. Anytime someone says they don't find LLMs all that useful, the exact same comments come out:
"They clearly aren't using the right model!"
"It's obvious they don't know how to prompt, or they would see the value."
"Maybe it can't do that today, but GPT-5 is just around the corner."
I feel more and more that people have just decided that this is a technology that will do everything you can imagine, and no evidence to the contrary will change their priors.
For me llm is faster stackoverflow without fear of my question being closed. I know the first answer will not be what I want. I know that I will have to refractor it to suit my style. I know it will be full of subtle bugs.
Oh and I expect it to be free, I ain't paying for this just like I wasn't paying for stackoverflow.
Finally I hope than in few years I will be able to just "sudo apt-get install llm llm-javascript llm-cooking llm-trivia llm-jokes" and it will all run locally on my low end computer and when I report bug, six months later it will be fixed when I update OS.
You are paying, the same way you paid for stack overflow.. you become part of the process, ask follow up questions, deepening the knowledge of the system.
The same applies to AI. The old learning material is gone, your interaction is now the new learning material and ground truth.
PS: Hourly rates for sw engineers: Range:€11 - €213 - so one hour on stackoverflow, searching and sub-querying resolving problems costs you or your employer up to 213€. It really depends on what you have negotiated.
And unlike Stack Overflow, which is available to everyone online and has an open content license, the IP the users of the ChatGPT style services is entirely proprietary to the company.
I am not interested in feeding their machine with my brainpower. On the other hand, I happily contribute to Stack Overflow, and open source software+hardware. I do not think I will integrate LLM into my engineering workflow until there is a good service/solution which builds up the commons. The huge companies already have way too much influence over key aspects of knowledge-based society.
Every month, every update, every week, every contract change, every day - the setting is in another menu, in another castle, the setting is a princess, kiss the button, the button is now on, now off, is now dark patterned, now it isn't, deliver proof of work to set the setting, solve 1 captcha, solve 20.. come on.. its rodeo time.
I'm with you. Every time I've used LLMs in my work, I've ended up using more time tidying up after the robot than it would have taken to just do the work myself from the start. I can believe that there are some tasks that it can do very fast, but my experience is that if you're using it for anything that matters, you can't trust it enough to let it do it on its own, and so you just end up doing the work twice.
It's like having an unusually fast-but-clumsy intern, except interns learn the ropes fast and understand context.
More likely to be confirmation bias; most of us ask the wrong questions, try to confirm what we already believe rather than choose questions that may falsify our beliefs.
I have some stand tests for LLMs: write a web app version of tetris, write a fluid dynamics simulation, etc., and these regularly fail (I must try them again on 4o).
But also, I have examples of them succeeding wildly, writing a web based painting app just from prompting — sure, even with that success it's bad code, but it's still done the thing.
As there are plenty of examples to confirm what we already believe, it's very easy to get stuck, with nay-sayers and enthusiasts equally unaware of the opposite examples.
I mean when the people saying it doesn't work for them, would kill them to give links to the chats on ChatGPT.com so everyone can see the prompts used? when they do, it's a different conversation. like the number of R's in strawberry, or 9.11 - 9.2. When the complaints are generic with no links, the responses are similarly generic, because both sides just biases that they're right and the other side is the one that's wrong.
I welcome people picking apart chats that I link to. it's not that I believe that LLMs are magic and refuse to adjust my model of how good these things are and aren't, but when people don't give specific evidence is hard to actually move the conversation forwards.
because yeah, these things are plenty stupid and have to be tricked into doing things sometimes (which is stupid, but here we are). they're also pretty amazing but like any hammer, not everything is a nail.
It codes great for me, helps me deliver faster, better tested and more features. It literally saves time and money every day. If you can't do that, maybe, just maybe, you have a you problem. But there are many like you in this thread so you are not alone.
I feel the exact same way. I have felt 0 need to use an LLM in my current workflow. If I could explain concisely what my problem is in English - then I would already know the answer. In that case why would I be asking AI the question. And I don't want opinionated answers to subjective questions. Why run that through a filter, I can investigate better myself through links on the internet.
By the way - I think AI or ML whatever has some valid uses right now. but mostly in image processing domain - so like recognizing shapes in some bounded domain OK yea. Generative image - NOT bad but theres always this "AI GLOW" to each image. Something is always off. Its some neat tools but a race to the bottom and mostly users want to generate explicit content lets be real. and they will become increasingly more creative and obtuse to get around the guards. nothing is stopping you from entering the * industry and making tons off money. but that industry is always doing good.
a friend recently suggested to use AI to generate generic icons for my game. Thats a really good use case. but does that radically change the current economy?
[BTW GENERIC STUFF ONLY UNTILL I could hire someone because i prefer that experience way more. you can get more interesting results. 4 eyes are better than 2.
>> If I could explain concisely what my problem is in English - then I would already know the answer. In that case why would I be asking AI the question. And I don't want opinionated answers to subjective questions. Why run that through a filter, I can investigate better myself through links on the internet.
I am an experienced programmer, but I find myself (for the first time) doing a deep-dive in SQL and specifically building code that will run against multiple SQL engines.
I fed my list of engines into AI. As I'm discovering the really weird corners of SQL I asking the AI to compare one db against the other. My prompts are usually no more than 3 or 4 words.
It gives me quick helpful answers highlighting where things are the same and where they are different. I can then follow up in the specific docs if necessary (now that I know the function name.)
Personally I'm somewhat anti-hype, I'll let others rave about "changing the world". But I have found it a useful tool - not so much for "writing my code" but for acting as my tutor as I learn new things. (I'm using it for more than just SQL or computers now.)
I'm not sure it "changes thr economy" - but it can certainly change individual lives. Some jobs will go away. Others might be easier to do. It might make it easier to learn new skills.
Recently I saw the process of filling for 2 insurance claims through 2 different entities. First one used a great ai voice agent that does the process of filtering your query. it understood me perfectly. But then I still had to wait in line for an actual agent. Ok wait for 6 hours. Whatever. Call again - the 2nd time going through the same agent is painful. Its so slow. And all this just to connect me to a human - who was EXCELLENT. What did ai add? And for now a human has to be involved in any complex permission changes to your account. AND I like that.
The 2nd agency I called them through phone. No ai. But it is excellent cause they do async processing. so you reserve a slot and they call you back. i don't care if those are not answered urgently. Because I just want to talk to a human.
AI lets the agency better be able to afford to hire and keep the excellent humans around to solve complex problems. It's better than a human that also can't fix your issue themselves and has to ask you to transfer to the other person. Ideally, the AI agent would have let you book a callback time with the specific right human who's best able to fix your issue. Some companies are able to do this, and you, the customer, never realize that of the 100 specialists in the building, you got connected to the one person who understands your issue and has the tools to fix it.
Customer service is hard because it has to filter out 90% noise, 9% fairly straightforward tasks, and the <1% of complex issues that need to be sent to the right human.
DONT get me wrong. What you're saying I actually support. If this allows that company to have quality staff I'm all for it. The less time we both spend on each other the better. So just offer async processing.
Your point is that AI is bad at some things, and in some cases misused. Which is of course abundantly true.
But equally it doesn't prove, or even assert, the opposite. A bicycle may be bad at cross-country road trips, but that doesn't make it a bad choice for some other situations.
Hence my earlier comment - I (and I suspect others) are finding it useful for some tasks. That is not to imply it is good at all tasks.
Is it over-hyped? Of course yes. Welcome to IT where every new thing is over-hyped all the time.
I think llms are useful when you're trying to write something in a language you don't know well; then it speeds up the part where you need to check for simple idiosyncrasies.
If you don't know the language at all it's dangerous because you may not understand the proposed program (and of course if you're an expert you don't need it at all).
But llms won't help to find solutions to a general, still unspecified problem.
I've been very productive using LLMs, without any expectations of them "writing functional code". Instead, I've mostly used them as if I was working with a human research librarian.
For example, I can ask the LLM things like "What are the most common mistakes when using the Vulkan API to render a triangle with a texture?" and I'll very rapidly learn something about working with an API that I don't have deep understanding of, and I might not find a specific tutorial article about.
As another example, if I'm an experienced OpenGL programmer, I can ask directly "what's the Vulkan equivalent of this OpenGL API call?" and get quite good results back, most of the time.
So I'm asking questions where an 80% answer is still very valuable, and it's much faster than searching for documentation and doing a lot of comparison and headscratching, and it works well enough even when there's no specific article I could find in a variety of searches.
Anything better that the technology gets from here just makes things even easier yet!
I was skeptical like you, but recently decided to try it out. I wasn't expecting much, and as such I was slightly surprised.
For example, just now my NAS stopped working because the boot device went offline. So I got to thinking about writing a simple syslog server. I've never looked at the syslog protocol before, and I've never done any low-level TCP/UDP work in C# yet.
So I asked ChatGPT to generate some code[1], and while the result is not perfect it's certainly better than nothing, and would save me time to get going.
As another example, a friend who's not very technical wanted to make an Arduino circuit to perform some automated experiment. He's dabbled with programing and can modify code, but struggles to get going. Again just for kicks, I asked ChatGPT and it provided a very nice starting point[2].
For exploratory stuff like this, it seems to provide a nice alternative to searching and piecing together the bits. Revolutionary is a quite loaded word, but it's certainly not just a slight improvement on what we had before LLMs and instead feels like a quantum leap.
This is what the AI is great at - the topics might be obscure to you but they are not that obscure in general, so the AI has had a lot of training data.
I've also assembled a Kubernetes cluster overnight, despite not knowing much about Kubernetes before, and I ran the kubectl files ChatGPT made for me past some devops folks, and it passed the smell test.
I consider much of coding to be a magic spellcasting tutorial - we do conceptually simple things, and the difficult lies in figuring out how to use novel libraries and get them to do what you want.
Edit: After checking out the Arduino sketch, I'd take issue with all the floating point calculations in there - most microcontrollers don't have FPUs, and the performance is awful on 8-bit AVRs. It's not great on Cortex M3s either as all this stuff is done in software, and each FP operation is like a hundred cycles.
I'd definitely try to rephrase the issue with integer math. It might work, but no self-respecting embedded dev would write like this.
> the topics might be obscure to you but they are not the obscure in general
Exactly, it's a very nice alternative to searching the web and discovering new stuff.
> most microcontrollers don't have FPUs, and the performance is awful on 8-bit AVRs.
I used to think like you. But then I realized the Atmega 328p is running at 16 MHz, so even hundreds of . As you can see here[1], it can do 94k double-precision FLOPS, more than enough for simple sketches like this. This jives with the benchmarks I did several years ago.
Sure if I was writing some tight control loop or similar I wouldn't use floating point.
Still it does not sit right with me - it might work, but figuring out the math with fixed point is not that hard, just requires some sitting down and thinking about it.
It's like I have some light fixtures in my attic that are connected with wires twisted together and covered with electrical tape - they certainly work and have done for a decade, but its still not right and I wouldn't recommend anyone do it this way.
Very interesting. A couple of notes here on the C# version.
Its using the old format where the Program.cs file has an actual class, whereas as of .NET 6 thats not required.
You said barebones, but for any real server you would want to use the generic host https://learn.microsoft.com/en-us/dotnet/core/extensions/gen... which gets you a lot of the boilerplate and enables you program to be wrapped in a windows or systemd service.
Finally, parsing can be simplified since ASCII is a proper subset of UTF-8, you can just parse the entire string as UTF-8. IMHO I am disappointed that the AI didn't point that out.
> You said barebones, but for any real server you would want to use the generic host
True, I intentionally said barebones as I wanted a minimal example. I asked it to modify the code to use the generic host, and updated the chat link (so refresh). Keep in mind this is the free ChatGPT, but I still think it did reasonably good. The example compiles as-is, and is very close to functional. I've not used the generic host stuff before either, so again this would save me time searching and piecing together code.
> Finally, parsing can be simplified since ASCII is a proper subset of UTF-8, you can just parse the entire string as UTF-8.
I don't think that would work, because the free-form text message part at the end must contain a BOM if it's UTF-8 encoded, according to the specification. AFAIK you can't have the BOM in the middle of a string.
I’m similar: LLM can improve my productivity by 10-20% when I’m working on something familiar, 30-50% when I’m venturing into a popular domain that I’m unfamiliar with.
I just don’t understand where the hell is this magical LLM capable of generating flawless files or even entire projects that many people are talking about. I rarely accept a large block of LLM-generated code without close inspection, and I’ve ended up with a subtle bug that I wouldn’t have written myself at least ~5 times now. Unless you don’t give a shit about code quality, error handling, proper logging, and subtle bugs, you shouldn’t run LLM-generated stuff without painstakingly reading and fixing everything. Or maybe there really is a magical LLM somewhere.
This is how i currently utilise and view The AI tools. I replaced the googling in my flow with it. It might have taken me 20minutes before to get the boilerplate together, now its a minute. Depending on the type of code legacy overheads you have, results may vary. If you can break your problem down into small descrete bits.
The way the OP here was talking about sonnet being above way above chatgtp in this case, it could be true. Google probably has the largest Go codebases on search to train the AI on higher quiality inputs. Go is a simpler language with less variation over time compared to something like .net also adding to its corner.
ive always been the type of person to use the right language for the each use case. For the last 10+ years ive primarly been building cross platform apps that target every common OS. So these "ai" tools like phind.com give me a universal API interface and generate code which is equiv to a SO answer. They are the ability of an outsourced junior dev who you would never let push code to prod, that doesnt have the language barrier but retains the fake dergee overheads ;)
My experience with CoPilot has been similar. I don't think it's given me a single piece of code that just worked. It always took several back and forth of me telling the "AI" that I need it to do X instead of Y (which was included in the original instructions but ignored).
It seems to work best if I start with something very simple, and then layer on instructions ("now make it do X").
Where I have found it saves me time is in having to look up syntax or "gotchas" which I would otherwise search StackOverflow for. But as far as "writing code" -- it still feels a long way from that.
I have a similar experience, but I still use LLMs, just a bit differently. I pretty much never ask it to generate complex code. I also rarely ask for definitions or facts, cause of the tendency to generate garbage answers. What I use it for is usually tedious stuff that is easy to do, but would take me more time to type, rather than ask the LLM.
For example:
* I need a simple bash script for file manipulation or some simple tasks like setting up a project (example: download a secret from AWS SSM, check if an executable exist, if it doesn't write instructions on how to install it on most popular systems etc)
* I need a simple HTTP API, nothing fancy, maybe some simple database usage, maybe running some commands, simple error handling
* I need a YAML file for Kubernetes. I say what I need and usually, it gets most of it right
* I want an Ansible task for some simple thing. Ansible is quite verbose, so it's often saving me time
* I have a Kubernetes YAML file, but I want to manage it in terraform - I'll then ask to convert YAML to a terraform entry (and in general converting between formats is nice, cause even if you have only a piece of what you want to convert, LLMs will most of the time get it right)
* Surprisingly, it often gets openssl and ffmpeg commands right - something I always have to google anyway, especially openssl certificates generation or manipulation
* Giving it a function I wrote and writing test samples after providing a list of what it should test (and asking if it can come up with more, but sadly it rarely does generate anything useful on top of what I suggest)
A friend, whose SQL knowledge is minimal, used an LLM to query data from a database over a couple of tables. Yes, after a lot of trial and error he (most probably) got the correct data, however the only one being able to read the query is the LLM itself. It's full of coalesce, subselects that repeat the same joins again and again.
LLM will do a lot for you, but I really hate this "this will [already did] solve everything". No, it did not and no, because it's quality is those of a junior dev, at max.
> because it's quality is those of a junior dev, at max.
Echo chamber/bias I guess. I know many, many seniors with big pay checks working for big companies who are vastly worse than sonnet at the moment. Juniors stand literally no chance unless very talented.
I understand a junior dev as someone who's missing (or does not have at a greater level) of maintainability, readability, longevity of a solution.
I don't know what companies pay for and I don't care, because if we go by that, every definition of every word is arguable (because there's everywhere someone out of the range of the definition)
Lots of companies are directly involved with LLMs, or working to leverage it for startups or their existing projects. And I think a fair chunk of all the employees working at these places probably post on HN (a crazy high percent of the recent batches of YC applicants were using LLM stuff, for instance). That's going to lead to a sizable number of perspectives and opinions that are not especially free of bias, simply because you want to believe that what you're working on is ultimately viable.
And I think it'd be extremely easy to convince oneself of this. Look at where 'AI' was 5 years ago, look at where it is today and then try to imagine where it will be in another 5 years. Of course you have to completely blind yourself to the fact that the acceleration has clearly sharply stalled out, but humans are really good at cognitive dissonance, especially when your perception of your future depends on it.
And there's also the point that even though I'm extremely critical of LLMs in general, they have absolutely 'transformed' my workflow in that natural language search of documentation is really useful. Being able to describe a desired API, but in an overly broad way that a search engine can't really pick up on, but that an LLM [often] can, is just quite handy. On the other hand, this is more a condemnation of search engine tech being frozen 20 years in the past than it is about an imminent LLM revolution.
Where LLM shine is as a faster alternative to Google and Stack overflow. "How do I reverse an array in language X?". This will give you the right answer in seconds without having to click through garbage.
Especially if it's a question that's hard to Google, like "I remember there is more than one way to split an array in this language, list them". This saves me minutes every day.
But it's especially helpful if you are working on projects outside your own domain where you are a newbie.
Something worth noting is that the parent comment refers to using Cursor, not ChatGPT/Claude.ai. The latter are general-purpose chat (and, in the case of ChatGPT, agentic) applications.
Cursor is a purpose-built IDE for software development. The Cursor team has put a lot of research and sweat into providing the used LLMs (also from OpenAI/Anthropic) with:
- the right parts of your code
- relevant code/dependency documentation
- and, importantly, the right prompts.
to successfully complete coding tasks. It's an apple and oranges situation.
> but every time I try I find it borderline impossible to get functional code even for pretty straightforward prompts.
I work in 9 different projects now and I would say that around 80% of functional code comes from Sonnet (like GP) for these projects. These are not (all) trivial either; there is a very niche (for banking) key/value store written in Go for instance which has a lot of edge cases etc, all the plumbing (x,err = etc aka stuff people find annoying) comes from sonnet and works one-shot. A lot of business logic comes from sonnet too; it works but usually needs a little tweaking to make it correct.
Tests are all done by Sonnet. I think 80% is low balling it on Go code really.
We have a lot of complex code generator stuff and DSLs in TS which also works well often. Sometimes it gets some edge cases wrong, but either re-prompting with more details or fixing it ourselves, will do it. At a fraction of the time/money of what a fully human team would deliver.
I wrote a 3d editor for fun with Sonnet in a day.
I have terrible results with gpt/copilot (copilot is good for editing instead of complete files/functions; chatgpt is not good much compared with sonnet); it doesn't get close at all; it simply keeps giving me the same code over and over again when I say it's wrong; it hardcodes things specifically asked to make flexible etc. Not sure why the difference is so massive all of sudden.
Note: I use the sonnet API, not the web interface, but same for gpt so...
It was a few hours work so it was not 'done properly', but the point was, it is working code and, as I know nothing about 3d or game programming, I didn't do any of it. People here claim it (current LLMs) cannot produce working code for entire software's without a human doing a lot of it; it clearly can and for non trivial stuff. I would say the real point is ; it cannot do complete code for non trivial programs without human doing most of the coding IF the person prompting is not an experienced coder. I am, so I can review the generated code, prompt in English what it should change and then it works. It is often faster to write code instead of that (so the thing blurbs 100% working code but with 20% wrong/bugs which I fix as a human) but in domains I know nothing about, English is faster for me.
Essentially to me it feels like almost all LLM's are either mid -> terrible if it's any system programming. Especially I've not had luck with anything outside of web programming and moreso anything NOT javascript / python. (jeez I wrote this sentence terribly)
I had a decent experience putting together frontend code for a demo control system with Mixtral within a couple days. I'm a seasoned programmer but I don't do JS. It stumbled a dozen times but it fulfilled the task of me avoiding to learn JS.
However once you step outside JS or Python, the models are essentially useless. Comprehension of pointer semantics? You wish. Anything with Lisp outside its training corpus of homework assignments? LOL. Editing burden quickly exceeds any possible speed-up.
LLM coding assistants are more like another iteration in the jump from nothing -> Stack Overflow type resources than a replacement for you doing coding work as a programmer just because you can phrase the task as a prompt. If you measured the value of using resources like tutorials Stack Overflow posts by blindly merging the first 2 hundred line examples of the things you want to do, finding the result wasn't ideal, and declaring it a useless way to get functional code people would (rightfully) scratch their heads at you when you say they are living in a different world. Of course it didn't work right away, perhaps it was still better than figuring all of the boilerplate out on your own the first time you did it?
For LOVR (Lua based 3D/VR framework), I found these LLMs pretty much useless, both ChatGPT and Claude. Seems all is trained on old APIs, so it takes quite a bit of effort to make any suggestions work with newest LOVR version.
I wonder if quality would improve if one would upload the latest LOVR documentation and uploaded LOVR codebases using the newest version and instruct it properly?
Gpt seems to have gotten worse, Claude is the new hotness.
But, I agree with your sentiment that asking it to do stuff like that often doesn’t work. I’ve found that what it _can_ do is stuff like “here’s a Model object, write a query to fetch it with the schema I told you about ages ago”. It might not give perfect results, but I know how to write that query and it’s faster to edit Claude’s output than it is to write it from scratch.
I'd go one step further and suggest that one of the few things LLMs are good at is acting as astroturfing agents for themselves. I must be crazy to not be completely changing my workflow and job title with [THE_GOOD_ONE] and not that other one, and wow so many other people vocally feel the same way on every other forum that's extremely easy to manipulate.
Fwiw, I've had some helpful successful prompts here and there, and in some very narrow scopes I'll get something usable, like parsing JSON or scaffolding some test cases, which is real saved time, but I stopped thinking about these tools long ago.
To get real value out of something like your example, I'd be using it as a back and forth to help me understand how some concepts work or write example questions I can drill on my own, but nothing where precision matters
It's the same for me. It has to work 100%. I have 15+ years of coding experience, so I can code arguably fast. If I ask the LLM to code something, and it only partially works, then debugging and fixing the generated not-so-optimal code takes me more time than simply writing it from scratch.
There are also some more gotchas, like the generated code using a slightly different package versions than the installed ones.
> I feel like I'm living in a different universe sometimes. The consensus on HN seems to be that you can be pretty productive with LLMs as coding assistants, but every time I try I find it borderline impossible to get functional code even for pretty straightforward prompts.
Same, it can't even fix an xcode memory leak bug in a simple app. It will keep trying and breaking it non-stop. Garbage
> The consensus on HN seems to be that you can be pretty productive with LLMs as coding assistants
If you define "productive" as writing a simple CRUD web application that your 13-year-old cousin could write between two gaming sessions, then you'll consider LLMs as sacred monsters.
Snake oil vendors always had great appeal over people who didn't know better.
Counterpoint: I have gotten real value out of dumping broken SQL into ChatGPT and have it fix it for me. I could 100% have done that by myself, but it would have meant I have to go and google the right syntax.
AI is great for me, but it is more like a junior developer you are pairing with than a replacement.
> I gave GPT two small twists so it's not a simple case of copy/paste
Why? I see it like querying a database of human knowledge. I wouldn't expect a SQL database to infer information it's never seen before, why would I expect an LLM to do so?
I use it where I know a solution exists but I'm stumped on the syntax or how to implement it in an unfamiliar environment, or I want to know what could have caused a bug based on others' experience etc.
what did it do when you told it all of those things? was it able to fix the problems when you pointed them out? did you give it one prompt and expect perfect code out on the first try? is that how you code? all your code complies and runs flawlessly first try? I'm jealous. it usually takes me a bunch of passes before I get things right.
here's a chat for a uc and LCD chip that I picked at random (and got the name wrong for) (and didn't want raspberry pi code for so it stopped it short on that response)
Right now AI is like a chissel. It's a very useful tool, but not useful for everything. Banging your head against the wall of capabilities will give you an intuition when you will pull this tool. Just like you learned how to use a search engine effectively over the last 20 years.
When you are familiar with LLMs, then a question from someone who doesn't use AI is very obvious. It's the same feeling you have when you roll your eyes and say "you could have googled that in 10 seconds".
It's either explaining code where you don't even know the lingo for or what the question could be. Or touching code with a framework you never used. Or tedious tasks like convert parts of text into code or json. Or sometimes your mind is stuck or drifts off. Ask AI for an idea to get the ball rolling again.
Yes, discovering what works and what doesn't is tedious and slower then "just doing it yourself". Like switching IDEs. But if you found a handful of usecases that solve your problems, it is very refreshing.
Try something like maestro which uses agents and orchestrator. The looping quality checks have been very helpful. Claude-engineer from same developer is also good to experience how superior that ia to regular chat.
Yeah, these people claiming AI has been a transformative experience are just full of sh*t. I ask various models questions all the time because it's often better than googling, but all of them make a lot silly mistakes. Frequently, it can be a bit of a process to get useful results.
one trick is to let it come up with what it wants (lots of functions, no texture), then run the code, give it the errors until that's fixed. then ask it to inline them, then add the texture, etc.
I look at this contrast as what I call the difference between a "programmer" and a "software engineer". These jobs are really two different universes in practice, so you're not wrong.
I saw an LLM demo at one point where it was asked to write FFT and add unit tests for it which really drove this point home for me.
A programmer is a nicer term for a code monkey. You ask them to write FFT and they'll code it. All problems can be solved with mode code. They can edit code, but on the whole it's more just to add more code. LLMs are actually pretty good at this job, in my experience. And this job is important, not all tasks can be engineered thoroughly. But this job has its scaling limits.
A software engineer is not about coding per se, it's about designing software. It's all about designing the right code, not more code. Work smarter, not harder, for scale. You ask them to write FFT and they'll find a way to depend on it from a common implementation so they don't have to maintain an independent implementation. I've personally found LLMs very bad at this type of work, the same way you and others relying to you describe it. (Ok, maybe FFT is overly simple, I'm sure an LLM can import that for you. But you get the idea.) LLMs have statistical confidence, not intellectual confidence. But software engineering generally works with code too complex for pure statistical confidence.
No offense to the LLM fans here, but I strongly suspect most of them are closer to the programmer category of work. An important job, but one more easily automated away by LLMs (or better software engineering long-term). And we can see this by how a lot of programming has been outsourced for decades to cheap labor in third-world countries: it's a simpler type of job. That plus the people biased because their jobs and egos depend on LLMs succeeding.
I stop it from coding, just ask it to brainstorm with me on a design or plan. After a few iterations it knows what I want and then I ask it for specific outputs. Or I code those myself, or some combination.
I’ve found that it’s sometimes amazing and sometimes wastes a lot of my time. A few times it’s really come up with a good insight I hadn’t considered because the conversation has woken up some non-obvious combination. I use ChatGPT, Claude, Perplexity and one or two IDE tools.
I just signed up for the free version. Claude Sonnet does properly use malloc/free to manage the buffer where GPT-4o screws up (Yay!) It manages to gets through the whole process of initializing a graphics device and grabbing a queue from the device. It took some questionable shortcuts to get there (and didn't leave any comments explaining those shortcuts and the problems they could cause down the road), but fine, the code works.
After that it goes completely off the rails by trying to issue draw commands before binding a graphics pipeline, which is both illogical and illegal. After a lot of prodding, I did manage to get it to bind a graphics pipeline, but it forgot about the texture.
So Claude Sonnet is definitely better than GPT-4o, but it still feels raw, like a game of whack-a-mole where I can get it to fix a mistake, but it reintroduces an old one. I also have to be the one offering the expertise. I can prompt it to fix the issues because I know exactly what the issues are. If I was using this to try to fill in for a gap in my knowledge, I would be stuck when I ran the code and it crashed - I would have no idea where to go next.
Update: Took about 50 min of experimenting, but I did get Claude to generate code that doesn't have any obvious defects on first inspection, although it cut off about halfway through because of the generation limit. That's the best result that I've seen from an LLM yet. But that's after about a dozen very broken broken programs, and again, I think the domain expertise here is key in order to be able to reprompt and correct.
I agree, that could explain a lot of it. I also suspect that the length of the generated code plays a role. In my experience, LLMs sometimes peter out a bit and give up if the generated program gets too long, even if it's well within their context limit. (Where giving up means writing a comment that says "the rest of the implementation goes here" or starting to have consistency issues.) Python and JavaScript tend to be more succinct and so that issue probably comes up less.
Yes, you have figured it out. LLMs are terrible for graphics programming. Web development - much better. Sonnet 3.5 is the only good model around for now. GPT 4o is very poor.
I've had some moderate success asking Claude to "translate" smallish pieces of code from, eg, C++ to Python. One simple C++ file parser it managed to translate basically 100% right on one try. I wouldn't particularly trust the output- well, not until I run it through tests- but for quick "could this possibly work, what does the performance look like, what might this code look like" exploratory stuff it's been very useful, especially for code that you are likely to throw away anyway.
One example that I am still using: I wanted to generate a random DICOM file with specific types of garbage in it to use as input for some unit tests, and Claude was able to generate some Python that grabs some random DICOM tags and shoves vaguely plausible garbage data into them, such that it is a valid but nonsensical DICOM dataset. This is not hard, but it's a lot faster to ask Claude to do it.
I decided to fire up GPT-4o again today to see if maybe things have gotten better over the past few months.
I asked GPT to write code to render a triangle using Vulkan (a 3D graphics API). There are about 1000 tutorials on this that are almost certainly in GPT-4's training data. I gave GPT two small twists so it's not a simple case of copy/paste: I asked it 1) to apply a texture to the triangle and 2) to keep all the code in a single function. (Most tutorials break the code up into about a dozen functions, but each of these functions are called only once, so it should be trivial to inline them.)
Within the first ten lines, the code is already completely nonfunctional:
GPT-4o declares a pointer (VkPhysicalDevice) that is uninitialized. It queries the number of graphics devices on the host machine. A human being would allocate a buffer with that number of elements and store the reference in the pointer. GPT-4o just ignores the result. Completely ignores it. So the function call was just for fun, I guess? It then tries to copy an entire array of VkPhysicalDevice_T objects into this uninitialized pointer. So that's a guaranteed memory access violation right off the bat.