Looks like a solid post with solid learnings. Apologies for hijacking the thread but I’d really love to have a discussion on how these heuristics of software development change with the likes of Cursor/LLM cyborg coding in the mix.
I’ve done an extensive amount of LLM assisted coding and our heuristics need to change. Synthesis of a design still needs to be low cognitive load - e.g. how data flows between multiple modules - because you need to be able to verify the actual system or that the LLM suggestion matches the intended mental model. However, striving for simplicity inside a method/function matters way less. It’s relatively easy to verify that an LLM generated unit test is working as intended and the complexity of the code within the function doesn’t matter if its scope is sufficiently narrow.
IMO identifying the line between locations where “low cognitive load required” vs “low cognitive load is unnecessary” changes the game of software development and is not often discussed.
With LLM generated code (and any code really) the interface between components becomes much more important. It needs to be clearly defined so that it can be tested and avoid implicit features that could go away if it were re-generated.
Only when you know for sure the problem can't be coming through from that component can you stop thinking about it and reduce the cognitive load.
Regarding some of the ‘layered architecture’ discussion from the OP, I’d argue that having many modules that are clearly defined is not as large a detriment to cognitive load when an LLM is interpreting it. This is dependent on two factors, each module being clearly defined enough that you can be confident the problem lies within the interactions between modules/components and not within them AND sharing proper/sufficient context with an LLM so that it is focused on the interactions between components so that it doesn’t try to force fit a solution into one of them or miss the problem space entirely.
The latter is a constant nagging issue but the former is completely doable (types and unit testing helps) but flies in the face of the mo’ files, mo’ problems issue that creates higher cognitive loads for humans.
> I’d really love to have a discussion on how these heuristics of software development change with the likes of Cursor/LLM cyborg coding in the mix
I would also be interested in reading people’s thoughts about how those heuristics might change in the months and years ahead, as reasoning LLMs get more powerful and as context windows continue to increase. Even if it never becomes possible to offload software development completely to AI, it does seem at least possible that human cognitive load will not be an issue in the same way it is now.
Many would be quick to argue that they are interlinked but I strongly agree with you - particularly in the context of this thread.
For me, optimism still provides a lot of value in my worldview. The beliefs that have changed the most for me in the last 25 years is the limits and boundaries of what tools can do for us.
Technology ‘fixing humanity’ (when that ‘fixing’ is based on your own personal value system) is certainly a fools errand. But that limitation shouldn’t get in the way of imagining a utopia which is worth having.
Sure, I miss my childhood feelings of the 90s. But I also never expected a techno utopia to be ‘easy’.
Ok I got a first cut of Ruby parser in - the latest commit on Rails was a little sluggish but it works!
Right now the parser intentionally tags modules as classes, we made some assumptions in the generic parser code for other languages that don't quite align with ruby's notion of modules, but that'll be adjusted in the near future.
hey this is awesome. I'm using it and I think it's got some wheels. But please add a quick link to submit feedback in app!
Here's what I got so far:
- specific issue: I updated the triage text before submitting to the planning stage. Although the triage text maintained my edits I'm pretty sure the prompt that went into the first Plan generation did not include my edits.
- rough idea: Plans are long and take awhile to regenerate. There's fear of submitting for a regeneration when all you want is it to understand something different about the high level thoughts. Maybe keeping a history of plans or even diffing them could be useful.
Both items above coalesce into a desire to 'rewind' back to the top of the process and try to get the context understood at the beginning, and have that clearly reflected to the user so that as you move through to details it gets easier to handle the nitty gritty without as much editing.
We will definitely add a feedback button, thanks for the suggestion!
We will look into why the Triage edits were not reflected in the plan. And I like your suggestions on being more specific on updates to the Plan.
I have a few suggestions in the meantime that could help to get you part of the way there as we work on these new additions:
- If you select a piece of the plan, you could ask that step to be regenerated based on new edits and then replace that piece either with the "Replace selection with summary" button or copying directly
- You can Ctrl-Z in the document editors to go back. It sounds like this might help a bit with rewinding as you describe
The interactions between Ants and Spiders gave me some associations with Butler's Xenogenesis trilogy (aka Lilith's Brood). Particularly, I loved how both were painting an alternative evolutionary path but 'grafting on' to existing notions and understandings of what we know to be true in species development. I wish there was more of this! I felt Children of Ruin was weaker in this regard, maybe because the conflict for the species was absent. The Spiders vs Ants and then Spiders vs Humans being conflicts which created a fanstatic narrative to explain alternative solutions to prisoner's dilemma (spiders choosing to co-opt their enemies' strengths or in Lilith's brood, Oankali being a hybrid of alien/human). I'd be curious to learn if there's more examples in zoology/ecology of species choosing this route instead of competition every time - and also, what factors might impact this.
I could have written this same comment. I fully understand where the mental irrationality comes from. I’ve done therapy etc. But what I call “the hearth of the city” means it’s hard to ever really relax. Caring for a family accelerated it.
I’ve done an extensive amount of LLM assisted coding and our heuristics need to change. Synthesis of a design still needs to be low cognitive load - e.g. how data flows between multiple modules - because you need to be able to verify the actual system or that the LLM suggestion matches the intended mental model. However, striving for simplicity inside a method/function matters way less. It’s relatively easy to verify that an LLM generated unit test is working as intended and the complexity of the code within the function doesn’t matter if its scope is sufficiently narrow.
IMO identifying the line between locations where “low cognitive load required” vs “low cognitive load is unnecessary” changes the game of software development and is not often discussed.