Quick tips on LLM-based coding

Keeping this one somewhat short.

If you've seen this video, you know what the image above is based on.

If you have experience writing software, you know they're full of it.

But one thing they are right about is there will be a new normal. It's just... that's been the modus operandi in the software world since, literally, its conception:

1. We used to compute things mechanically.
2. We used to program by physically configuring wires or punching holes in cards.
3. We used to have no better tools than writing machine instructions.
4. We used to have to compile our code for every target machine.
5. We used to not have intellisense and autocomplete in IDEs.
6. We used to not have good environment virtualization.
7. We used to develop with waterfall style process.
8. We used to check out files with a lock shared over the network.
9. We used to not be able to run tests automatically in the cloud.

The list goes on.

Now, we used to have to write every line of code manually, or copy it from someone who did.

So what's the new normal going to be?

Well I'm not a world-renowned LLM researcher, or some OpenAI CEO, or anything like that, but I am an early adopter and have spent concrete effort finding techniques that act as force multipliers for LLM based programming.

Ways of thinking about LLM code tools

Yes ✅

• It is a rhetorical calculator: You have to understand why your input leads to the right output, otherwise it might be crap and you'll never know
• In-IDE LLM tools are a litmus test for code quality, meaning it's much more likely to produce good output if your scaffolding makes it inevitable. Hate to break it to you, but if your code is not especially bleeding edge research and you still don't get good suggestions, it's probably because your code is smelly and muddy.
• It is an extra pair of hands. If you know precisely what needs to be done and can succinctly articulate it faster than you could code it, congratulations: your job is nearly complete.
• It can increase the quality of your code. Since you now have extra hands, you can get the same job done much faster, and without skipping all the best practices you've been avoiding all these years because of how long it takes, and how little value they deliver. Bonus: those best practices improve the quality of the LLM output, so not only is your quality better, but you get even more out of your force multiplier. It's exponential.

No ❌

• It won't think critically for you. It is great for riffing because the model can quickly compute rhetoric related to its input, including some rote fact recall. It thinks "fast," not particularly "well."
• It is not an agent with any goals, interested in producing good quality – the onus is entirely on you to "innervate" its embeddings to produce desirable output.
• It is not a domain expert. Every single researcher who writes software will tell you it does not perform well when you get deeply into domain expertise
• It can not code for you. You still need to be the expert, sorry CEOs who think passing an exam maps to real-world value.

Techniques that work well

• Conceptual Scaffolding: Focus on naming things consistently and cleanly, and scaffold your functions and types with comments and documentation before implementing. This doesn't mean over-comment everything a la Java OOP and Doxy comments. Remember: scaffolding is removed when construction is done. Instead, it means scaffold functions with steps, convert each step to be self-documenting code, and leave comments to explain the why.
• SOLID: If you follow these principles, you make good code nearly inevitable, no matter who writes it. Think of your new job as setting up an environment/system/architecture where quality is inevitable, and even the error-prone LLM can't fail.
• Test Driven Development: Even in a good environment, LLMs introduce bugs and issues. This is true for both product code and test code. However, the intersection of bugs introduced in either product or test code has extremely small overlap. Thus, TDD is effective because test code catches bugs in the product code, and product code catches bugs in the tests. LLMs tend also to be quite good at expounding on patterns, and unit tests should be succinct, so it can quickly throw out several tests for different combinations. Stuff you know you should be doing, but don't because it takes too dang long – now it's magically (nearly) free.
• Assume someone already solved it: Let's just admit that most of the code we write isn't doing something truly novel. Turns out LLMs are best at doing that boring stuff anyways. If you don't already have a feel for when "there's got to be a tool or best practice for this kind of thing," start developing that. That already helps you re-frame hard problems into solvable questions. It also has the benefit of leading to better LLM prompting.

Things to avoid

In general, if Uncle Bob and the Craftsmanship Crew (their rap group name) say it, you should probably do it – and doing so will almost definitely improve LLM output.

Here are a few examples:

• Cyclomatic complexity. Your control-flow graphs should not have a bunch of branches. Use CNF whenever you can, avoid using else in many cases, have a "happy path proceeds and bail if failed" function layout, etc.
• Long functions. As a rule of thumb, if you or another developer are more likely to get lost in the context of a function, and if every industry "best practice" leader says it's bad, then it's probably going to reduce LLM output quality.
• God objects. One key to good results is to cleanly break down problems and limit scope of responsibility. More discretely, the scope of a god object's influence is too great to fit into (current) LLM tools' context.
• Smelly code. You're probably picking up on the pattern here. Avoid code smells and follow best practices. Maybe someone is an immortal genius who can navigate the messes and still deliver value, but most of us are mere mortals, and it turns out LLMs are more like us mere mortals.

How I know I'm right

I'll write about this later. For now, "trust me bro."