For anyone living under a rock, there has been major advances in AI technology over the past few years.
The initial release of ChatGPT (November 2022), led to wild claims about the value of AI, and the demise of various industries. As time has progressed, a lot of that hype has so far failed to be realised. Having said that, some of the anti-AI hysteria has also proved to be overdone.
As I write this, know that there is currently a “race” going on where a lot of money is being spent on finding ways to integrate these LLMs into products that serve us better. This post isn’t going to address that aspect (I’m not much of a technology forecaster, what’s going to happen is going to happen, but how it will look is beyond me).
What this post will focus on is how I am finding using AI in my day to day activity as a Software Engineer.
Let me further preface this by saying that I have been writing software (professionally) for the better part of the last 15 years, longer if you count before I went to University.
Because of the length of time that I have been in “the game” I have seen a few things come and go, and I will use that experience to evaluate the AI tools.
I have been using two LLMs over the past couple of months that are free to use, and “do the job” - they are Gemini and Claude - I have not been using ChatGPT, even though they have been”leading the race” and have released a new version (5.0).
The first thing to say, I use AI as a “pairing partner” and to help me “search” for answers - the LLMs have a better Natural Language Processing ability that “understands” my query than a query submitted to Google.
This means that I will get answers that I would get from sites like StackOverflow, or blog posts, or whatever else Google would find for me. The knowledge isn’t “knew”, it’s just (IMO) easier to access and/or find.
Subjectively I can say that I have found Claude to marginally “better” than Gemini, it has been producing less hallucinated code examples for me. I have had Gemini provide me example code that used incorrect types in function calls.
To give a better understanding, I am using TView in my version of Solitaire that runs in a terminal.
At one point Gemini suggested that the function it was proposing I use to solve an issue required a tview.Key, which doesn’t exist within the library. What was actually required was a tcell.Key (defined in TCell).
In general I have been able to do things like “paste all of my project code ino a Claude prompt” and ask Claude “Why is this broken”, and got a lis of issues to look at.
I have also had Claude help me immensely by telling me how to rearchitect my code (after I asked it to), which produced this commit (Note: I edited the code as suggested by Claude, and I found that asking Claude to present it’s suggestions as a diff made my life a LOT easier)
However there have been a couple of notable issues with both Claude and Gemini that I feel need raising. I had the experience of an issue with an incorrect cell being highlighted when clicked. The LLMs both suggested three possible issues, that proved to be (at least partially) wrong.
Further, I would tell the LLMs that they were wrong, and provide evidence, they would claim to accept that they had erred, but, some moments later… give me the same “solution” to the problem again.
Quite infuriating.
I eventually found a fix for the problem I was having, but that was through my own creativity, not due to any help from the LLMs (or Google for that matter).
I have found that I treat the LLMs as “pairing partners” - I can ask them for their input, and they give me their /opinions/ freely. I still have to use my experience as a developer, and my intuition to help me realise whether to use the advice, or not, but I have the same filter working for me when I interact with humans.
One thing that I do find the LLMs are superior than me at is rewording comments, and creating names. I had a linter complain that the comment I was using in a //nolint directive wasn’t good enough - but it gave no indication of why or how to fix it.
The following is the transcript from Claude
can you improve the comment on this for gocritic
// nolint: mnd // The magic numbers in here are too magic.
Edit
Here are some better comments that actually explain why you're disabling the magic number detector:
Option 1: Explain the context
go
//nolint:mnd // Card game constants (suits, ranks, deck size) are self-documenting
Option 2: Explain why constants aren't better
go
//nolint:mnd // Small numeric literals for UI positioning are clearer inline than as named constants
Option 3: Domain-specific explanation
go
//nolint:mnd // Standard card values (13 ranks, 4 suits, 52 cards) are universally understood
Option 4: Performance/readability trade-off
go
//nolint:mnd // Simple numeric values used once don't warrant separate constant declarations
Option 5: Configuration/layout explanation
go
//nolint:mnd // Layout dimensions and offsets are more readable as literal values
The key is to explain why magic numbers are acceptable in this specific context, not just acknowledge that they exist. Choose the option that best matches what your code is actually doing.
What kind of magic numbers are you dealing with? That would help me suggest the most appropriate comment.
I chose the final suggestion, and the linter was happy. I honestly wouldn’t have thought to come up with that.
Hopefully it’s clear to you that the LLMs are usable, and helpful, but only to a point. As a user I still need to understand, and sometimes verify, what the LLM is telling me. There’s no “blind acceptance”. It’s easy, if you don’t pay attention, for the LLMs to create problems.