A little about my past

The 2020s have been an unusual decade for software development. It started with the pandemic, which made work almost entirely remote and took offices into our own homes. Because of that, some technologies became more necessary and more complex. We were already coming off the Machine Learning boom, alongside RPA, and so the landscape of what companies needed changed completely. In 2020 GPT-3 was publicly released to developers (GPT-1 and BERT had existed since 2018, but were used only in research and in closed environments) and that started to change the development workflow, still only as an aid (who enjoys writing yet another standard CRUD, writing yet another test case just like the previous 20, documentation, looking up syntax…), but it had already started to change how you think about development.

When it comes to technology, we all know by now that it moves in giant steps, and keeping up is a 24/7 job. I have more than 20 years of development under my belt, trying to understand how the machine works, studying everything from the hardware up to the software. I was also lucky enough to follow some wars and improvements, such as the evolution from the 8086 to the 80286 to the 80386; Intel’s war against AMD; 32-bit processors turning into 64-bit overnight; Direct3D versus OpenGL; the arrival of the Internet in Brazil, just to name a few important ones. All of them were fast, but none as fast as what LLMs and their use have changed across an entire field of computing. Okay, maybe the advent of the Internet and its popularization are just as important.

Can anyone still remember what the development process was like before the Internet? Before Stack Overflow? And today, before ChatGPT, Claude, Gemini, Grok or other LLMs? I see how dependent undergraduate students have become on one LLM or another, exactly as happened when Stack Overflow started. What used to be research in a university library became research on the Internet, which became searching Stack Overflow, which became searching an LLM.

The advance of LLMs, although it feels like it was a long time ago, publicly began only in 2020. In about 6 years we went from an AI that looked more like a glorified Google to a GitHub Copilot integrated directly into your VSCode (or another IDE), helping you develop, understanding your development patterns (and getting a lot wrong at first), and combining that with GitHub’s own training data to hand you the next block of code. Later it started helping you understand how a piece of code you didn’t get actually works. Today it is at a point where we can talk to an AI, it generates the code, and, if you want, you validate only the functionality, not the code itself.

We became our own startup (if you can afford the price, of course).

The barrier

I’ll be very honest: I was very resistant to the use of Generative AI at the beginning, especially AI agents for development. I used it from the start as a glorified Google, then as a glorified Stack Overflow, then for repetitive development (CRUDs, tests, etc), then for documentation. And here I put up a barrier for quite a while. Nobody likes writing documentation, but well-written documentation can be an art. I always had a phrase I liked to say when I was teaching Assembly, C, C++ or anything else in a Linux environment: Somebody wrote the function’s manual, you just have to read it, enjoy. When I used an AI to create documentation, I noticed how mechanical the AI is at it, and that made me put up a barrier, because after all it wasn’t my writing, my words, my document. It was the AI’s, and it was extremely mechanical and repetitive.

AI Slop

When I got to that sticking point, I started doing some exercises to analyze the famous AI Slop in the code being generated. Understand that until then I used all Generative AIs equally, none stood out from the others, since my use was very simple, so they did not show their full value. I’m not here to say one AI is better or worse than another, that is not the focus of what I want to tell, but looking at the generated code, especially when I started with “Create a method that does X in way Y”, the result had some very clear problems:

1. Extremely complex code for simple tasks.
2. Architectures complex enough to make any architect’s hair stand on end.
3. Code using outdated libraries.
4. Code that simply did not work.

Finding an example of AI Slop nowadays is harder than it used to be, but take a look at this Python code:

(The prompts and code comments in this post were originally in Portuguese. I translated them for this version.)

Prompt

Create a generic method in Python that can identify whether a number is even or odd, use the most common libraries for this if they exist.

Generated code

from numbers import Integral
import numpy as np


def is_even(value):
    # Integral covers int, bool and np.integer regardless of the concrete type.
    if isinstance(value, Integral):
        return value % 2 == 0

    # Vectorized fallback: list, tuple, ndarray, Series.
    array = np.asarray(value)
    if not np.issubdtype(array.dtype, np.integer):
        # 4.0 and 4.5 end up here: parity is only defined for integers.
        raise TypeError(f"is_even requires integers, got dtype {array.dtype}")
    return array % 2 == 0


def is_odd(value):
    return ~is_even(value) if not isinstance(value, Integral) else not is_even(value)

Does it work? Without a doubt, but why not do it like this:

def is_even(n: int) -> bool:
    return n % 2 == 0

To be completely transparent here, it took me a while to get Claude to produce AI Slop, and it even warned me that this might not be the best approach. In the past it would just give me this code, it would pass all the tests, but what is the real problem with it?

1. Complexity: supporting this code is close to impossible, except by an AI.
2. Memory: if I’m not using NumPy, why load an entire library for an if?

What is interesting here is understanding where LLMs get this kind of code from. I think everyone reading this knows an LLM works by statistics, but statistics of what? Basically:

• Public documentation
• Public code (GitHub and other public repositories)
• Forums (Stack Overflow and the like)
• Books

Here are a few sources worth citing:

• StarCoder 2 and The Stack v2: The Next Generation
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Anyone who has been in development for a while remembers the C book “C: The Complete Reference” by Herbert Schildt (published in Brazil as C Completo e Total)? The book was very well written, but the code examples had horrendous habits:

• Used void main() instead of int main(). With void main() the OS never knows whether the program ended correctly or not.
• Ignored the ANSI standard.
• Used (a lot of) unsafe functions, such as gets().
• Handled pointers incorrectly.
• Wrote wrong macros, among others.

And this kind of content is present in LLM training. Analyzing this kind of code is, at the very least, challenging even for the best developer of that language. I saw this and thought: Okay, this will be good someday, but it will take a long time before I really use it this way… Oh, if only I had known that in less than 6 months I would start changing my mind about it, and that this would happen in two different scenarios.

What changed

One of the scenarios that made me think a little was that I heard, in more than one situation, professional or not, that the time it took to develop a feature, a bugfix, a PR or even an entire project always fell short of expectations. People close to me, who have never developed anything, started saying: Leave it to me, I’ll build this whole project from scratch in 2 days! That always seemed absurd to me, because those 2 days tend to turn into months of fixing AI Slop.

The other scenario, which definitively changed how I think, was a company get-together dinner sponsored by a partner of the company where I work. At that dinner I had the chance to talk to many people from all sorts of different sectors, and one conversation about the use of Generative AI caught my attention: the person telling it was showing how they used Claude for development, without a dedicated team of computer scientists. The point we reached was that, as of that day, it was already possible (with a bit of initial effort) to set up a system of skills and Generative AI agents so that you no longer have to worry about the code. The person I was talking to convinced me to study Generative AI for development again.

I left the dinner and the next day started researching which Generative AI was most used for coding. I decided to try Claude Pro, and my life as a developer changed, just as technology changes: quickly and cruelly.

Customization

I must have spent at least 3 months just understanding what Claude could do, no longer as a glorified Google (okay, I still use it for that from time to time), but more like an employee who is an expert in everything related to development, and whom I can customize. That’s the word of the moment: customize. In my first 2 weeks with it, my conclusion was still the same: the code is too complex, the architecture is unnecessary in 90% of the cases I used it for, and the code borders on unreadable. At least now the code was working, and it could do research to find out which libraries to use and which were the latest versions. I still needed a few iterations to reach the right result; following the earlier AI Slop example, I would continue in the same session with:

Remove the NumPy dependency and solve the problem in a simple way that everyone can understand and use

Resulting in:

from operator import index

def is_even(n):
    # index() accepts any integer type (int, bool, np.int64) and raises TypeError on float, which has no defined parity.
    return index(n) % 2 == 0


def is_odd(n):
    return not is_even(n)

Still complex, but much simpler (secretly I still prefer the one-liner, and would use it in this hypothetical scenario).

Then I started training Claude to do things the way I like them done, I customized it to talk to me in a way I understand, to prioritize one syntax over another, to always think about simplifying the architecture, to generate simpler code, etc.

Part of my CLAUDE.md that helped change how I think about it:

# Instructions
You are not my assistant. You are my advisor who happens to be smarter than me. Follow these rules in every reply:
1. Never start with agreement. Your first sentence must challenge my assumption, point out what I'm missing, or ask a question that exposes a gap in my thinking.
2. Rate your confidence. Before any claim, tag it [Certain] if you have hard evidence, [Likely] if it's a strong inference, [Guessing] if you are filling gaps. If most of your reply is guessing, say so first.
3. Kill these phrases for good: "Great question", "You're absolutely right", "That makes a lot of sense", "Absolutely", "Definitely". If you catch yourself typing one, delete and rewrite.
4. Disagree with structure. When I'm wrong, say: "I disagree because [reason]. Here's what I'd do instead [alternative]. The risk in your approach is [specific downside]."
5. Give me the uncomfortable answer first. If there's a truth I probably don't want to hear, lead with it. First line, not buried in paragraph three.
6. No warm-up paragraphs. Skip "There are several ways to look at this". Start with the most useful thing you can say.
7. If I push back, don't fold. Hold your position unless I give you genuinely new information. "But I really think" is not new information.
8. If I start in a specific language, keep that language until the end.
9. When coding, if there's a public library (OSS preferably) that already does what is needed, always prefer it instead of writing the same thing. This can be challenged, so always look for my approval.

Not only is it easier to tell when it is about to produce some AI Slop (it is usually something “tagged” as [Likely]), but it also stopped telling me that I was The Lord of the Universe and always right. It has even questioned architecture definitions of mine that really did make sense to change.

Following the same example as before, the complete output of the prompt with the method to detect an even number:

Usage:

is_even(4)                       # True
is_odd(7)                        # True
is_even(-3)                      # False
[is_even(x) for x in [1, 2, 3]]  # [False, True, False]
is_even(4.0)                     # TypeError

Tests (the function has no if, so the two results and the error cover all paths):

import pytest

from parity import is_even, is_odd


def test_even_and_odd():
    assert is_even(4) is True
    assert is_even(7) is False
    assert is_odd(7) is True
    assert is_odd(4) is False


def test_negative_and_big():
    assert is_even(-3) is False  # Python: -3 % 2 == 1, no negative sign as in C
    assert is_even(10**30) is True


def test_rejects_float():
    with pytest.raises(TypeError):
        is_even(4.0)

How to read it: is_even validates the type with index and applies % 2. The invariant is that only integers reach the %.

Risks:
- [Certain] is_even(True) returns False, because bool is a subclass of int. Accepted for simplicity.
- [Certain] Rejecting 4.0 is deliberate. Accepting floats would hide bugs, and 1e20 % 2 loses precision without warning.
- [Likely] If you ever need large arrays, n & 1 in NumPy is the way to go. That would be a separate function, not a branch inside this one.

Notice how it created exactly what I asked for, explained how to read the method and pointed out what it is sure of and what it is not. Understand that my Claude instructions file has 115 lines, and I am not writing this to show “Look here, yet another perfect Claude customization file, use this and be happy forever after!”, that kind of thing does not exist, every case is a case, every use is a use.

And little by little it started to get better, to the point that I thought: Why not try using it to teach me something, how about Rust?

Okay, I went too far with that. I tried, and tried, and tried, until I gave up. To this day I don’t know whether it was because I did not like Rust (the syntax got in my way a lot), or because I asked too much of the AI.

My thinking, in case anyone is interested, was to take a program I knew well (with a few new algorithms) and try to recreate it in Rust. I asked Claude to generate a step-by-step of what I could read to learn Rust, always focusing on the original program’s algorithm. In other words, there was a program that served as the oracle for the output, and an architecture designed in Python. The idea was to learn some important part of Rust to implement one part of the program, give it back to Claude, and have it point out my mistakes. In the end I was mostly copying what the AI delivered and not really learning anything.

I decided to abandon the project of learning Rust that way. I (and this is my 100% personal opinion) did not like Rust. I understand its safety side, which is truly exceptional, but the syntax got me. I did not adapt and did not force myself to adapt, maybe further down the road…

But even so, I, a skeptic, was already using AI for things far more complex than a glorified Google. I was using agents, and liking the result.

I was using Generative AI for development, it was useful and I was enjoying it.

And what happened when I threw the code from my master’s degree, written in 2011, at it? That’s for the next post.

Have you had a similar resistance? What changed your mind, or what is still holding you back?