The first exposure I had to software testing was in a Data Structures class in 2021. We were implementing language features such as Sets, Maps, Heaps, and Stacks in Java.
The tests simulated functionality of performing operations that were known and defined. For example, a heap may have functions like heapify(), push_heap(), getMin() that are well defined. You could draw out a heap and understand what these functions did.
We didn’t get all the unit tests—that would’ve been too easy—so students often came up with their own test cases. Some of the best examples tested an entire range of integers, randomly generating millions of numbers to ensure every possible case passed. I will find an example when I have time, but they seemed to test properties rather than asserting one-off examples.
Having that degree of test consistency is something I struggle to find in modern software today: in work environments, people would just try to reach 80% test coverage by adding random things. You might have a component that says a specific line, and there’s a Jest unit test asserting specifically that line which breaks if you were ever to change it. Annoying, right?
Two types of software development
Exoskeleton
Test cases are like an exoskeleton for your software: it will grow to fit the shape of those tests, and it is quite painful to edit or modify that skeleton. You might have pre-commit hooks that run tests or CI/CD pipelines which yell at you, and you have to dig around just to change a single line of copy if your tests are brittle.
This annoyance reminds me of using commitizen: I could quickly type in the terminal git add . and git commit -m "CONVENTIONAL_COMMIT_HERE" with a conventional commit but using commitizen is slow: you have to cycle through menus and pick the right one.
Even choosing between prefix types is a mental overhead! I want it to be as simple as typing “commit.” The problem, I think, is that Github doesn’t show the files changed, so we have to add things into the commit message such as the file name (or the ISSUE-123 number, which is duplication because we probably already have a branch ISSUE-123/whatever-feature-name) just so we can keep track of things.
You could in theory go from a very well-defined PRD to a TDD with all the test cases laid out first. It’d require a large amount of planning with the spec laid out, then the tests are written and everything is set, and then then you start writing code.
I think it makes sense to have many test cases when something is already established: it’s a production API or there are known properties you are implementing. But in many cases, it seems to slow things down as people fake the idea of code coverage.
Tests are bad for marketing pages where the copy could change at an instant, but good for apps with core functionality.
Jelly
Jelly-like software development is soft and fleshy. It’s what I do when just programming by myself, and it’s something I’d like to replicate in a team environment too. Instead of everyone coming to morning standup feeling they have to make a list of things so it seems like they are productive, it would be better to align the financial incentives and to ensure people feel like they are connected with their teammates and have trust to ship a good product.
When we look at the created world which is done by an individual, it looks so:
Think something -> do it -> the constructed world has changed
For groups, we do this, which is more rigid:
Member thinks something -> documentation -> prodded to do things during standup -> coordinated action
But it could also be like this:
Member thinks something -> words, thoughts, ideas, and communication -> group feeling and trust -> coordinated action like a Navy SEAL team
I don’t believe a stand-up where people feel like they are herded like cattle is conducive to trust. You probably need to keep the meetings small: otherwise some people are just spectating things not relevant to them. If somebody is acting like the leader but the authority is not respected, this will also ruin group feeling.
So in this case, somebody would see something that needs to be done. They just go talk to other people or send a Slack message, quickly check out the branch instead of getting blocked by tests/processes, and commit the change.
The problem I am seeing not just in software testing but in society more general is that the material constructs and manufactures and rules constrain the people when it should be groups of people exercising agency over their own lives. Cars and highways contort our ways of being, regulations prevent things from being done.
Of course, I am speaking from my own perspective: some other people need these rules to fit into some bureaucratic structure to feel safe and that they have a place. The battle of our age is between the top who wants to do more in the world and the middle that wants to hold them back, is afraid, and reduce any sort of variance.
I think these constraints also often happen in places that are large and empire-like because they need some way to scale, and trust is hard to scale.
There may be some relation of this topic to formal verification:
I don’t even think most hardware even is “formal” and “set in spec.” Look how many new models are constantly released. It’s just that engineering requires more setting of the specs in advance.