I think at this point we’re familiar with what’s going to come: I’ll ramble vaguely about the topic to hopefully pique your interest while I make self-deprecating jokes about my Schopenhauer-esque humour.
It’s a pretty effective protocol that maybe doesn’t work wonders for SEO, but certainly does for getting me in the flow for the text I’m about to write, which is good enough.
The major flaw of Information Systems
Every Information System has a flaw, but before we get to it I think I must establish a common language for us to understand each other. The astute reader will notice that this article is tagged both as non-code and code at the same time and this gives you a clue that what I’ll talk about here might expand beyond the frontiers of software development. As for the definition of Information Systems, I’d like to start with a basic setup: Any structure through which information is conveyed. That might sound too abstract at first, but it is the right foundation for our discussion. Then I want to clarify what I mean by information: we start with a small particle, a fact. A group of unprocessed facts is just data. Data, when processed and interpreted becomes information. Information, when curated, analyzed and evaluated becomes knowledge.
So an Information System is hierarchically superior to a data system. It is one which data has meaning, context, importance. And as a structure through which information flows, it communicates. So this category of human-built constructs encompasses anything from software to laws, moral codes, cultural values, etc - quite a big range.
They all suffer from the same issue: They’re built without documented justification.
The decadent mind of the forward thinking modern human
We’re living the era of “move things and break fast”… I mean, “move fast and break things”. And, as I hinted in the previous article, doomed by spending too much time looking at the future - and too few at the past. I do want to distance myself from the saudosismo sentiment that this article might infuse, but in times long gone we used to build wonders and produce unfathomable knowledge with a fraction of the tools we have available today. We’re seeing the consequences of hyper-specialization as a resonant wave from the industrial revolution in the erosion of intellectual reasoning and the mechanization of what once was intellectual work.
This leads to an inevitable situation in which our Information Systems exist and not a single person understands the problem they’re trying to solve. More often than not, we’re incapable of determining whether they are - or remain - valid solutions to those problems. On rare occasions where we do, assessing by how much poses itself as a challenge.
This is not an inherent problem to software engineering - but conversely we have our own manifestation of this event that is independent enough to be named individually. I bet you’ve come across laws, rules, cultural norms, company processes or ceremonies that you’ve failed to understand why they were needed and there wasn’t a single person directly involved in making it happen that could explain that context. And this problem exacerbates over time, naturally, as new people step up to take charge of such long lived Information Systems with plenty of tacit knowledge and empiric education, while a formal historiographic education would be impossible as such documentation is non-existent and those who’d know it by heart are long gone.
Our own little monster
Of course I need to be trendy and dunk on our favorite scapegoat, the agile methodology. Truth be told, it’s seldom the original idea that is criticized and I wouldn’t be starting that flamewar here. But the manifestation of that bigger problem, in the context of software engineering, is symbiotic to the growth of agile methodologies. It is precisely when companies feel tied by bureaucratic processes (which, funnily enough, they didn’t understand why those processes were needed in the first place) that an alternative where more freedom and velocity are offered shines through. The cost is naturally disregarded because few would, in fact, understand and argue for the value of that kind of documentation.
Are we doomed then?
No. There might be systems that do, in fact, have a strict set of documents, have periodic follow ups, review their processes, revalidate their raison d’être, measure their objective impact, etc.
I would estimate that few contexts even demand that - think of military, avionics, or life-support - of their Information Systems and, although not all players would implement them correctly, many such cases would exist.
The word of caution here though is that I’m yet to witness one such Information System, but logically speaking, the absence of those properties would have catastrophic outcomes so I’m tempted to believe that we do know how to do it.
Also, minor instances of those beneficial properties exist in things like ADRs or well-written git histories so it is not like we live in complete chaos - my writing would sound more alarming without this section but I’m bound by my personal values to be fair and truthful, which would make me a broke journalist, but I can still sleep an night.
So, no, we’re not doomed. We know how to do it, we know it is valuable. But the demand for velocity and for high release throughput puts us in a situation where this becomes second nature, whereas I believe this should be the core of our craft.
So, is it time to start a counter-revolution?
No. Absolutely not. Here, you see. I don’t think we should throw our agile methodologies on the pyre and reestablish waterfall as our lord and saviour.
I think - and thinking seems to be getting harder and harder these days so I’d be happy with some appreciation - that we need our technological renaissance. In times where AI is challenging assumptions we once held as solid, assumptions we wouldn’t have dared to question a few years ago, it is precisely a dramatic increase in depth (of thought) that we need to survive as a healthy industry. And if it wasn’t obvious before, I’ll say it with all the words now: I do think our craft has become too shallow in thinking. We excel at using frameworks but even identifying the patterns in the requirements that would demand those same frameworks is getting even harder by the day.
And even if the answer was that we needed to fight back “violently” against this movement, the harsh truth is that there’s far too little one can do from their home-office writing on a dark-themed blog in some corner of the web.
It is not going to be the first time - nor the last - in history where we see the world turn to a suboptimal route because humankind picked the wrong option.
History is the image that forms from the patterns of the crowd’s resonant noises. When I chose to be dissonant, I know I’ll hardly change that picture. Yet, there’s a chance more people feel connected with this message.
There’s also a chance that I revisit this, years in the future, and laugh at my own bitterness. Alea iacta est. :x