Introduction
Over the last couple of years in my journey as a developer, I have come across two books, Thinking, Fast and Slow and A Philosophy of Software Design, that have given me insights into why refactoring works and why it’s ignored. I have seen my fair share of codebases where programmers (myself included) have tactically gone in and made changes with no forethought about improving the codebase for other developers. It happens to the best of us. We agree that we should want the best for our codebases, but there are psychological forces that make this virtuous task extremely difficult. We will look into those now from the perspective of these two books.
Thinking, Fast and Slow
Daniel Kahneman is an Israeli-American psychologist who is best known for his groundbreaking work on the psychology of judgement and decision-making, in addition to his seminal contributions to the field of behavioural economics. His book Thinking, Fast and Slow gives us the psychological framework to understand why refactoring works and why it’s ignored.
Kahneman invites us to think of the brain as running in two fictitious modes: System 1 and System 2. He describes System 1 as the fast and intuitive aspect of the brain. It operates automatically and quickly, with little or no voluntary control. You can think of it as your gut instinct, like when you just know something is off but you can’t put your finger on it. Or when someone has practiced chess for so long that they just intuitively know the best move without thinking hard. System 2, on the other hand, is slow and more deliberate. When you have to add the numbers 1337 + 77 (difficult to do with System 1), you’d switch to System 2, where you have to think more carefully and deliberately. We devote attention to the effortful mental activities that require it, such as complex computations like the one above. In doing so, we expend energy, and it’s easy to see why this leads to burnout and why it makes us want to operate in System 1. System 2 is the one capable of reasoning, being cautious and weighing options when making decisions. However, it tends to be lazy for a lot of people. Most things in nature seem poised to take the path of least resistance.
So how does all this relate to refactoring? Most developers, myself included, operate mostly in System 1. It’s the territory of the flow state (when work feels effortless). I like to be in the flow state, when what I’m doing is just within my grasp and I’m able to do it with ease. It’s a good feeling. You may find yourself switching to System 2 when the task becomes unfamiliar, but it’s still easy to fall into old habits and judge the situation like ones you’ve encountered before. This leads to overconfidence and, in turn, to bad designs and implementations. Many overconfident people are prone to placing too much faith in their intuitions (System 1). They find cognitive effort (the opposite of flow) at least mildly unpleasant and avoid it as much as possible. When in a good mood, such as the flow state, people become more intuitive and more creative, but also less vigilant and more prone to making logical errors. It doesn’t stop there: being hungry or in a state of ego depletion (lacking motivation) further compounds intuitive errors and diminishes performance on tasks.
Now we will turn our attention to A Philosophy of Software Design, which applies these intuitions.
A Philosophy of Software Design
A Philosophy of Software Design is one of the most relevant books on the topic of refactoring, as it is intended to be read by software developers. My key takeaways are the concept of designing something twice, the nature of complexity, and tactical vs. strategic programming.
Ousterhout believes that our first attempt at a solution as programmers is never our best. He elaborates that when designing large software systems, no one is good enough to get it right the first time, no matter how smart they are. Many smart people are overconfident and think they’re done at the first implementation of their idea. This causes them to underperform and miss their true potential (which makes them difficult to work with as well). They may subconsciously believe that “smart people get it right the first time”, so if they were to attempt multiple designs, they’d be considered not smart after all. That’s indeed a dangerous fallacy to hold and is detrimental to a dev’s overall growth and improvement. Designing and implementing something twice not only improves your design but also enhances your design skills. This has a huge payoff for you in the future, as it will accelerate your intuitive (System 1) ability to tell good designs from bad ones.
Ousterhout describes complexity as anything related to the structure of a software system that makes it difficult to understand and change. One of the consequences of poor design in a codebase is the accumulation of complexity, which, if not addressed, makes the code brittle and difficult to work with. It leads to high cognitive load, which refers to how much a developer must understand to be able to make a change. This is System 2 territory because it forces us to think hard and can potentially lead to burnout. Complexity accumulates as developers continually favour tactical programming over strategic programming. Tactical programming lacks consideration for the future and is done ad hoc. Strategic programming, on the other hand, applies forethought: you make life easier for future developers.
Putting it all together
Thinking, Fast and Slow showed us that the brain operates in two modes, System 1 and System 2, with System 1 being responsible for our faster, intuitive responses and judgments. It identifies that the part of our brain responsible for slower, more deliberate work is lazy in a lot of people. People use System 2 less because it can lead to unpleasant feelings of exhaustion and ego depletion, which diminishes performance on tasks. System 1 is known for feelings of ease and is associated with the flow state; most of us would prefer this to thinking hard.
In A Philosophy of Software Design, we then see how the effects of System 1 play out to impede developers’ ability to write their best code. We see that developers want to jump to their first intuitive solution. Ousterhout suggests that without refactoring, our work is not complete. Avoiding refactoring eventually leads to complexity in codebases, which leads to developers relying on System 2, increasing cognitive load and burnout.
Refactoring is a virtue, but there are psychological forces that make it difficult. I’m just as guilty of sometimes operating in System 1. I acknowledge that it’s not just psychological forces that make the task hard; companies’ priorities can stand in the way too.