I often find myself in conversations with customers and prospective customers regarding how we build software. How do we build software? What’s our process? How are we tracking tasks? What documentation do we create?
With some customers, we get a lot of pushback on the lean, fast process we use. According to these people, we don’t generate enough documentation. We don’t do enough manual testing. We start coding too soon. I’ve observed an interesting quality to these conversations: The person (or people) questioning our process is most familiar and has extensive experience developing software at least 20 years ago. Equally importantly, they no longer develop software, either professionally or casually.
Like all conversations, everyone brings their own perspective to the conversation. A couple decades ago, our tools were different. That meant a different process was better. These customers are bringing that natural bias to the conversations, and it comes out in their questions. Knowing where that bias starts makes it easier to explain the differences in a positive way.
Picking up that new codebase today
Let’s start with how today’s tools support and enable our current process. Suppose I was handed a large codebase we’ve created, and I needed to make some changes. The first thing I would do is to get that project from source control, build it, and run the automated tests. That would give me the following information: I’d expect all the tests to pass. If they didn’t, we’d have a serious problem. I’d expect that the tests cover a strong cross-section of the codebase. (I wouldn’t expect that we’d have 100% coverage, but I’d expect something in the high 70s, at a minimum).
Already, I’ve got a level of confidence. If I start making changes to this codebase, without doing any deeper investigation, I would expect that the automated tests would alert me if I’d made any change that introduced a regression bug.
Read that sentence again, because it’s a huge confidence builder for a new developer on any project: As a developer, I will know if I’ve broken anything even before I check the code into source control.
Next, I’d string trying to find the area of the code I need to modify. I’d use the search function in my IDE to find classes, methods, or modules with names that make sense. I’d pay special attention to tests the exercise the feature I’m going to work on. I’d read the test code. I’d use the “go to definition” feature of my IDE to find the code being exercised. I’d learn about the sections of code I’d need to modify.
Once I felt a bit comfortable, I’d start writing new tests to express the changes I would make. I’d leverage intellisense to see what the capabilities of the types I’m using are. I’d expect some reasonable method names to help me understand what’s already implemented.
Overall, I’d feel reasonably comfortable making changes and adding features within a day. I’d probably be quicker if the codebase was smaller.
Many open source developers learn a new project the same way. They look, they learn about the project using the source, they dive in.
Picking up a new codebase in years past
But it wasn’t always like this. Decades ago, we didn’t have powerful IDEs. In those dark ages, we started by reading voluminous documents. Those documents gave developers the roadmap and that important first handle into a large codebase. Next, a developer would read and digest detailed specs on the modules to be modified. Then, and only then, would someone make the first tentative steps into modifying the code.
Unit tests weren’t standard practice yet. Developers approaching a new codebase wouldn’t have that security and that safety of knowing that mistakes would be caught early. Any mistakes will escape to a QA department. Even worse, any mistake might even make it into production or distribution.
The common languages we used were very different. C was by far the most common language. The other main players were PASCAL and FORTRAN. Today’s common idioms for encapsulation were rare. You had to be careful about modifying global variables. Global variables weren’t a code smell. They were necessary. That introduced more risk into each change. That increased risk meant more process.
The extra process was necessary because the tools weren’t able to support greater speed. The costs of mistakes were high; we didn’t have a high probability of catching problems before customers did. Worse, we couldn’t give new developers the confidence to move quickly, especially on an unfamiliar codebase.
That extra process also carried real overhead. At my first job, we had librarians. They were responsible for helping developers find the necessary documents or code for a particular problem. Yes, their entire job was managing the documents we created. That overhead may sound like waste today, but it wasn’t. These people were instrumental in helping developers be productive.
We also had a much larger QA and test department than you would expect for a similar sized organization today. That’s because so much of the testing was done by hand. And, of course, the test organization meticulously followed test plan documents. There was also the science of sampling those test plans for sniff tests and regression tests (and rotating the sample so that over time every test was executed).
Understand Upper Level Management
The upper level managers I talk with had similar first job experiences to mine. One big difference is that they often left the ranks of main line developers to become managers many years ago. When they first hear about the lightweight process we use, they don’t see it as leveraging the tools. Instead, they see it as being sloppy. I have to frame that discussion in terms of moving fast and leveraging modern tools if I want to make any headway.
I can’t talk about the lack of test plans; I have to talk about the overwhelming number of automated tests.
I can’t talk about lack of documentation; I have to talk about documentation in the form of an Open Source project’s wiki, and IDE tools to browse, search, and learn about a codebase. Most importantly, I have to separate the act of creating a design (thinking and collaborating) from the act of writing a design document.
I can’t talk about lack of a firm plan; I have to talk about measuring velocity and responsiveness (even to change).
Most importantly, I have to explain in very clear terms how the artifacts we needed to create in the past are not necessary because the tools enable us to do more, do it faster, and become familiar with a project much faster.
I’m not saying that the people I’m referring to are dinosaurs, out of touch, or anything of the kind. I’m writing this to point out that they, like all of us, bring our own biases to every conversation. These same biases color our decisions. If you want to move these managers away from those processes they remember as necessary, you must explain why they are no longer necessary.
Remember this anecdote when you find yourself trying to change a process that’s no longer bringing value:
A newlywed couple was working on a roast for dinner. The young bride cut the ends off the roast and put them into a second smaller pan. The young husband had never prepared a roast this way, so he asked why. The bride said she didn’t know why, but it’s what her mother always did. The called the bride’s mother and asked why. She said she didn’t know either, but it was what her mother had always done. So the young couple called the bride’s grandmother. The wise grandmother laughed and laughed. When she finally stopped laughing, she said “I can’t believe you’re still doing that. I only cut the ends off the roast because when your grandpa and I were young we didn’t have a roasting pan big enough for a roast that would feed all of us.”
Don’t continue to use a process that’s no longer necessary with modern tools. Equally important, remember that those processes once had a purpose, and you need to explain why modern tools render them obsolete.
