I’ve been spending a lot of time with engineers all over the world to talk about software documentation. Something that always comes up is overcoming the inertia of starting when it comes to writing code documentation.
Tl;dr: Making documentation continuous is an essential part of any strategy to pay down documentation debt. Clues to what to prioritize can be found by running call graphs and debug log analysis, while a goal-oriented approach helps make shorter work of the writing process. Swimm is a tool that, out of the box, helps with all of this.
It all starts with a problem – and a plan
Approaching a code base with an eye for documenting it is something that we’ve all put off. We know that the lack of documentation coverage is debt that we need to pay down, but we tend to rationalize delaying it because we can’t reconcile spending the amount of time we think it’s going to take against the increasing business needs to get new features and fixes out the door.
We keep telling ourselves that we’ll “find a good time” to just sit down and write, but that never happens. Later, we start to feel guilty, but rationalize putting it off even more because we know the documentation is just going to go stale and unmaintained again in six months, right? Does any of this sound familiar?
I’d like to tell you that it doesn’t have to be this way, that documentation “sprints” seldom if ever actually solve the problem, and that the real solution is most likely simpler and more attainable than you might imagine. It all comes down to the technique you use while figuring out how to start, how well you can get others on board with not letting it pile up again, and what kinds of tools you use to manage your builds. That doesn’t sound terrible, does it? Let’s dive in!
How did I figure this stuff out?
I’m not the world’s greatest anything, but I do a considerable amount of technical writing and did a ten-year stint where all I did was take over projects for hosting companies and try to get them back in the positive. I’ve fought documentation debt in the face of high turnover and chaos and learned a lot about what doesn’t work when solving it.
I struggled, worked late, cried, toiled and eventually conquered my way into something that’s simple enough to work repeatedly and I really hope you find this of use. Solutions don’t really work unless we make them our own by bringing our own needs and objectives – so don’t treat this as a step-by-step tutorial, treat it as a starting point.
Throughout the years, I got good enough at this that Swimm hired me to help you get good at it too. I should probably say that I really love Swimm and what it’s doing so I’m biased, but this isn’t an advertisement – this is about helping you get out of a docu-rut.
What you should probably not do
There are some things that may seem like great ideas, but more often than not, don’t pan out well. Before we jump into how to get started, let’s make sure we’ve got workable ideas rattling around that you might build on as you read this.
These ideas should definitely not be among those currently rattling around, along with a brief description of why you might consider avoiding them:
- Documentation-a-thons: These can sometimes be helpful, but make sure you put emphasis on the quality of the coverage and the quality of the documentation. Quantity does not equal quality in either case. Unstructured, these organized efforts don’t typically help much when it comes to actual onboarding, and the resulting documentation goes obsolete pretty quickly if nothing is keeping it up-to-date. That’s not a great use of your employee’s time, or time that folks in your community might be able to give.
- Hiring third parties: Some organizations might reach the size where hiring someone to look after technical / code documentation makes sense. If you’re not a large company, this is probably not a good solution, especially on a freelance / part-time basis. If understanding how everything works was really as simple as just looking at the code then we wouldn’t need documentation, so make sure you realize anyone you bring in will need to fully onboard to be effective. Then you still have the problem of things going stale. It’s better to use hiring as an occasion to see how well you’re doing, as we’ll describe below.
- “Voluntolding” Developers: Not everyone is great at writing. For documentation to just happen naturally, people have to want to contribute to the solution. They may not necessarily find a love for writing, but they want the best for their team. Singling out one, or even two people and arbitrarily making it their problem is just going to create animosity on the team, resentment, and while numbers may move in a better direction, the documentation will read in the same spirit that it was written.
You should also not do stuff like kidnapping best-selling authors, but we can’t stop you. What we can do is just suggest that you consider all aspects of any solution that seems like it’ll require enormous amounts of energy up-front with no plan for how you’ll avoid the documentation going stale once you have good coverage.
Before you start
The first thing to do is announce to your team that you’re coming up with a plan to solve the documentation problem long-term, and you plan to have a document to share on it within a week. Maybe you should make it two weeks or a month if one week doesn’t seem enough – what matters is it needs to be a time that’s going to pass soon, and they need to know you’re committed to doing something by then. This is important.
Share this with your team early and ask them for feedback. Encourage people to email you the last few questions they’ve answered about the code, or any that frequently come up. You could probably do this yourself, but remember, it’s a team effort – there’s more to this than just accomplishing a list of things.
Figuring out coverage
The next thing you need to do is figure out what needs prioritizing as you dive in. I know you want to document everything and that’s a truly noble goal, and I hope you nail it! But, let’s make sure that if you get diverted to something else next week that important things still get attention.
Cover the welcome mat
Remember that list of things folks answer frequently? That’s a very good place to start. Make sure that any documentation about setting up your build environment is current (as this isn’t as tightly coupled to code semantics). There are some places where you just don’t need any more insight into visibility because you know everyone is going to have questions about it. These “welcome mat” parts of the code base need the most attention.
Run some call graphs
Wise programmers say that 90% of your code only runs 10% of the time, with the remaining 11% running 90% of the time, or something like that. Joking aside, you can start to grasp areas where you should be looking in a similar way that you’d approach writing missing unit tests.
Through static or dynamic analysis, many programming languages have tools that can offer up something known as a call graph, which in its simplest form is a chart showing how often certain parts of the code are entered during execution. Theoretically, that which is entered the most will benefit from thorough examination for documentation needs.
There’s no rule of thumb that helps translate call graph percentages into the number of words that should explain any arbitrary code path. Some code paths really don’t need explanation beyond comments, some need cautionary tales of heroics.
Saw some logs
No snoring on the job! I mean literally take a log saw and group entries by type and the calling methods and classes if possible. This is going to yield a look at areas of the code that need to be called out as they’re entered while debugging, as well as any console writes that are still firing which are pretty good indicators of integration hotspots in the code. This is not an exact science, but can be useful when overlaid on a call graph – intersecting points become interesting to investigate.
Breaking down the writing
Don’t just start writing. You can certainly do that, and you may come up with some great stuff that you can certainly use, but let’s face it: you’re running a marathon that is going to be frequently interrupted by many other things. Structure this in a way that is kind to the future you, the one who has to come back to this in two days and wonder where they left off. Writing without guardrails usually leads to getting lost in what you’re writing.
Reader goals are writer clues
Start new units with a simple sentence:
“At the end of this unit, you will be able to [thing1] [thing2] [thing3] … and be ready to [next]”
That’s all you need to get started in a grounded way that allows you to map dependencies backwards.
Then, you can fill out all of the [things], or create a unit where you can define what they should learn in [next]. You can also toss in a [previously] and do the same thing. If at the end of an hour you end up with a dozen modules that have clear goals of what readers should take away, you’ve had a very productive hour.
Remember, this is for your team. Once you start to have what resembles a skeleton for some documentation established, break out a document to share with your team where you describe your process so far, and ask them who they think would be the best people to consult on certain modules. Let those folks know you’ve generated some sparse outlines to help them write a couple of paragraphs about code they know a lot about and ask them to contribute when they can. Each time you ask someone else to help in the effort, let them know the effort that has already gone into it. Every small unit of effort that folks put into this makes the larger effort more important because everyone has time put into it, which makes the group conscious of potentially wasting that.
Keep writing goal outlines of what the reader should know until you’ve got something for every part of the code base you identified.
Making documentation continuous
You’ve heard of continuous integration and continuous delivery, continuous documentation is very similar in concept and application. When new code is checked in you really need two things to happen consistently – and ideally automatically:
- If the change makes the documentation obsolete in some way, your team should at least know about it. In some cases you might want to block a merge until this is resolved.
- If a change significantly lowers your documentation coverage, it needs to be at least acknowledged.
Swimm’s CLI tools simplify this (swimm verify / swimm coverage) in a way that you can plug into most build systems and use with Github and Atlassian hooks. You can also operate directly from the diffs to count insertions relative to source files / documentation files, or other automated script checks to help keep tabs on documentation.
The objective here is up to you and your team – do you want your repositories to always be in the best coverage state? Enforce rules to that end. Do you just want to avoid having to do massive documentation overhauls several times a year? You might be able to be a bit more relaxed, as long as you’re tracking what will eventually need to be done. The point is, it’s up to you.
Finally, don’t optimize for ways that folks might be able to “game” around documentation checks. You can work your way around things that exist because they want good things for you if you treat those things as obstacles. Instead, optimize for getting new folks on board with the need to care about documentation as much as everything else in the repo.
Sign up for a Swimm 1:1 demo and learn more about Swimm’s knowledge management tool.
