We recently welcomed Ryan McKenna, staff engineer at RVO Health, to share his strategies and insights into creating an effective documentation culture.
Ryan has been evolving his approach to culture building around the developer experience for years and has great insights to share with you.
This transcript is from a Swimm Customer Series where Swimm customers come together to share key learnings to help the community and answer questions.
__________________
Tom: Ryan, can you tell us about RVO Health and your role there?
Ryan: Sure, I’m a staff engineer at RVO Health. We’re one of the largest health companies you’ve probably never heard of.
As a staff engineer at RVO Health, I lead a lot of the front end architecture and other product research and development for new tools and products. I also handle developer experience situations where we’re trying to solve problems at scale in our company.
Tom: What led RVO Health to invest in documentation?
Ryan: Documentation has always been a struggle for us. We actually originally had an internal team dedicated to documenting things
So we created our own documentation portals and our own process for all this. However, we didn’t really solve this problem. Not entirely.
In the last year or two, we started doing developer experience surveys to really start measuring this problem. It was something that we always knew – the experience of people didn’t improve.
We always talked about how it was still difficult to manage documentation in our company.
But without measuring it, we really didn’t have a way to really know how we were moving the needle. So documentation just kept coming up. It’s this number one issue in our development experience surveys.
So we kind of needed to come up with something and we started exploring options and Swimm came up as a tool to try. And, you know, here we are about a year later using it.
Documentation just kept coming up. It’s this number one issue in our development experience surveys.
Tom: Why were previous documentation efforts unsuccessful?
Ryan: We’re in a move quickly environment, you know, getting things done at the end of the day was the most important thing for us. And in companies where that’s the case, it usually comes down to prioritization. And that was the issue for us. You know, we don’t just see the issues with documentation.
I imagine some people can relate to having issues with testing their products as well. And all these things get prioritized when you have deadlines that are coming up. We’re like; “Hey, we need this project done by X time. And if we don’t, there is money on the line.” Business makes decisions and you have to get things done.
And as engineers, a big part of our job is choosing which parts we can kind of let slide sometimes. So when we say why it hasn’t been successful up until more recently, it’s because we didn’t really have the tooling around this that took into account how our time was being spent.
Tom: It sounds like a classic situation where documentation is seen as something that is technical debt. And particularly, when there’s a business need, it feels like there’s a competition between both. So how did you get executive buy in for a project like this?
Ryan: Well, I’m really glad you mentioned it’s like technical debt because fundamentally, it was. When a company moves this quickly, if documentation doesn’t stay up to date, it’s by default, technical debt.
So how we got executive buy in, most importantly, was starting to measure that problem.
Like I said, the developer experience surveys really helped at the time. When it comes down to communicating with executives and business partners, product stakeholders, especially at a large company, you have to learn to speak the language of the organization and the audience that you’re talking to.
As engineers, we talk about things in a different way than business often does. And so, measuring and then translating from what not having this documentation means to what is at stake is very important for buy-in.
It’s not just that developers aren’t happy, it’s that when you put someone new on a team it takes this much more time to ramp that person up.
And then time becomes measured against this developer costs X a year. There’s this much time where they’re not productive. You know, it becomes a different kind of conversation when you translate that over. So getting the executive buy-in is basically learning how to speak that language a little bit.
When a company moves this quickly, if documentation doesn’t stay up to date, it’s by default, technical debt.
Tom: In a lot of conversations I have with developers about documentation it seems like a love hate relationship with it. Where they love it when it’s there and and they hate thinking about it or working to make it happen. So how did you think about rolling out documentation practices to your engineers?
Ryan: I think something that was a struggle for us, and it’s something that we’ve been really trying to change, is that engineers are fickle creatures sometimes. There’s a part of us that loves change. We love seeing the new thing on the street, and it’s like, oh, I want to try it right now.
But, you know, we’re also really resistant to change in our workflow in some ways. We want to try technology, but we don’t want to change how we interact with our technology. So, for us, we wanted to have a strategy that said; “Okay, how do we lower the barriers? We have to lower the friction cost for getting people to write documentation.”
In our current business mindset, we’re very customer focused. And it’s something that’s very appropriate. You gotta almost ask yourself in the same way, right? If we were just as employee focused as we are customer focused what kind of effects would that have?
You would never ask the same things – a customer goes to Amazon, they streamline them to shopping with things like one click purchase.
Why isn’t it one click documentation?
Also, you’re asking an employee to do five, six different things. You’d never ask a customer to go to seven different tools to do their job. So for us, it was like, let’s get documentation done where an engineer does their job already. So integrating it deeply within their code writing tools, the IDE, was a central part of this.
So, reducing the friction to where they could actually effectively write documentation was the most important part. And then we could layer on top functionality from there. So cool, now we can integrate it with their code tools. Let’s then see how we can integrate it with their CI tools, then let’s integrate it with examples of diagramming. Let’s layer all these things in so it’s not just an abrupt change because you want to have change be gradual.
Tom: In an organization where documentation wasn’t part of the culture, did you also have to change something about incentivization?
Ryan: Yeah, but not too much. There was definitely a little bit of incentivization that needed to happen. But it turns out that most engineers really recognize this problem.
Most engineers join a team and are like, “Where’s the docs?”
And so we just had to change how we communicated that if we’re going to spend 40 hours in a week, for easy math.
If we could spend a quarter of our time doing documentation, it would have these impacts. What would that look like in six months or is it better to do it in two weeks? We just had to change the incentivization of how we organized all of our work time.
Tom: Who was involved in implementing these incentivization changes?
Ryan: The effort required alignment across various roles, from technical project managers to product managers. It was vital that everyone involved in planning and prioritizing work understood and supported the documentation initiative.
Engineering is probably some pattern of getting work done, some process for how we prioritize the work being done. This is a conversation that has to happen with the people who make the priorities and also are in alignment with how those priorities get set. So, we have a product that we say is going to take 8 weeks to get done.
Okay, well let’s add 10 percent of that for documentation.
Now instead of 8 weeks, it’s going to take say 10 weeks. Is that functionally a huge difference for the business? And then you kind of have a negotiation about that and see where we can slot our time in?
And if we can set new expectations about 10 percent of our week is going to be dedicated to documentation then that becomes the norm.
A lot of companies have those days where engineers have one day a week where they’re working on something else or they have a whole day dedicated to bug fixes. Documentation needs to be part of the same mentality.
Coming up with a real process that works for people, it’s not going to look the same in your company or in every company, but that’s a conversation that needs to happen.
Tom: Who is that conversation with? You were championing documentation and Swimm but who are the folks that you saw as your colleagues and partners in this particular project? And who are the folks that you had to get help from or to recruit?
Ryan: So it depends on your organization.
In mine, it was very much that I already had buy-in from executives. At the end of the day, some executives are much more involved than others when it comes to how the work gets done. Sometimes they just care whether or not it does in fact get done right. There is a big distinction there.
So if your executive is in the room helping you plan your sprints, then you need to talk to them about how 10 percent of your sprints go to documentation. So it’s very much who’s in the room planning the work since they need to be on the same page about how much work goes to documentation whether that is your product manager, your direct manager, or directors.
The most important part is to have buy-in for it.
So if your executive is in the room helping you plan your sprints, then you need to talk to them about how 10 percent of your sprints go to documentation.
Tom: Another thing that I see a lot, particularly with organizations that have a very poor state of documentation, is getting started. This feeling of being overwhelmed by the possibilities with so many possibilities where to start. It could also put you in a position where you can’t start.
So, how do you decide where to start? What part of your code are you going to start documenting?
Ryan: I think there’s a few questions here.
What’s the most important code to document? What’s going to be changing the most? What should we be keeping up to date? What’s critical to the company?
I kind of have a little joke about this when it comes to testing. There’s a lot of parallels here. You don’t test something that’s not mature, right? There’s almost no point especially if it’s changing next week Why are you testing that? You know, test the things that are going to be here for a test to matter.
Documentation very much is in the same vein. So when I try to get my business more aligned with that, the first thing I say is let’s take your user stories. Take a product person and say, here’s the application. Describe to me the most important things that need to happen in this application.
That’s our first thing that you document.
User stories are usually the foundational aspects of products. Then apply that to your code base. Say, okay, well, translate the user story in the code. Here’s how it flows through. It touches these parts of the application and in these ways.
And from there you can branch out and flesh out the organization’s documentation much more simply. It also kind of gets business to care a little bit because you’re starting it from the basis of their knowledge. So that’s one part – start with the important code.
And then the other part is to start with people who have the time to write the code.
In my organization, I actually don’t have a lot of time to write this kind of code myself. You have to start with the new engineers sometimes.
When they get to something that they don’t understand. Let’s show them how to document that the first time it happens. And so we’ll continuously build documentation as things come up.
Sometimes it’s daunting to say, like, let’s go back and document everything. But if we just started documenting new things today, that will have a very circuitous effect on how our documentation grows over time.
Tom: It sounds like you are going for things that are relatively new? And you’re utilizing mainly newer folks? Is that because they are easier to get on board?
Ryan: I would say it’s, sometimes they’re easier to get on board, but usually new stuff doesn’t have documentation.
You know, I think something that we’re talking around a little bit is that sometimes people do already have documentation. Swimm is a net new tool. And it’s very rare for there to be zero documentation that exists for a project that’s been long standing, right?
There’s probably some kind of documentation that a team has been maintaining.
It just might not be very good or very useful. The level of effort to convert that is just cognitively higher.
It’s a hard, convincing challenge. You have to change a team’s workflow.
What I found is that over time the more new things are started with this, the more the people who use the old stuff are seeing the benefits and are more interested in getting those benefits for themselves.
So it’s definitely easier to start with, okay, what needs to be documented now versus going back and documenting old.
Tom: So to get the ball rolling, you’re going to start with new stuff. At some point you’re going to reach a wall where there are things in the code base that are slowing you down. And the only folks that know about it are folks that have been there for a while and might be set in their ways. Anything else that could help to bring those folks on board?
Ryan: So I actually turned this into an exercise more recently. We have a lot of senior people at our company, and some of them have been around for a long time.
There are people who know where all the skeletons are buried, and no one else knows what they know, kind of thing. And that is a problem. It’s not the kind of problem that people think about as a real issue. Because that person’s always available, but one day, what if they’re not?
And so what we’ve turned into an exercise is that people really struggle with the blank page problem. Like, you asked me to write down everything I know about documentation. I’m going to start with a page and say; “Oh, cool. I know so much about this. Okay, where do I even start telling you what I know?”
So we turn it into an exercise where we have a newer engineer on the same team interview the more senior person and take those notes. They then turn them into documentation as a part of their onboarding process.
It’s much easier to say, what do you know about documentation?
But no. Start with, what do you know about this part of the code? Why is it here? When did you come up with it.
And then turn that answer into a process of writing down everything, So, a conversation about what they know becomes transcribed useful information.
Tom: You rolled out Swimm and you described pretty much how it went. Was there anything you were surprised by?
Ryan: Yeah, I definitely was surprised by some things, but not in how Swim worked.
In my organization, I wasn’t entirely sure how it was gonna go.
The teams I thought would use Swimm the most did use it but they were not the teams who actually ended up using Swimm the most.
The teams with the most usage were much younger teams. And I don’t mean that in the sense of age, like with people. I mean like they’re just newer, fresher, they’re working on products that are not as tethered down on like old tech or old legacy or processes.
Those teams took to it in stride because once it was in their code editor it was much easier like keep the code going and the documentation up to date. Those teams have latched on to it and they’re not letting it go.
Tom: What about for non-developers? Has improving documentation and its discoverability impacted collaboration between engineers, business leaders, and product teams?
With Swimm, developers know that the documentation is also serviceable up to business and product leaders.
And the impact has been huge.
Before, we had a fragmented environment: business documentation was in one place, code documentation was somewhere else, maybe in Confluence or other tools—and no way to find it all. Discoverability, especially for those beyond engineers, has been crucial and made a significant difference here.
Participant question: I would like to ask about giving 10 percent of development time to documentation. We also tried to do it and eventually we found ourselves using this time to do other stuff like bug fixing or an extra feature that we can add. How do you make sure you force the team to actually use this time for documentation?
Ryan: That’s a fun one. The 10% time is not a hard and set rule. Not everything takes 10 percent of a week to document, right?
You use JIRA or some other ticketing system, right?
Don’t say, okay, here’s a ticket for this feature then here’s a documentation ticket.
We tried that first. That doesn’t work. We ran into this issue where it just doesn’t seem to happen. How many times does this just sit in the backlog forever? This just doesn’t happen, right?
They’ll jump from one ticket to the next ticket, they don’t want to do it all at once. So what we did was we moved the story point of the documentation ticket that was separate and we rolled it into the original ticket. Then we said this ticket does not get moved to done unless it’s also documented.
So maybe the ticket was originally like a four hour adjusted ticket. That’s where the 10 percent comes in. Make sure the ticket is budgeted for five hours. And if they save time on it then fantastic – there’s no harm in that.
Participant question: Did you present Swimm to multiple teams or was the organization in one primary development team?
Ryan: We have 7 teams using it now. Our organization has about 15, so it’s not fully rolled out throughout the whole organization.
But our adoption is growing. To start out with, I picked one team. It’s much easier to roll out something in a larger organization when one team has already proven a successful way of using it,
I joked with Tom, “I was like, I used to Swimm a lot my first two months because all I was doing was writing examples and teaching that one team how to use this really well. In the last five months I barely used it myself because I’ve trained everyone else to use it and they’re all using it now.
Right? So the shift of investment is different. Train one team to do it really well. It’s much easier to say, I have an example of this thing working and it’s awesome. And let’s replicate that in other parts of the organization. Let’s try it. Start small and then grow from there.
And it’s also much easier from a buy in perspective. You know, you don’t have to sell something to an organization. Sell it to a team that needs it, who wants it. And then have the conversation of expanding that.
