Should we still care about documentation now that we have LLMs?
If it was easy already…
“Clear code documents itself”
It is a common enough refrain from developers that code itself shouldn’t actually need documentation if it is well written.
If you hold this axiom, it is an easy step to believe that AI ingesting your entire codebase will be able to return accurate answers to any question a developer might want to ask.
“How do I use this method?” – easy, let’s see what the code says. LLMs that iterate over massive quantities of code are particularly well suited for this kind of work.
Documentation is a fundamental practice
Documentation remains a core part of the software development lifecycle. And that’s because there will always be core company knowledge that can’t be communicated by code.
For example, code will never be able to explain why a decision was made NOT to do something. Don’t want developers using certain pointers to memory? AI doesn’t know that.
Your product teams have created business logic that is expressed in code that needs to be understood by AI.
Lead developers and DevOps/Platform teams have their own internal best practices and ways of consuming services. None of this is readily apparent to a probabilistic AI model.
AI doesn’t replace documentation. Documentation is a requirement for AI. And AI is great for documentation.
Hybrid methodologies of using LLM outputs combined with retrieval frameworks like RAG will be dramatically more effective at providing true code understanding with accurate, up to date documentation that is tied to code.
Instead of generic code answers based on a corpus of open source training data, advanced code understanding assistants will be able to specifically relay your business specific context, providing useful answers.
Documentation is therefore a critical part of a development “knowledge layer” to power advancements in AI by contextualizing them to human reasoning.
Of course, Generative AI’s promise is to reduce mundane tasks so that us humans can focus on higher level reasoning and problem solving.
Most developers we’ve met say code documentation is the most qualified mundane task they know of for AI assistance.
How then can AI help with documentation while making that documentation useful for LLMs?
The core tenets of AI assisted documentation are that:
- Documentation is primarily written by AI with humans adding the missing delta of company specific knowledge that AI will never know.
- Documentation is always kept up to date automatically, with human input only for key changes.
- Feedback loops to create or enhance documentation exist in the chat interface, driving the expansion and improvement of the documentation.
Interested in learning more about how Swimm will enable the future of code understanding? We have a big announcement coming on January 17. Register for launch event to learn more.
