Most organizations have a Research and Development department, or R&D. Within software organizations, R&D departments are made up of developers, or engineers, and have clear goals to accomplish that are usually represented in a task management system (like Jira). Developers (or engineers) write, test, and deploy code.
But what about the R of R&D? There appears to be some ambiguity here. Some argue that every developmental task includes an element of “research” – you need to test your code and potentially explore various approaches.
So is this Research?
I am less focused on semantics here. However, many individuals can distinguish between the process of a development task (involving trial and error, reviewing others’ code, and more) and “deep” research – where the individual encounters a problem without a clear solution.
All software organizations have Development tasks, but with Research it varies. Some organizations have a lot of Research tasks, while others have none. There are even organizations with dedicated Research teams, and in others, only a select few are ever assigned these mystical tasks.
So what is really Research in R&D organizations? Is it a skill that can be learned, or is it something that is a bit more “exclusive”?
In this series, we will dive into these questions, and consider various aspects of solving Research problems.
A working definition of research tasks
For the purpose of this article, let’s adopt the following working definition for “Problem Solving” by Alan H. Schoenfeld:
“You are engaged in Problem Solving when you are trying to achieve something, and you do not know a straightforward way to do so.”
For example, calculating 9481759*201471 may not qualify as Problem Solving (for someone who learned how to do it, and I assume you did). It’s not easy, it’s time consuming and error-prone, but at the end of the day, it’s not Problem Solving.
When you encounter a math problem without a clear solution or face a blank page while attempting to write an essay, you are engaging in problem-solving. It’s about figuring out the best approach, whether it’s tackling numbers or crafting words.
Many Development tasks involve Problem Solving: you need to think about how to implement a specific function, or browse through code to try and find a hint as to why something works the way it does.
Many D(evelopment) tasks involve Problem Solving – you need to think about how to implement a function, or browse through the code for a hint to how something is working. Yet, if you are thinking about the implementation of something you know can be implemented, I will not consider it as R(esearch). It is a R(esearch) task, when you simply do not know how to proceed – what to look for, and whether the path you are pursuing can even produce a relevant solution.
Throughout these posts, I will relate to such tasks, and the “R” in R&D in general, as “Research” (with a capital R).
A true story
It was a hot summer day in Atlanta, Georgia and I was at the Render conference with the Swimm team.
I had an idea: draw attendees to our booth with a problem, “wrapped” as a game, that included a prize. Anyone could attempt to solve the problem and win. And if they did, they would receive one of our classic Swimm rubber ducks.
You can see the game from the original booth here:
In the image above, the duck is on spot number 41. On each player’s turn, they can move the duck between 1 to 6 spots. For example, if the duck is on spot 41, the player can move it to any spot between 40 and 35. The player who moves the duck to spot 1 wins the game.
Anyone who approached the booth could play. They had the choice to play first or second, and could take as much time as they needed to think. I played against them, and if they won, they would get the duck.
None of the players won on their first try. Many attempted multiple times but still failed. At that point, they didn’t know what to do. It wasn’t obvious to them that there was a solution (until I told them), or how to approach the problem.
In a way, this is like a small research task.
Some people came back after a few attempts and solved the puzzle. Most didn’t—they just gave up. It’s hard to say whether they weren’t motivated enough to succeed or didn’t believe they could. Still, many questions arise: What separated those who succeeded from those who didn’t? More importantly, is this a natural ability, or can people improve their problem-solving skills?
We will return to this puzzle and solve it later in this post.
Is research something you can “learn”?
People aren’t born with the ability to solve the game I just described. In my experience, here are three ways one could go about improving their Research skills:
1. While solving real tasks
When you encounter real research tasks—whether at work or elsewhere—you may succeed, fail, and learn from your attempts. For some, this might seem like the only way to improve research skills. Indeed, this approach has its advantages. Nothing is as effective as learning by doing, and tackling difficult research tasks can significantly enhance your skills.
However, consider the downsides of this method. First, it takes a lot of time. Complex research tasks can span months or even years. But an even bigger challenge is that when you’re assigned a research task, you often don’t know how difficult it is—neither does your boss or the person who gave you the task. They may estimate the level of difficulty, but they can’t know for sure. The task might be unsolvable, or it might only be solvable by someone with much more experience than you. Is the best way to learn math to be suddenly confronted with a 10th-grade problem when you’re only in 3rd grade? Probably not.
2. Learning from others’ research
Learning from more experienced researchers—whether by watching their talks or reading their research diaries (more on that later)—is certainly faster than learning solely by doing. You can gain insights from others that you might never discover on your own. However, there’s a limit to how much you can absorb from someone else’s research. Ultimately, personal experience plays a crucial role in deepening your understanding and honing your skills.
3. Gradual, focused & guided problems
For example, if you take a “research course” where you’re given bite-sized puzzles to solve, you can dedicate time to solving them with the reassurance that each puzzle is solvable and appropriate for your current skill level. Afterward, you can discuss the problem with peers or an instructor and receive feedback on your solution. This approach allows you to gradually improve your research skills in a structured way. However, having access to such a course isn’t always feasible, and learning from these controlled problems isn’t quite the same as tackling “real” research tasks.
Actually, the most effective way is probably some combination of the three. You can’t learn to swim without getting into the water, but you can learn a lot about swimming by watching others swim, or by practicing in a controlled environment.
Back to the game
As I mentioned earlier, most people who encountered this challenge didn’t know how to approach the problem and eventually gave up. You might wonder, what does this game have to do with research? After all, it’s just a simple game, not a research task, right?
In the next section, I’ll introduce a framework for problem-solving that I believe can be applied to research tasks. We’ll analyze the Spiral game using this framework and explore how it can be applied to more complex problems.
But first, let’s solve the puzzle. If you’re up for it, give it a try—can you figure out a strategy to ensure you always win?
The game: solution
Let’s tackle the solution methodically: it would be much easier to start from the end.
Let’s think about this methodically.
It would be much easier to start from the end:
- Let’s say the duck is on spot `2`, we would want to move it `1` step. That’s clear
- What about `3`? Move `2` steps. Thats’ clear too.
- We can keep going this way until we get to `7`, where we need to take `6` steps.
What happens if we find ourselves on spot `8`?
If we were to take `1` step, our openent will have their duck on spot `7` and we know that means they could win. For every possible turn, we can see that if the duck is on spot `8`, the player who moves first will move to duck to a position between `2` and `7`, where the other player can win.
Since there is no “good” move for the player who moves first when the duck is on spot `8`, we would want to be the second player in that case.
Now, we can also look at it as a new game, where the player who reached `8` first wins.
Now, we can look at it as a new game – the player who reaches `8` wins. If we move the duck to spot `8`, then it’s the other player’s turn, and we know for certain that we can win. Therefore, we should only focus on reaching spot `8`.
Let’s keep going
So, let’s keep going. If the duck is on spot `9` – we can take `1` step and get to `8`, wishing to be first. If the duck is on spot `10` – we can take `2` steps and get to `8`.
And so on…
Until we get to `14`, where we need to take `6` steps to get to `8`.
At spot `15`, any move we make would leave the duck between spots `9` and `14v, where we already know we want to be the first player. Therefore, if the duck is on spot `15`, we would want to be the second player.
And now – once again – we can consider this a new game: reaching `15` to win.
`8` and `15` are exactly `7` spots apart. This is no coincidence. It is the same difference between `1` and `8`, and we win when we get to `1`. When a player has to play on spot `8`, they can move at max `6` steps which is not enough to reach `1`, and then the other player can make the move to reach `1`.
In general, if we keep the following we always win:
After each turn we make, the duck will be on a spot of the form `7k+1` (where `k` is a non-negative integer). So `1`, `8`, `15`, `22`, `29` and so on – are all spots where we want to be second.
(You can find a formal and complete “proof” – in the appendix)
So, what can we take from this puzzle? Let us consider a framework for solving problems first.
Framework for problem solving
I would like to adopt a framework for Problem Solving by Alan H. Schoenfeld (1992, see references). While this framework was first devised to describe how people solve math problems, I believe it can be applied to Research tasks in general.
In his paper, Schoenfeld describes the components of Problem Solving**
- The knowledge base: This is what you know. For example, when faced with an optimization problem, are you familiar with profiling tools and optimization algorithms? Do you know how to use them?
- Heuristics: These are the strategies you can use to solve problems. For example, when faced with an optimization problem, you can try to profile your code to see where bottlenecks exist. You can also debug a specific use case, or draw a diagram of the problem. If the problem you are solving is writing a compelling article, heuristics can be creating an outline before starting to write, using a specific structure, or writing the conclusion first.
- Control: The ability to monitor progress, and adjust your strategy if needed. For example, if you’re trying to solve a problem and you see that your strategy isn’t working, try a different approach. This is what distinguishes experienced engineers from novices. What matters isn’t just what you know, but how and when you decide to use that knowledge.
- Beliefs and attitudes: These are your beliefs about your ability to solve the problem. For example, if you don’t believe you can solve an optimization problem, you may not even tackle it in the first place. In Schoenfeld’s research, he describes the beliefs of students when it comes to math problems, like “I’m not good at math”, “math is a matter of memorization, “there is one right answer”, and the list goes on. These beliefs can greatly affect your ability to solve problems.
Applying Schoenfeld’s framework to the Spiral game
- The knowledge base: In order to solve the game, you need to know how to add and subtract numbers. But, to prove that you always can win, you need to understand “modulus” – that is the remainder of a division.
- Heuristics: We mainly used three heuristics to solve this problem. First, we tried to solve the problem by starting from the end and working back. Second, we considered cases systematically and listed them. Third, we generalized the solution to all cases.
- Control: I provided a clear solution to the problem, but most people initially approach it by making various “guesses.” For example, they might think, “Maybe I just need to be first on odd-numbered spots.” When they test this hypothesis, a form of Control is to find a counterexample that disproves it. Understanding that you should start from the end, work backwards, and approach the problem systematically—using the heuristics described earlier—is also a form of Control. This helps to refine and focus their approach, leading to a more effective solution.
- Beliefs and attitudes: If you believe that you can solve the problem, you’re more likely to try. Many of those who tried, simply gave up after a few minutes, usually accompanied with a shrug and saying something like “I’m not so good at these kinds of things.” Another belief is that you should solve this problem in your head, rather than using a pen and paper. To systematically consider different spots, it is actually very helpful to write them down.
** Actually, Schoenfeld (1992) described five components (which he terms “categories”), but I focused on four of them.
Learning to research
So, again, what does the Spiral Game have to do with Research?
The same Heuristics and Control we used to solve the Spiral Game can be applied to more complex Research tasks. The Knowledge base would probably be different. By solving tasks like this, talking about how you solved them, and learning from that experience, you can improve your research skills. It doesn’t happen overnight but with practice on many problems. I’ve seen people get better by starting with small challenges and working their way up to harder ones. Research by Schoenfeld shows that people can learn to use smart strategies for problem-solving and can also learn to track their progress and adjust their approach (Control) as they go.
The next posts in this series will discuss additional common heuristics in problem solving, and how one may acquire them by tackling more complex problems.
