Summary: Research as a Stochastic Decision Process (Jacob Steinhardt)

How can you maximize your impact across a set of potential projects? Arguably, the most important thing is to discover as quickly as possible which projects are dead ends. The more efficiently you can rule out unviable projects, the more time you free up for viable ones. This is the main insight of Jacob Steinhardt’s essay, Research as a Stochastic Decision Process.

Steinhardt’s thesis is doubly important if, like for many people, a minority of your completed projects are responsible for a majority of your overall impact. You may have heard of the Pareto principle, or the 80/20 rule, which states that in many systems, the majority of the outputs come from a minority of the inputs. Here is a nice visualization (from Khe Hy):

Suppose that your projects follow this pattern too: most are punts, and few are home runs. Suppose also that you can’t predict the impact of a specific project; you only know that home runs are rare. Under these assumptions, it pays to complete as many projects as possible so as to maximize your chances of hitting a home run. That’s why it’s so important to figure out as efficiently as possible whether you are working on a project that is unfeasible, so that if it is you can move on to a new one.

How can you accomplish this? The key is to correctly prioritize the essential tasks for each project. Say you’re a psychologist and you have an idea for a study. Like all projects, this involves a set of essential tasks that you must complete for the project to succeed. For example:

  • Recruit participants.
  • Hire assistants.
  • Book a space.
  • Get ethics approval.

If this project is to fail for reasons outside your control, you want to minimize sunk costs. To this end, you can ask a guiding question about each task: how likely is this task to fail, relative to how long it will take to complete? The more likely a task is to fail, and the less time it will take to complete, the more you should prioritize it. Let’s ask this question of each of the tasks above:

  • Recruit participants.
    • Very likely to succeed, and should take a few hours of focus.
  • Hire assistants.
    • Likely to succeed, and should take a few days of focus.
  • Book a space.
    • Very likely to succeed, and should take a few minutes of focus.
  • Get ethics approval.
    • Uncertain whether it will succeed, and should take a few hours of focus.

In this case, your first priority should be to get ethics approval. Since there is uncertainty about whether you will even be approved to perform this experiment, you should find this out before spending time on surefire tasks like hiring assistants and recruiting participants. In Steinhardt’s words, prioritize tasks that are “most informative per unit time”.

Of course, this advice doesn’t apply only to researchers. It applies to all forms of production. Recruiters sourcing candidates, salespeople closing deals, teachers creating lessons, writers writing articles—we can all benefit by terminating our unviable projects sooner.