Data Driven Task Prioritization

Every day, you are making decisions about how you, and your team, spend your time. I’ve found the generation of new ideas outpace the ability to implement them, so the list only gets longer each day. There are so many different tasks or initiatives that you could be working on, so the most important question becomes what you should be working on. Alas, we never have the time to do everything at once.

Outlier is a new kind of business intelligence. Instead of providing more dashboards and charts, Outlier monitors your business data and notifies you when unexpected changes occur. If you’re interested in seeing a demo, schedule a time to talk to us.

So how can you answer that question? With data of course! This week, I’ll talk about my preferred approach to solving this problem — by creating a scoring system. Each possible task is scored based on the most important factors that you think should be considered when weighing the benefit of undertaking it [1]. Using a formula, you can create a single number that can be used to compare and prioritize every task.

For example, the key factors I like to focus on are:

• Reach: How many customers will this influence?
• Value: How much money or avoided lost money will this generate?
• Effort: How difficult is this to accomplish?
• Confidence: How confident am I in my estimates?

We’ll break down the process of using these factors to prioritize your tasks into four parts: Reach and Value, Effort and Confidence, Scaling, Formulation and Examples.

Part 1. Reach and Value

We will start our conversation about task prioritization by talking about two factors that influence decision-making:

• Reach: How many customers will this influence?
• Value: How much money or savings will this generate?

At first glance, these factors seem very similar — wouldn’t it always be the case that efforts that impact all customers have high value, and vice versa? Nope!

With Reach, measured in the number of customers, the focus is on how many of your customers will the task influence, over some time period. For example, if all of your active customers will see a change of the color of buttons from dark blue to light blue, its reach would be very high. On the other hand, a customer-specific feature request would only apply to a single customer, so its reach would be very low.

The Value, measured in dollars, factor focuses on the expected financial impact of a task, over some period of time. Using the examples above, changing the color of buttons would probably have very little value as it won’t add new customers or prevent customer churn. On the other hand, a customer-specific request only reaches one customer, but it could unlock a substantial amount of revenue.

You can plot the Reach and Value of tasks on a chart to get a sense of how impactful they will be. The ones that are going to be more impactful are in the upper right quadrant; they impact many customers and will generate or save large amounts of money. The least impactful are in the lower left quadrant; they impact a handful of customers and generate or save small amounts of money.

Of course, you want to undertake the most impactful tasks. However, just because they are the most impactful does not necessarily mean that they are the highest priority — you need to consider the Effort and Confidence first.

Part 2. Effort and Confidence

Yesterday, I talked about the first two factors in task prioritization, Reach and Value. Today, I’ll talk about two other key factors that should be taken into consideration:

• Effort: How difficult is this to accomplish?
• Confidence: How confident am I in my estimates?

Effort, usually measured in time (days) or in “points” (a measure of effort you define yourself), takes into consideration the amount of time it will take to complete a task. This is really important to take into consideration in conjunction to the factors we discussed yesterday, because often times high Reach and Value tasks take a long time to complete. For example, moving simple pieces of previously written and tested code to different parts of the site is pretty straightforward, so it would score as a low effort. Building entirely new customer experiences, on the other hand, take more time because the experience must be designed and new code has to be written and tested.

Confidence, measured as a percentage, is a way to take uncertainty into consideration. Ideally, you will be able to measure your estimates for Reach and Value based on customer usage data from your product, customer surveys, or other research you’ve conducted and Effort with the engineering team’s input. However, that information is not going to be available in all cases, so you might have to use your best judgement. In the cases where your estimates for the factors are well-researched, you should use a high Confidence value closer to 100%. The more you have to rely on only gut instinct, the more you should discount your Confidence. For example, to the extent possible you would want to estimate the Reach, Value, and Effort of a task by using customer usage data from your product or other research you’ve conducted and in doing so you can be confident in your estimates of these factors. However, if you don’t have any data to support your your hypotheses, then you can use the Confidence factor to adjust for that.

Now that we’ve outlined the four primary factors in prioritization scoring, it’s time to talk about how we can map each measurement to a number that we can plug into a formula.

Part 3. Scaling

With all of these factors in hand, now we need to translate them to a common framework so we can combine and compare them. We will do that using scaling.

The challenge with a scoring system such as this is that each of the factors we’ve discussed measures something entirely different. Reach is the number of customers. Value is a dollar amount. Effort is a number of points. And Confidence is a percentage.

We could leave the raw values from each factor as they are to use in calculations, but then the resulting scores and proportionate changes will be influenced by the magnitude of the values themselves. If the Reach doubles from one customer to two, should that have the same impact as the Value doubling from $10,000 per month to $20,000 per month?

You can control for these issues by using scaling to translate the raw values into more comparative values. Scaling can be as simple as multiplying each value by a weight, or as complex as having functions that map a value into a new, scaled, value.

For example, we could scale the values of our factors as follows:

In this scaling example, I mapped every factor’s value to a scale of 1–10. But, if desired, you could weight some factors more than others. For example, if you really want Value to be the main driver of your prioritization, you could scale it to values between 1–20 (and keep the others between 1–10).

This part of the process is a bit subjective in that you will need to decide on the size of the relative steps between each value and whether to make the buckets increase linearly or not. These are the nuanced choices you’ll need to make to be prioritize your business, but by having a framework in place, you will at least be able to make choices based on a consistent measurement.

Now that we have our scaled values for each factor, everything is in place to create a formula to compute a prioritized score for each task.

Part 4. Formulation and Examples

Now that we know how to scale each of the factors, we can use a formula to compute a score. The formula I use is:

Using this formula to calculate the score gives you exactly what you want, the confidence-adjusted impact (i.e., combination of Reach and Value) per unit of effort to achieve the task.

Let’s have a look at the two most impactful initiatives from the other day to see how they would be scored and prioritized using this system, starting with the “create a native smart-phone app” task. Suppose we’ve done some customer research and found that about 100 new or existing customers would find an app useful, so we estimate Reach to be 100 customers. The app would be so useful that between reduced churn and new sales we expect a Value of over $20,000 per month. However, these Reach and Value estimates are based on surveys and a few customer conversations, so they are not guaranteed. With regards to Effort, building both iPhone and Android apps is a significant endeavor, so we estimate this as a 5-point task. Given our uncertainty over the Reach and Value, and that new software development inevitably has some bumps in the road, we put our Confidence of our estimates at 50%. Now, let’s scale these results and create a score:

Now, let’s consider the second task, “integrate with a new data source”. In this case, we have had conversations with 10 current customers who have requested this integration, giving us a Reach of 10. These customers have each signed paperwork committing to upgrading their service to each pay an additional $200 per month to take advantage of the new integration, meaning there is a Value of $2,000 per month. The data integration is similar to one that we’ve done in the past and we’ve already researched their API, so while not completely trial, this is still a pretty straightforward for engineering — we estimate an Effort of 1 point. Given that the paperwork is signed and the execution is pretty clear, this is close to a sure-thing as there is, so we put our Confidence in the estimates pretty close to 100%.

As we can see, even though the creation of a native smart-phone app has a higher Reach and Value, it would take a much more significant Effort and the Confidence in this task is lower, causing the overall prioritized score to be lower than the task to integrate a new data source. So, given the factors we’ve considered, we should prioritize the new data integration over building the app.

After you score each task you are considering, sort them in decreasing order by the score to see what the highest priority tasks are. Remember that this is only an input into your process. You cannot trust these scores blindly for a number of reasons:

• There are a lot of estimates and subjectivity going into these scores. You may have hidden bias.
• There can be other important factors that will influence your decision that are not captured in the four I’ve discussed. It might be best to include these in your scoring if they are common, but if not use your judgement..

So there you have it, a data-driven way to organize your tasks! If you use a different system that works, please share, as I’d love to hear about it!

[1] This is an adaptation of the approach Intercom uses for its prioritization. The main differences in my approach is that I use the term “value” instead of “impact” to tie back to more directly to the financial impact of a task. Also, I scale the value of each factor differently.