How to evaluate and assign urgency to CRO and website tasks

We talk a lot about prioritization frameworks in website optimization. An unprioritized backlog of issues that need to be fixed or tested on a website can start weighing down your team really quickly. Not a lot of priority models have built-in urgency scores, however.

I feel like this is because it’s a lot more complicated to determine what is urgent and what is not. Especially for bigger organizations with lots of stakeholders. 

Without applying rules and a mental model to the issues reported, absolutely everyone will feel like their issue is of the utmost urgency and that their whole life could fall apart if the Jira ticket/test idea/bug report doesn’t get immediate attention from the website team. There are teams that look at and prioritize "just fix it" items separately from experimentation ideas but for the purposes of looking at an average web team's responsibilities holistically, I’m talking about categorizing all research insights, because at the end of the day, not all issues should be tested, but all ideas should still be prioritized and assigned urgency.

When this holistic understanding of how to assign urgency to issues across the organization is missing, you’ll see a ton of friction and frustration between teams who in theory should rely on one another. Each team has its own KPIs and goals to achieve so naturally, the issues that need to be fixed are urgent to any specific team trying to own the solution to the problem.

For more context, I’m talking about a typical structure where the website tasks are owned either by a single person or a small team depending on the size of the company. The web team has to work with a ton of stakeholders whose professions might include branding, sales, customer service, or any other marketing activity within the company. Still, there will come a point where every one of these stakeholders will need to either change, add or improve something on the company website. 

How this process usually goes, is that the stakeholder will submit the issue or idea through some predetermined process to the web team and the web team will add it to the backlog and try to prioritize the issue or task at hand. 

Things website teams should consider when assigning priority to issues and ideas:

1. How did the problem/idea come to light?

   Is it just someone's personal idea/opinion or was it an insight from a research activity like user testing, customer feedback, analytics data, or surveys? Do different research activities point to the same problem? The more evidence you have that this is actually a problem for the customer, the higher the priority should be.

2. Are there any experiments running or other work being done on the page where the problem exists? 

   You shouldn’t make any huge changes to the site while any tests are running, for example, this will invalidate the results and mess with your experiment which means you’ve lost a lot of time and resources. 

3. How many people are actually looking at the page where the problem lies? 

   Having a proper sample size goes of course for experimentation as well, but it makes sense to consider this even for smaller issues that just need to be fixed. When your backlog is full of tasks related to very high-volume pages, you cannot spend time and resources on an issue that bothers a couple of people per month. 

4. How much time and resources, in general, will it take to fix this issue?

   If someone proposes a task that will take up all of your design and development resources for the next few months, it should score lower on the priority list. This is an especially difficult area to navigate as it can be hard for marketers to understand and evaluate how much resource is needed from developers to bring an idea to life. Switching out some copy or images is not an equal task to building new functionality. 

5. Are the issues reported device and browser specific? How popular are these?

   A site should of course function well across all browsers and devices. However, in the case of a really backed-up backlog, this can serve as a helpful data point in your prioritization model. 

6. How much money could this potentially make for the company? 

   This is more related to launching experiments and can be calculated based on your sample and conversions. Knowing where the highest impact comes from is essential for prioritizing tests.

Now let's say you already have a backlog full of 3 months of work and the newest task submitted will land somewhere in the middle in terms of prioritization. This means you won’t probably get to it until next month at least. You communicate this to the stakeholder who submitted said task and they are not happy. They are not happy, because the task to them is…URGENT. To you as the website specialist it is not, because you already have a backlog of tasks, all marked urgent by a bunch of other people in the company. For the stakeholder at hand, however, pushing that task for a month can mean that a lot of other important things will get held up for their team and their boss will yell at them for not getting it done. 

What’s your next move? Which should be established first in this case - urgency or priority? Should urgency score be one of the variables that determine the priority of the issue? 

The two main things that can help assign urgency are:

1. Is the company losing money right now because of this?

2. Is this preventing customers from completing the main conversion on the site, are they not able to navigate and find the most basic info necessary to do what they need to do?

In our work at Koalatative, we take a simplified approach to this. We assign three different levels of urgency to ideas, tasks, and problems we find on a website. 

• Low urgency - The issue is not costing anyone any money, customers can complete necessary actions, but when the team gets some time, we should still address this

  • Cosmetic changes

• Medium urgency - Related to a company-wide launch of a new campaign/feature on the site that has a deadline

  • For example branding related initiatives

• High urgency - A really huge problem falls in the category of one or both:

  • losing the company a ton of money 

  • frustrating customers as they are not able to complete their tasks comfortably

Urgency and priority are not the same things although they sometimes get treated as such. A better way to go about would be to either:

1. include urgency in the priority score as a factor but give it more weight than all the other factors, or

2. look at them as two completely different categories.

Depending on how your team organizes tasks, the first might make more sense in the context of test ideas and the second for general “just fix it” items.

CRO and web teams should function as a bridge between different departments and stakeholders. Whether you factor in urgency in your priority score or not, it helps create clarity for filled-up backlogs to look at urgency and prioritization as two separate things that influence one another.