3

I am the product owner of an application used internally at our company. Our employees are consultants so time or rather saving time is essential to them.

Based on feedback and interviews we developed a new feature that we estimate save the consultant about 20 seconds every time (from 30 to 10 seconds). Now I would like to present the improvement we created and obviously looking at the time saving would be natural.

The problem is that so far the feature has been used about 2000 times which equals a time saving of just 11 hours which is nowhere near the development cost of this feature. Just looking at this KPI in itself could never motivate the feature.

But given the positive feedback we received after launching this feature I strongly believe that the actual value is a lot bigger than just the time saving itself. The situation we moved from was a pain and required many steps and we know that this was at the top of the list of what the consultants wanted.

What are your thoughts about quantifying those "soft" values?

Improve this question
1
  • I think that some of the other quantitative metrics might not have been captured since you only focused on time. By making things quicker to do you may have simplified upstream or downstream processes, removed bottlenecks or reduced error rates. A holistic approach is required to understand the end-to-end process, and benefits that might not just sit within the application itself (perhaps even other parts of the organisation). Commented Apr 17, 2023 at 21:57

3 Answers 3

Reset to default
1

It sounds like you might be looking for a happiness metric, which is part of the larger HEART framework for measuring improvement.

Here are some questions you could ask to measure happiness in both your old and new designs, and then you can compare the scores to evaluate improvement:

I found this feature easy to use.

  1. Strongly Disagree
  2. Disagree
  3. Neither Agree Nor Disagree
  4. Agree
  5. Strongly Agree

I would use this feature even if there was another way to do it.

  1. Strongly Disagree
  2. Disagree
  3. Neither Agree Nor Disagree
  4. Agree
  5. Strongly Agree

I would recommend this feature to someone in a similar role who is doing this task (0 - 10, NPS)

Improve this answer
0

...quantifying those "soft" values...

...is called Qualitative Research.

What is qualitative research?

Qualitative research is a study methodology that focuses on unquantifiable traits or responses, such as opinions about an event or interviews about personal experiences. This method aims to describe various events while generating theories and learning what causes them and what effects they have.

To help people see the impact qualitative aspects contribute to the overall achievement of the organization's goals, show consultant comments about the new features, their level of frustration or satisfaction with the old method versus the new, describe before and after processes accomplishing particular tasks. A study of Qualitative Research will introduce more techniques and metrics that shed light on the less tangible principles affecting people's attitude and performance.

We have Quantitative and Qualitative research methods, each lending to different perspectives, values, and outcomes on the matter examined.

It seems people tend to pay more attention to qualitative values, such as "we saved 11 hours of time". Likely because these metrics are easier to collect, measure, compute, communicate, and grasp.

However when people overly apply this perspective we deride them as "bean-counters", and "number-crunchers" in response to the pain we feel when the less measurable and understandable qualities are ignored.

The broad spectrum of important matters to study do not all require both methods to tease out an effective understanding and therefore response—although the area of overlap is great. Carefully consider each matter at hand to determine the combination to use and how much weight to give one or the other.

Businesses tend to focus on the short-term quan-titative metrics—the bottom line. But when important qual-itative needs are left wanting the long-term quan-titative metrics suffer along with them.

Improve this answer
0

If you'd like to quantify it, there are a couple of potential key candidates:

  • What percentage of total users are using the new feature
  • Of the 2000 uses, how many are repeat users?

But really, it's whether the feature prevents confusion/ annoyance/ redundant actions/ unnecessary bloat in the process. If it does, then your feature is doing a lot more good than just reducing time taken to complete a process.

It's just how our brains work. If a process causes confusion for example, it can cause potential errors and reduce success rates.

I'd suggest to add a 'Give feedback' button right inside the feature, and ask direct questions as Izquierdo has suggested.

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.