At OutSystems we work hard to make sure our platform is easy to use. A small part of this is providing contextual help. Imagine you find something you don’t understand in OutSystems IDE, or are stuck using a certain feature.

In a perfect world you would never get stuck, and would easily learn every feature available. In a less perfect world, you would press the F1 key (conventional way of getting help in the Windows platform) and we would direct you to an help page that has all the information you need to get unstuck.

So this is a two-step job: ensuring that every element on screen has its own help page, and ensuring that help page has the minimum necessary information to help you using that feature. The topic itself does not need to be complete or explain the feature in depth. It just needs to ensure that if you land on that page you can learn the minimum necessary about the feature so that you can use it and get unstuck. The topic should not bore you with irrelevant details, but should point to more advanced topics that might help you explore the feature in depth.

So one of these days I was thinking that no one uses F1 to reach the contextual help. I didn’t knew why I felt that way, since I have no data to back it up. So I started thinking that probably I was biased, and was extrapolating based on my own experiences.

On personal bias

You need metrics to guide your decisions. Without them you are biased and you don’t even realize it. When someone asks you difficult questions, your brain translates those questions in simpler questions. This is a kind of shortcut we developed and allows us to think fast.

An example of this is when someone asks me “How many people press F1 to get help on the Windows platform?”. At first sight, it does not seem such a hard question because in no time I have formed an opinion about it. In this case my quick opinion was that no one uses this feature.

The problem is that in order to come up with an answer so quickly, I didn't even realize that I was totally biased and only thinking about my own experience with the IDE. My experience is not the same as the rest of the 99.9999% users.

So when I heard that question, to speed up things, my brain translated the question into a simpler one: “Did you ever used the contextual help feature, either on the OutSystems IDE, or on any other Windows application at all?” The interesting part is that I didn’t even realized that the brain had translated the original question into a totally different one. So I did not realize that the answer I got might not be correct for the original question.

As a side note, this is why you should go to the supermarket after eating. If your stomach is empty, your brain will translate the question “What should I cook for tomorrow’s dinner” with the question “What should I eat now?”. And you’ll probably buy stuff that fulfills your immediate needs, and not the needs you are going to have in tomorrow's dinner.

The importance of metrics

The only reliable way of overcoming these biases is by measuring. In this case you’ll need to count the number of people that pressed F1 to get some contextual help.

After you’ve collected these metrics for a certain period, you can then see if the time it takes you to implement and maintain this feature is really worth it or not.

A good example of metrics guiding decisions

One of the best examples I know of using metrics to guide decisions was the 2003 version of Microsoft Office. In 2003 Jensen Harris and the Office team launched a cleaner version of the Office Suite. One of the biggest changes they introduced in the product was what they called “The Ribbon”.

Now, whether you think that the Ribbon is a good or bad design decision does not matter. What matters is that the Office team collected lots and lots of metrics to guide their decisions.

A good example of overcoming bias was their decision of leaving Save, Undo and Redo buttons in the interface. They even promoted these operations in the interface, to make it easier to find the buttons for these operations.

If you are a proficient user, you will always use CTRL+Z and CTRL+Y (or SHIFT+CTRL+Z) to perform undo/redo operations without taking your hands out of the keyboard.

So if you worked at the Office team, and someone asked you “Should we hide the undo/redo buttons from the interface”, your instinctive answer would be YES. Your translated the original question into “Do I use the undo/redo buttons often?”, without even realizing.

But since they acknowledge their personal bias, they went ahead and collected metrics of the buttons and shortcuts usage. Low and behold. They observed that 50% of users still used the undo/redo buttons in the interface. After watching that 50% of your customer base uses these two buttons instead of their shortcuts, it’s somewhat clear what you need to do.

If you are want to know more about the Office 2003 case study, I suggest you watch Jensen Harris presentation on the Story of the Ribbon.