User Story Driven Docs - Full Talk

On May 18, I travelled to beautiful Portland, Oregon to the amazing Write the Docs 2015. I wanted to share with 350 documentarians and developers, my experience of creating user story driven documentation at OutSystems.

You can find the recording of my presentation on youtube. But I also decided to share with you a written version of my thoughts about creating awesome documentation for your users.

I'm Joao Fernandes, and I'm an Academy Engineer at OutSystems. My team and I write the docs, create our free online training, and do a bunch of other stuff to ensure OutSystems Platform is easy to use.

At OutSystems, we have a platform that allows developers to create mobile and web apps in a visual way. You model the data, design the pages, and implement the application logic in a single IDE. And we take care of most non-functional requirements, like logging, security, and scalability.

I'll explain you how we stopped trying to document the UI, and started creating awesome user-story driven docs.

User-Story driven docs is just a fancy way of saying that we now try to focus on what users want to achieve and how to achieve it, independently of how many windows or buttons they need to go through. After all windows, buttons, and tools, are only there so that you can use them to accomplish what you want. Otherwise they have no purpose whatsoever.

A clear example of this, was that we had a full documentation page for the find text feature (CTRL+F). It described in excruciating detail everything you could learn just by looking into the UI.

Just by looking at the UI for 10 seconds, you realize that it's possible to:

• Do case sensitive searches;
• Only match whole word;
• Refine the search scope.

The documentation explained these self-explanatory options, but didn't explain some common tasks our users care about:

• How can I find all elements with a given name?
• How can I find all CSS rules affecting the headings?
• How can I find all queries to a specific database table?

On this post I'll cover:

• Why I think you should avoid documenting the UI;
• I'll give you a checklist that you can use to validate if you're already doing it;
• I'll then tell you the story of how we started to change to create user story driven docs;
• And tell you the process we're currently using to create documentation at OutSystems;
• To wrap-up I'll share what we've learned so far.

So let's get down to it!

Let me start by explaining why I think documenting the UI might not be the best option.

When you're creating documentation for your product, I think it's important to understand the impact that documentation will have for your users. You need to think about them first, since they're the reason you're creating documentation in the first place.

But you also need to understand the impact that documentation will have for you and your team. Afterall you'll be the one maintaining those docs on the foreseeable future.

Let me focus first on why I think documenting the UI might not be the best you can do for your users.

When you're documenting windows, buttons, or tools, you're essentially creating a siloed and microscopic view of your product. You're only focused on explaining how a minuscule part of your product works, without giving the proper context about the situations when it is used.

But what your novice users really care about, is understanding how the several tool and features are related to one another. They need to understand how to combine and chain them, in a way to produce the output they want.

So if you want to onboard your users really fast, what you need to do, is to give them mental models of how the tools are related to one another. How they can be chained, combined, transformed to produce a given output.

This means that if you're only documenting windows, buttons, and tools, you're essentially delegating to your users the task of creating these mental models. You're not making life easier for your users!

Shame on you and shame on us, given that we were making this.

Just to give you an example, we had around 13 doc pages just to explain how to query data from the database. Even if a novice user read all this, it was still pretty difficult to get a sense of how to implement what they really want.

As a developer, I want to fetch data from the database, in order to display it on a page!

And if you think about this, it makes lots of sense. Queries by themselves are useless most of the time. So you need to learn how two concepts are related - Query and Page.

And it frustrated me, having documentation that didn't help our users create mental models of how these two concepts are related. It frustrated me, because implementing this user story is pretty straightforward with OutSystems Platform.

For example, if you want to create a page that lists all your customers, you just need to drag and drop the customers table (on your right hand side). If you do this, the OutSystems IDE creates a screen for you.

This screen, that has been automatically created from a drag and drop, does what you want. Yes, it fetches the existing data from the customers, but also displays it on a screen.

So now, you just need to deploy your app. Click on that green button an BAM!

Now you've got a running application! So the documentation we had frustrated me, because not only it didn't help you understand how to query data and how to display it on a screen. Not only that, but the docs made it sound so complex, that users ended with the feeling that this user story was difficult to implement with the tools we've got.

And if documenting the UI makes life difficult for your novice users, I'm afraid the life of your expert users won't be easier.

From my own experience, when you're focused on documenting windows, buttons, and tools you end up mixing information that should be part of a user guide, with information that is purely reference. Believe me. I've been there and done that.

And when you mix up these kinds of information bad things start to happen. You start receiving feedback from your expert users just like this. And yes, this is a real quote from feedback we received by one of our valuable expert users.

Basically he said he had stopped searching on the documentation, and had to rely on his own experiments, posting questions on the OutSystems community, and sending support tickets. I can't even start imagining his pain.

If you are willing to make tests yourself to understand the expected output, and if you're willing to wait four hours for a response from the community or support, I can't imagine how useless the existing docs might feel to you. I can't say "I'm sorry" enough times to make this right for this developer.

And trying to document the UI not only has negative side effects for your users, but also or you and your team. Let me explain you why.

When you're focused on documenting windows, buttons, or tools, everything has the same priority. There is no single window that is more important that the others. There is no single button that is more important than all the others.

And if everything has the same importance, how can you prioritize your work? How can you prioritize your work, when there are multiple changes being made to the UI at the same time?

And as you start trying to decrease your product release cycles, and deploying your product with more frequency, it starts to get hard to keep the docs updated.

When deploying frequently, you stop having time to update everything, and so you start having a huge maintenance backlog. Issues that you need to address after the product has been release.

And so the timer has started ticking. The product is out, and the docs are outdated.

But this isn't even the real problem. The real problem is that soon, to fight the huge maintenance backlog, you start focusing too much on the problems you have right now. You start holding too much to your internal processes, that you start losing sight of the broader picture.

The broader picture is that your internal processes and excuses don't matter to your users. You need to make sure you deliver value to them.

And at the OutSystems Academy, delivering value to our users means that

We help our users become badass at developing mobile and web apps.

Or in other words, a huge maintenance backlog can break my bones, but user dissatisfaction is what hurts me.

So let me give you a checklist, that you can use to validate if you're documenting the UI.

The first symptom is that your users complain that they can't create mental models. The funny thing here is that you can't learn this by looking into Google Analytics. You really need to get your ass up from your chair and get some old customer feedback.

Interviews and over the shoulder observation, might work best to find out what your users think.

The second symptom is that everything feels like reference, and you're essentially giving your users, a siloed view of your product. You're not explaining how things are related to one another.

Another symptom is that, some doc pages might be slightly different, but end up stating the same. This happens because you feel the need to give more context to your users, but you end up putting the spotlight on a window, button, or tool, instead of a whole user story.

Another symptom is that although your customer base might be growing, you have some doc pages that remain with a constant number of page views throughout the time. This can mean one of two things:

• The information isn't relevant for your users;
• The information is relevant, but most users can't reach it.

And the final symptom is that you have a huge maintenance backlog, because the development teams keep on making changes to the UI.

So, can you guess how many of these symptoms did we had at OutSystems?

I bet you've guessed it right. We had ALL OF THEM.

So let me tell you the story of how we started to change. How we've stopped trying to document the UI and started creating documentation for whole user stories. It doesn't matter if a user story needs to use several features, transverse several windows, or click multiple buttons. What it matters is telling users what they need to do, to implement what really matters to them.

Around May 2012 we launched OutSystems Platform 7. One of the core features of that release, was a new tool to manage the application lifecycle process.

Since this was a new tool, build from the ground up, we had an opportunity to do things differently. And so we took that project with a different mindset. We sat down with the development teams, product management, and support to understand what features the tool would implement.

After some discussion, we identified around 18 user stories that we needed to implement. From the documentation standpoint, we decided to only document those 18 user stories only. This means that there wouldn't be any documentation for a particular window.

And since we had contextual help, when you clicked for help on a given page, you would see documentation around a specific user story, like:

• How to deploy my app;
• How to assess the performance of my apps;
• How to manage the permissions of users.

After starting the documentation project, we started noticing that this project was kind of easy to manage.

The project was easy to manage, because most of the dependencies were laid out in front of us. For instance, it didn't make sense to write documentation that explained how you can perform runtime customizations to your app before ensuring we had good documentation on how you can deploy you app.

The project was fun. And had some positive feedback both internally, and most importantly from our users.

But then we unfortunately backtracked:

• Our team had some internal changes;
• We started revamping our free online training, so we had to focus all resources on that project.

Fast forward two years, and it's May 2014. We were only four months away from another major release.

After seeing how the application lifecycle management project had worked out, I was getting kind of frustrated by seeing us repeating the same mistakes.

So I tried to show the team that we had progressed in the right direction, and now we were running on the opposite direction. For this, I showed them some documentation projects I think are amazing. I showed how we could be doing much better.

But I knew that this wouldn't be enough to make us change. So I also showed the team how I had documented one of the flagship features we were planning to release. And the documentation was only with user stories.

Again, there was no need to be documenting windows or buttons.

And so by that time, we all agreed that we should stop trying to document the UI. We needed to focus on giving mental models to our users, of how the several tools and concepts are interrelated.

But we didn't had a clear vision of how to achieve this. So it took us lots and lots of discussions until we reached consensus on what should be documented for a given user story.

By October, we had released OutSystems Platform 9. Documenting with user stories took lots of effort, so as we were already expecting, some smaller features ended up being documented in a mix o user story and UI.

By that time, we were all exhausted. So we decided to that that as an opportunity to step back and do a team retrospective. Just to get a detached sense of what was working, and what wasn't.

From that retrospective we learned that:

• This was the first time in years, that we had the documentation ready for a Beta;
• We had positive customer feedback, even during the Beta;
• It was easier to prioritize the work if we focused on user stories;
• And the discussions themselves became easier. We could leave our gut feelings aside and focus the discussion around a user story.

So the only thing we were missing at the time as adding a bit of process. Just to ensure with delivered with high quality and consistently, without taking us so much effort.

So let me walk you though the process we've been using so far, to create user-story driven documentation.

At OutSystems most teams work in a mix of SCRUM and Kanban. This means that there's usually a project kick-off, and demos at regular intervals.

So the development teams host a project kick-off at the beginning of a new project. They invite us, product management, and other relevant stakeholders. And then we agree on the user stories to implement, as well as their priorities.

Then, we use the development team demos as checkpoints. This allows us to understand how the project is evolving, measure the impact the changes will have on the product learning experience. We also start trying to understand when is the right time to start creating documentation and training materials.

If we start documenting too soon, we risk creating documentation for features that are not stable enough. As such, things might change so much, that we'll have to redo everything from scratch.

But on the other hand, if we start too late:

• We risk not being able to ship the docs when the product is launched;
• But most importantly, we risk no being able to give meaningful feedback so that the development teams can make the new features easier to use.

When we feel the timing is right, we start implementing the user stories we've agreed, with the new tools available. At the same time, we start trying to understand what's the best example we can use to explain a user story.

For example, if the user story is

As a developer, I want to fetch data, in order to display it on a screen

should our example be about fetching orders, tickets, contacts, or something else?

And again, when we finish this stage we've collected tons of feedback about the new features. So we take that feedback, prioritize it, and give it back to the development teams.

And then we start creating the docs. At this stage we write drafts, which means that we don't care about screenshots or formatting the text.

We only care about two things:

• Correctly identifying a user story, 80% of our users care about;
• Finding a simple example that our users can identify with. It makes it easier to understand with a concrete example, than an abstract explanation.

And we've got a round of feedback with the stakeholders, to ensure we've nailed these two down.

And to close our process, we've got a polishing stage where we:

• Add meaningful screenshots;
• Format the text;
• Ensuring the text is easy to read.

And we've got another round of feedback to close all this.

To wrap up this post, let me share with you what we've learned since we've started changing, back in 2012.

You should see small projects, as opportunities to innovate. They are a great way to experiment without much risk. They're also a great way to receive feedback, and gather support from other team members.

We've also learned that someone needs to lead these changes, and that someone is you! I mean, if you're frustrated with something, you should complain, but you should also do something about it.

In my own case, I was frustrated with:

• Documentation that didn't show our users how easy and powerful the OutSystems Platform is;
• Documentation that didn't help our users create mental models, and learn how the features and tools are related to one another;
• An ever ending backlog, that we just couldn't keep up with.

I saw how user story driven documentation could help us address some of these issues, and so I decided to document a new feature with user stories only. And yes, at the beginning most of the feedback I got, was that I was being inconsistent with the documentation we had at the time.

But slowly the feedback started to change, and now, user story driven documentation is the status quo at OutSystems.

And finally, we've also learned that creating user story driven docs can make us better developer advocates.

When discussing something, either inside our own team or even with the development teams, we can all leave our gut feelings aside. We can focus on the user stories most of our users care about. When we focus a discussion around a user story, only two things matter:

• User stories that are frequently executed by your users, need to be seamless to implement with the tools and features you've got available;
• And it's OK that user stories that are not that frequently executed, or most of your users don't even care about, to be clunkier to implement.

Just to give you an example of the impact we're having, just on 2015 we were able to convince the development teams and product management to stop developing two features, that otherwise would be part of our product right now.

This means, that we have a more focused product, and less code to maintain.

And we've also started to receive some positive customer feedback, since we've started to change.

And that's it! I think that we still have a long way to go, and there's still tons to learn.

I'd like experimenting using this process to document frameworks, APIs, or even some other stuff.

Only this way, we'll see the real limitations of this process.

I'm sure there are lots of awesome documentation projects that are following similar approaches for creating documentation. If you're doing it, I would love to hear your own story. So please reach out and tell me about your own experiences!

I'd like to thank Pedro Nogueira for his invaluable patience, help, and conselling. Without him this post wouldn't be as awesome. Thanks!