Ratings, comments, and collaboration

A requirement for any documentation that I create is that readers can rate and comment on the docs. I’ve been publishing documentation to systems that support this for about 10 years now, with a short gap in there. Not every tech writer agrees that these are necessary, or even desirable, so I’d like to analyze my reasons for being so stubborn about it.

The old way

Back when I started as a writer, creating documentation that was actually going to be printed, the boilerplate text at the front of the manuals included an email address that a reader could use to send comments to the doc team. I’m sure it was written in a very formal way. It might have included a phone number, too, but it was probably a support number. I doubt it was the doc manager’s desk phone.

At some point, I was curious enough to bug my manager to let me look through those messages. I think we got a handful during the 4 years I was at that job, and most of them pointed out a typo in one or another of the shelfload of manuals that we wrote.

Obviously, most users couldn’t be bothered to write to us; only the smallest percentage of pedantic readers even felt like informing us of an error (I’m pretty sure that half the messages came from the same person).

I don’t know if anyone ever called the phone number, but I’m guessing it saw a similarly low volume of complaints.

Why didn’t more users write or call?

I know that our docs weren’t perfect. I know that although we did put a lot of effort into researching and writing them, there were still sections that weren’t perfectly clear. And that was before they were translated into another language.

My guess is that the barrier was too high. Many people were not fully comfortable using email systems (and the email systems were pretty basic and unfriendly). There is also more distance between a printed manual, the tool itself, and their email.

And most users would just plow through and hope that the docs answered their next question, or contacted support. People expected the documentation to be bad, or at least not very good, and in the days before Google the readers had very few other options. If they were lucky, they had a coworker who knew about the system and was willing to answer their questions. Once you had an expert coworker, the manuals would be a rarely used reference.

But are ratings useful?

Ratings are an ego boost. Or a crushing blow, depending on whether it’s positive or negative. The problem with ratings is that a simple up/down doesn’t give you much information. I prefer systems that prompt the user to comment on the rating, or at least for downvotes.

I also prefer systems that force a reader to create an account or log in before they can rate a topic. It adds a barrier to entry, but it also prevents malicious users from spamming your topics with spurious votes. Although I suspect it really isn’t a problem for most documentation, it’s still an annoying possibility. I understand that’s not a rock-solid reason to add that barrier, but I really do hate spammers.

What’s so great about comments, anyway?

Speaking of spammers, I really prefer that comments are associated with a name, and an account that can be blocked. Comment spam is very common, and the doc team (or help site curator) will need to be vigilant about deleting spam posts.

So why bother? Because comment fields are a relatively easy way for your readers to give feedback on the documentation. It’s more immediate than email (the reader can comment directly on the topic), which means your readers are more likely to use it. It can also lead to comment threads, where your readers can add constructive information and help each other. Or they can pile on the docs with a list of errors and general grievances.

I’ve found comments to be extremely useful. In doc sites that I’ve created, commenters have pointed out errors in the documentation, asked for clarification, made feature requests, and have suggested alternate examples.

That’s incredibly useful information, even if I can’t do much about feature requests (other than forward them to the product team). But it allows you to communicate with your readers, to have a conversation with them and learn what they expect from the documentation.

To summarize, in a lengthy way…

Now that comment sections are found on every site, your readers have come to expect them. Doc sets are slowly adopting them, but I still see a lot of resistance to the idea. And although “Never read the comments!” is often wise advice, I’ve found that the good comments far outweigh the bad.

So: Ratings? Sure, they’re fine. They don’t hurt, and can help you identify problem areas. Just don’t let a bunch of positive ratings lead you to neglect those topics; and don’t get depressed by negative ratings (although I’m usually more frustrated when it’s not immediately obvious WHY someone gave a topic a bad rating).

Comments are much more useful, and a great way to engage with your readers. Even though comments are likely to be rare, they’re a good way to communicate with those of your customers who are engaged with your product (and documentation) and who want to help you improve the documentation. It’s also a good idea to let your customer success team know about frequent commenters; it’s valuable data for them, and they can help you find ways to reward and encourage those customers. Which is a great way to get even more helpful comments!

2 thoughts on “Ratings, comments, and collaboration

  1. Neal, you’re right when you say that readers have come to expect the ability to post comments. This is a big change that happened over just a few years, with the rise of social media. I agree that we, as a profession, have been too slow to embrace it.

    Nevertheless, care must be taken. For example it’s worthwhile to distinguish between readers merely posting comments and readers creating user-generated content, although I admit I don’t know where to draw the line. (By “user-generated content” I mean stuff like “I ran into this problem, and here’s the workaround.”) The latter kind of content needs to be monitored carefully — especially in regulated industries and any situation that could result in property damage or physical harm.

    So….We need some best practices on how to engage customers even when they use traditional formats, like PDF, and how high we erect the barriers without making them too high. Thanks for raising awareness.

  2. Thanks, Larry.

    “[User-generated] content needs to be monitored carefully — especially in regulated industries and any situation that could result in property damage or physical harm.”

    That’s a very good point. I like how Autodesk encourages their users to create documentation, but someone at Autodesk reviews it before it’s posted.

    And you’re right that a curator (doc team, probably) needs to keep up with the comments, and make sure that questions are answered promptly. For example, I would often get comments that were support questions (“I don’t see this feature”/”I’m getting an error”), so I would file a support ticket, let the commenter know, and then remove the comment.

Comments are closed.