User testing in relation to software development (incl. website design) is the phase during which persons other than the software developers themselves give the system a good going-over in an attempt to identify and – with the help of the software developers – resolve all issues that would otherwise stop the system going live.
Naturally the larger, more sophisticated, and more mission critical the system, the more attention must be paid to testing, user or otherwise. I (Ross Gerring) began my working career as an analyst programmer for a large international bank, and so I’ve seen first-hand the huge amount of testing, time, process and technology that goes into the rigorous pre-live and post-live examination of banking systems. The following advice is much more aimed at the relatively much smaller and less complex world of user testing for websites.
In no particular order, here are my top 10 hints and tips for website user testers:
- Expect bugs! Don’t be shocked that they exist! Bug or issue resolution is a completely normal and expected part of any software development lifecycle. Trust us on this one.
- Issue resolution is an iterative process. An issue is not resolved until you say it is. It might take a bit of to’ing and fro’ing between yourself and the developers until an issue is finally put to bed. Sometimes it might even feel like you’re going 1 step backwards for every 2 you go forwards (e.g. previously squished bugs reappearing). Expect this. It’s the nature of the beast.
- If you do nothing else, make sure you’re using great tools to help you report issues to your website developer, and then for the issues to be well managed thereafter until a satisfactory resolution has been reached. We use, and are big fans of, Bugherd. There are lots out there: 17 Bug and Issue Tracking Apps to Developers.
- Indicating priority is important. Make sure you can flag issues in terms of their business criticality, e.g. critical, important, normal, or minor. Remember this is about YOU telling the developers what’s business critical and what’s not. They could take educated guesses, but you don’t want them too. Generally speaking developers will tend to hit the most critical, showstopping items first. Here’s an example of priority and category labeling from the Drupal project: Issues for Drupal core.
- Grouping, categorising, or tagging issues is also very helpful. Suggested tags might be: typo, error message, graphic, broken link, content, logo, layout, etc.
- Try to distinguish between feature requests and bugs/errors. There’s no such thing as a finished system (it’s a competitive world out there – you can’t stop innovating!) so be careful about getting carried away at this stage with all the new stuff (feature requests) you’d like to see. Your ideas are important, but ultimately the business has to decide between “must haves” and “can wait”.
- Developers will typically respond to issues in batches. Your labeling of issues by priority and type will come in handy here. Issues of a similar nature can be more efficiently dealt with by developers, not least because fixing one of them will often fix lots of them at the same time.
- We all strive for perfection, right? News flash: there are very few (arguably zero) perfect systems out there. And by perfect we include ‘bug free’. There’s a genuine business trade-off to be made between getting something out to market quickly, versus spending more time attempting to getting it closer to perfection and with all the associated diminishing marginal returns that come with that effort. Indeed, some systems go live to the world with the word ‘Beta’ (or even ‘Alpha’) prominently displayed. ‘Beta’ is acknowledging the existence of bugs – perhaps lots of them – and is an open invitation to the public, typically motivated or incentivised early adopters, to help the development of the system by reporting back their experiences. A beta release of software is very suitable for some systems, but totally unsuitable for others – e.g. banking systems!
- Where possible and reasonable, DIY, i.e. fix an issue yourself. “What?! Isn’t that job of the developers?”. Not necessarily. Most websites have a content management system (CMS) behind them, i.e. a web interface to allow you to relatively easily self-manage many facets of the site yourself, especially content. Often it’s the future content authors of a site that are the user testers, and therefore it stands to reason that the user testing phase is the perfect opportunity for such persons to learn all about how to self-manage the site to the fullest extent that the CMS permits. Broken link? Typo? Photo needs replacing? DIY! Much better to learn how to (or not to!) do this now, than when the site is live.
- Consider automated testing of your website using software dedicated to the task, e.g. Selenium. If you’re finding yourself having to do lots of re-testing of a repetitive and/or complex nature (e.g. testing a shopping cart process), a more automated approach might help, This will requires its own investment (time and expertise) to set this up, which might be extensive, and therefore it’s a cost/benefit decision.
And here’s a hopefully obvious bonus tip #11 – be thorough! Sure, give extra attention to the more business critical components of a website, but this still assumes that you’ve started out with a solid understanding of the breadth and depth of the testing that you’ll be doing.