User testing in relation to software development (incl. website design) is the phase during which persons other than the software developers themselves give the system a good going-over in an attempt to identify and – with the help of the software developers – resolve all issues that would otherwise stop the system going live.
Naturally the larger, more sophisticated, and more mission critical the system, the more attention must be paid to testing, user or otherwise. I (Ross Gerring) began my working career as an analyst programmer for a large international bank, and so I’ve seen first-hand the huge amount of testing, time, process and technology that goes into the rigorous pre-live and post-live examination of banking systems. The following advice is much more aimed at the relatively much smaller and less complex world of user testing for websites.
In no particular order, here are my top 10 hints and tips for website user testers:
- Expect bugs! Don’t be shocked that they exist! Bug or issue resolution is a completely normal and expected part of any software development lifecycle. Trust us on this one.
- Issue resolution is an iterative process. An issue is not resolved until you say it is. It might take a bit of to’ing and fro’ing between yourself and the developers until an issue is finally put to bed. Sometimes it might even feel like you’re going 1 step backwards for every 2 you go forwards (e.g. previously squished bugs reappearing). Expect this. It’s the nature of the beast.
- If you do nothing else, make sure you’re using great tools to help you report issues to your website developer, and then for the issues to be well managed thereafter until a satisfactory resolution has been reached. We use, and are big fans of, Bugherd. There are lots out there: 17 Bug and Issue Tracking Apps to Developers.
- Indicating priority is important. Make sure you can flag issues in terms of their business criticality, e.g. critical, important, normal, or minor. Remember this is about YOU telling the developers what’s business critical and what’s not. They could take educated guesses, but you don’t want them too. Generally speaking developers will tend to hit the most critical, showstopping items first. Here’s an example of priority and category labeling from the Drupal project: Issues for Drupal core.
- Grouping, categorising, or tagging issues is also very helpful. Suggested tags might be: typo, error message, graphic, broken link, content, logo, layout, etc.
- Try to distinguish between feature requests and bugs/errors. There’s no such thing as a finished system (it’s a competitive world out there – you can’t stop innovating!) so be careful about getting carried away at this stage with all the new stuff (feature requests) you’d like to see. Your ideas are important, but ultimately the business has to decide between “must haves” and “can wait”.
- Developers will typically respond to issues in batches. Your labeling of issues by priority and type will come in handy here. Issues of a similar nature can be more efficiently dealt with by developers, not least because fixing one of them will often fix lots of them at the same time.
- We all strive for perfection, right? News flash: there are very few (arguably zero) perfect systems out there. And by perfect we include ‘bug free’. There’s a genuine business trade-off to be made between getting something out to market quickly, versus spending more time attempting to getting it closer to perfection and with all the associated diminishing marginal returns that come with that effort. Indeed, some systems go live to the world with the word ‘Beta’ (or even ‘Alpha’) prominently displayed. ‘Beta’ is acknowledging the existence of bugs – perhaps lots of them – and is an open invitation to the public, typically motivated or incentivised early adopters, to help the development of the system by reporting back their experiences. A beta release of software is very suitable for some systems, but totally unsuitable for others – e.g. banking systems! Bottom line: there’s no such things as a finished or perfect website – in the same way that a business is never ‘perfect’. There is always room for improvement. Got ideas on how to improve your website? Great! Jot them down on a wishlist for the future when time, budget and priority permit – but don’t let them hold you back making your otherwise awesome website live!
- Where possible and reasonable, DIY, i.e. fix an issue yourself. “What?! Isn’t that job of the developers?”. Not necessarily. Most websites have a content management system (CMS) behind them, i.e. a web interface to allow you to relatively easily self-manage many facets of the site yourself, especially content. Often it’s the future content authors of a site that are the user testers, and therefore it stands to reason that the user testing phase is the perfect opportunity for such persons to learn all about how to self-manage the site to the fullest extent that the CMS permits. Broken link? Typo? Photo needs replacing? DIY! Much better to learn how to (or not to!) do this now, than when the site is live.
- Consider automated testing of your website using software dedicated to the task, e.g. Selenium. If you’re finding yourself having to do lots of re-testing of a repetitive and/or complex nature (e.g. testing a shopping cart process), a more automated approach might help, This will requires its own investment (time and expertise) to set this up, which might be extensive, and therefore it’s a cost/benefit decision.
Bonus Tips
- Naturally you’ll want to give most of your attention to the most mission critical components of your site, but this assumes that you know which these are! The larger and more sophisticated your website, the more important it is that you have a plan in advance of starting to test. At minimum, ensure you’ve identified all the essential functionality of the site (so you don’t miss anything important), and more generally that you’re clear on the size/breadth of your site so that you don’t accidentally overlook sections. Be sure to include any instances of where your site exchanges data with external systems, .e.g. are your ‘Contact Us’ form submissions getting through to the inbox of the correct recipient(s), and is your newsletter subscription form to MailChimp or CampaignMonitor working as intended?
- Avoid reporting duplicate issues. It just floods your developers with extra reports that they have no choice but to tale time to review. Common causes of this are:
i. Multiple persons testing the same site… so perhaps make clear rules about who’s testing what areas of the site, or which functionality.
ii. Reporting the same issue in lots of places on the site, even though you only need to report it once. So wherever you have common or recurring functionality, design, or layout on your site, rest assured you only need to report the issue where it occurs on a single page, and not every page where you see the same issue. For example: the font size might need to be bigger for the title of each page. When your developers resolve the issue in question, it will almost certainly resolve the same issue on all other pages simultaneously.