Related Research


Various researchers [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13] have presented methods for collecting usability data; however, only Chignell [3, 4], Mehlenbacher [7], Nielsen [9, 11], and User Interface Engineering [13] discuss the usability of online documentation and Web sites.

The following topics take a brief look at these ideas, as well as existing usability checklists, before moving on to describe how the concepts were applied to developing the checklist for a Web site.

Defining a Usable System

Before you can study the usability of a system, Mehlenbacher [7] reasoned that you must first determine the characteristics of a usable system. He summarized eight ways to evaluate online systems and documents in a paper at SIGDOC93 and concluded that a usable system is one that is accessible, maintainable, visually consistent, comprehensive, accurate, and oriented around the tasks that users intend to perform.

The Importance of the Navigation Strategies

Chignell and Valdez [4] concluded that online documents should incorporate elements of the "book" metaphor, and include such navigation devices as a table of contents and index. Finding that there is no single measure that tests hypertext usability, they also determined that hypertext authoring requires many of the technical communication skills, such as concise writing and indexing that are required for printed material.

Writing for the Web

Many studies [2, 5, 11] have concluded that users prefer writing that is concise, easy to scan, and objective (rather than promotional) in style [11].

After rewriting and redesigning a Web site for technical users at Sun Microsystems, Nielsen reported [11] that the revised website scored 159 per cent higher than the original site in measured usability. The revisions included:

Other Neilson studies [11] have shown that you can double the measured usability of a Web site by better writing.

The Value of Heuristic Evaluation

Heuristic evaluation is an informal method of usability analysis. In user interface design, heuristic evaluation occurs when a number of evaluators are presented with an interface design and asked to comment on it according to a set of guidelines. Nielson [9, 11] warns that it is important to have several evaluators who conduct independent evaluations. Nielson's experience indicates that 3 to 5 evaluators can usually identify about 75 per cent of the usability problems of a particular design.

Because usability guidelines often include several hundred items, Nielsen reduced the number of rules to nine basic usability principles:

Jakob Nielsen's Web site

On Jakob Nielsen's Web site [11] there is lots of usability information that you can use to develop questions for your checklist. Every two weeks, Nielsen publishes an Alertbox column on Web usability that is distributed by email.

User Interface Engineering

In a 1997 study [13], Jared Spool and colleagues found that users were only successful 42% of the time when asked to find specific information on 15 large commercial sites in the United States. When asked to rate "overall ease of use", people scored these sites 4.9 on a 1 to 7 scale (7 best); somewhat better than the neutral rating. This latter result highlights why it is not sufficient to simply ask people whether they like your site: people tend to be polite and give relatively high ratings even when the site is not usable [11].

User Testing a Web Site

In a 1998 project [8] at the Technical University of Denmark, Rolf Molich and Christian Gram collected data from 50 teams of students who conducted usability tests of commercial websites. The average time spent by each team was 39 hours. Furthermore, the students had previously attended 15 hours of lectures on user test methodology. These numbers are an upper estimate of the time required the first time you decide to user test your site [11].

In this Danish study, the usability problems were defined as:

According to Nielsen [11], with experience it is possible to conduct much more rapid user tests of a site. Good test tasks can be written in one or two hours, recruiting can be outsourced to a focus group company, the actual test can be done in a day, and the results can be analyzed in a few hours. If you are a member of the design team, there is no reason to write an extensive report which nobody will read, so reporting can be done in a one-hour meeting supplemented by a summary that takes 2-3 hours to write. In total, a discount usability study can take only two work days after you know what you are doing.

Even though experts can do the work more efficiently (and usually with better results), it is encouraging that even beginners can complete a full Web usability project in less than one week. This proves that "limited budget" and "lack of time" are not valid excuses for inflicting difficult sites on your users [11].

Any Browser Campaign

The Best Viewed With Any Browser site by Burnstein [1] promotes the use of platform-independent Web pages that can be viewed by any browser. Burnstein believes that some pages may look better in a particular browser but you should be able to read all pages in any browser. This site has many links to Web usability information, for example, The Ten Commandments of HTML.

In addition to reviewing the related research on usability, existing checklists were considered.

Existing Usability Checklists

Several existing checklists served as models in the development of our checklist; for example, the Ravden [10] human-computer interfaces checklist, as well as the checklists developed by Chignell and Keevil [3].

The Ravden checklist includes about 120 detailed questions in various categories, including visual clarity, consistency, compatibility, and information feedback. The Chignell and Keevil checklist has over 180 questions for evaluating user guides and was used as a model for developing the checklist for Web sites.

Web Usability Issues

According to Neilson [11], Web usability problems fall into two categories:

After reviewing the research, we applied the results to Developing the Checklist.