Summary


This paper describes the development of a simple checklist that you can use to measure and increase the usability index of your Web site.

Summary

Using a checklist based on guidelines is a valuable way to identify usability problems in Web sites. Information developers can use the checklist to measure how easy it is to find, understand, and use information displayed on a Web site.

The advantages of using a checklist to measure the usability index of a Web site are:

The disadvantages are that evaluators may introduce bias in their interpretation of questions; however, we attempted to reduce bias by using questions that require a Yes or No answer. This method reduces the wide range of values that are assigned when using a scoring system.

From this study, we learned that the advantages outweigh the disadvantages and that using checklists based on guidelines is an effective way to measure the usability index of your Web site. We also learned that to attract return customers, it is essential to update your Web site weekly and use some interactive techniques.

Finally, we learned the importance of getting help to solve development problems. Throughout the project friends and colleagues from Keevil & Associates and SIGDOC inspired, reviewed, and tested our ideas. The cooperation added considerable value and helped us avoid the duplication that would have occurred had we worked alone.


Glossary

This glossary defines the terms and abbreviations that are used in this paper and checklist.

ACM - Association for Computing Machinery.

browser - A multi-platform software application, such as Netscape Navigator, for searching and displaying documents on the Internet or on a local workstation.

checklist - A list of questions that require a yes or no answer. The checklist questions are designed to assess certain quantifiable features of Web documents, such as the conciseness of their writing style and the degree to which their organization is user-centered.

desktop - The background on your screen that contains the objects for a software application, such as icons, menus, and windows.

dialog box - A box in a software user interface that asks for input from the user.

document - See Web document.

download -To transfer a file from a host computer to a personal computer or workstation.

Frequently Asked Question (FAQ) - A list of questions and corresponding answers for a particular topic.

heuristic - A general principle or rule of thumb that can guide a design decision or be used to critique a decision that has already been made.

heuristic evaluation - A method for structuring the evaluation of a system using a set of simple and general rules.

hypertext - Text that is organized by links and jumps that move the reader from one piece of online information to another in a nonlinear fashion.

Hypertext Markup Language (HTML) - A markup language that assigns attributes and links in a Web document.

icon - A small picture that represents a screen object, such as a disk, folder, program, or document.

Internet - A large network of computers made up of many smaller networks.

link - A tag in an HTML document that allows users to jump from one topic to another.

online information - The information stored in a computer system that can be displayed, used, and modified interactively without the need for printed copy.

QuickTime - A multimedia (video and sound) file format allowing Macintosh and Windows users to create and view QuickTime movies if they have the appropriate authoring or viewing software. You can configure Netscape Navigator (Version 2.0 or later) to play QuickTime movies.

SIGDOC - The ACM Special Interest Group on Systems Documentation reviews and examines the process of producing documentation for the computer industry, the process of producing documentation using computers (including hardware and software), and the end results of the documentation development process.

Uniform Resource Locator (URL) - An address that a Web browser uses to locate, retrieve, and display a document.

upload -To transfer a file from a personal computer or workstation to a host computer.

usability - How easy it is to find, understand, and use the information displayed on a Web site.

Usability Index - A measure, expressed as a per cent, of how closely the features of a Web site match generally accepted usability guidelines.

usability test - In system development, a test to determine that an implemented system fulfills its functional purpose as determined by its end users.

user - A person who requires the services of a computing system.

user interface - The hardware, software, or both that allows a user to interact and perform operations on a system, program, or device.

World Wide Web (WWW) - An Internet service that allows users to browse linked documents. Also called the Web.

Web document - The document files stored on a Web server. The document can contain text, graphics, sound, video, or links to other documents. Web documents contain the support information that help people to use software applications, purchase products or services, and search for information.

Web site - The document stored on a Web server that display information about a particular company, topic, or event. A Web site can be a simple 200-word document that announces a community meeting or hundreds of documents that contain detailed support information for a software application. To view the documents, you use a Web browser application running on your local workstation.


References and Web Sites

1. Burnstein, C. D., (1997). The Web site called Best Viewed With Any Browser. (http://www.anybrowser.org/campaign/)

2. Cherry, J. M. and Jackson, S. (1989). Online Help: Effects of Content and Writing Style on User Performance and Attitudes. IEEE Transactions on Professional Communication, Vol. 32, No. 4, December 1989.

3. Chignell M. H. and Keevil, B. E., (1996), Developing Usable Online Information for a Web Authoring Tool, presented at the annual SIGDOC96 Conference in Research Triangle Park, North Carolina, USA on October 21, 1996. (Copy of the paper in HTML format)

4. Chignell, M. H. and Valdez, J. F. (1992). Methods for Assessing the Usage and Usability of Documentation. Third Conference on Quality in Documentation, at the Centre for Professional Writing, University of Waterloo, Waterloo, Ontario, Canada, pages 5-27, ISBN-0-9393527-2-7.

5. Horton, W. (1994). Designing and Writing Online Documentation. Second Edition, John Wiley and Sons, Inc., New York, NY. ISBN 0-471-30635-5. (http://www.horton.com/)

6. Keevil, B. E. (1995). The Web site called Technical Information Centre of Keevil & Associates, includes checklists for measuring the usability of software user guides and Web sites. (http://www3.sympatico.ca/bkeevil/)

7. Mehlenbacher, B. (1993). Software Usability: Choosing Appropriate Methods for Evaluating Online Systems and Documentation. SIGDOC93: The 11th Annual International Conference Proceedings. New York, NY: The Association for Computing Machinery (ACM), Special Interest Group on Documentation, pages 209-222.

8. Molich, R. and Gram, C., (1997). A Web site report on Testing Commercial Web Sites. Technical University of Denmark. 1998. The report, in Danish, is available as a 1.3 MByte file in PDF format.

9. Nielsen, J. (1990). Heuristic Evaluation of User Interfaces. ACMSIGCHI'90: The Conference Proceedings. The Association for Computing Machinery (ACM), Special Interest Group on Computer Human Interaction, ACM New York, pages 249-256.

10. Nielsen, J. (1993). Usability Engineering. Academic Press, San Diego.

11. Nielsen, J. (1994). The Web site of Jakob Nielsen (http://www.useit.com) includes information on Web site usability including a monthly email Alert service that you can subscribe to, for example:

12. Ravden, S. J. and Johnson, G. I. (1989). Evaluating the Usability of Human-Computer Interfaces: A Practical Method. Ellis Horwood Limited, Market Cross House, Cooper Street, Chichester, West Sussex, P019 1EB, England. ISBN 0-7458-0614-7.

13. Spool, Jared M. and others (1997), Web Site Usability: A Designer's Guide, User Interface Engineering (http://world.std.com/~uieweb/index.html) North Andover, Massachusetts, USA. 156 pages.

14. World Wide Web Usability, a Special Issue of the International Journal of Human-Computer Studies (http://www.hbuk.co.uk/ap/ijhcs/webusability/) (1997) 47(1) pages 1 to 222.


Biography of Benjamin Keevil

Benjamin Keevil is Manager of the Technical Information Centre of Keevil & Associates, a small part-time technical writing business centered in Toronto, Ontario, Canada, with a worldwide network of associates.

A member of SIGDOC since 1992, he attended the 1992 conference in Ottawa, Ontario, the 1995 conference in Savannah, Georgia, and presented a paper "Developing Usable Online Information for a Web Authoring Tool" at the 1996 conference in Research Triangle Park, North Carolina.

Benjamin grew up in mid-town Toronto. After graduating from the University of Waterloo, he moved to Ottawa, Ontario, where he conducted oil pollution research on the "Behavior of Oil Spilled Under Ice" and helped set up a cable television research institute to develop new cable services. In 1985, he recycled himself and graduated from the Technical Writing Program at Algonquin College, worked as a technical writer at a small computer company, and in 1990 moved back to Toronto. During the past 10 years, he has worked for small, medium, and large telecommunications companies developing well-structured, fully tested hardware and software user guides in printed and online formats.

His current professional interests are usability of software user guides and the convergence of computers and communications on the World Wide Web. His hobbies are classic cars, wooden boats, microcomputers, and reading.

For more information, refer to Benjamin Keevil's home page.


Trademarks

Excel is a trademark of Microsoft Corporation.

Netscape Navigator is a registered trademark of Netscape Communications Corporation. QuickTime and Macintosh are trademarks of Apple Computer, Inc.


Copyright

Copyright © 1998 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to re-publish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the Publications Department, ACM Inc., at facsimile (212) 869-0481, or email (permissions@acm.org).


This paper was presented at the annual SIGDOC98 Conference in Quebec City, Province of Quebec, Canada, September 24 to 26, 1998. The theme of the conference was Scaling the Heights: The Future of Information Technology. Reprinted by permission, © ACM, Inc.