Here are the following topics:
To date there is no comprehensive usability testing methodology available for documentation. The fact is that software documentation is not used in the same way as the software. Some users only refer to documentation as a last resort, and frequently documentation is only used to answer specific questions.
This paper describes the development of two simple checklists for enhancing the usability and quality assurance of documents. The checklists were used to test the usability of an online user guide for a World Wide Web authoring tool called Tapestry. Tapestry uses drag-and-drop techniques and allows you to create Web pages without learning Hypertext Markup Language (HTML).
Using checklists based on guidelines is a valuable way to identify usability problems in documentation, as well as in software. Technical communicators can use the Usability Index checklist to measure the quality of their writing during the documentation development process. Production people can use the Quality Assurance checklist to ensure that a document contains all the required information before it is shipped to a customer.
The advantages of using checklists to test the usability of documents are:
The disadvantages are that evaluators may introduce bias in their interpretation of questions; however, we attempted to reduce bias by using questions that require a Yes or No answer. This method reduces the wide range of values that are assigned when using a scoring system.
From this study, we learned that the advantages outweigh the disadvantages and that using checklists based on guidelines is an effective way to determine usability problems in documentation as well as in software.
Our experience also taught us that developing new software features has a higher priority than usability testing in the fast-moving Web market. We gained a significant advantage by modeling our software and documentation on systems that have already succeeded in the market to maximize usability.
This project also illustrates the importance of the World Wide Web for distributing user and product information. This paper, the Usability Index and Quality Assurance checklists, the Tapestry software, and support services including Frequently Asked Questions and QuickTime movies are all available on the Web.
Finally, we learned the importance of getting help to solve development problems. Throughout the project friends and colleagues from Concept 1 Communications, Keevil & Associates, SIGDOC, the Society for Technical Communication, and the University of Toronto, inspired, reviewed, edited, and tested our ideas. The cooperation added considerable value and helped us avoid the duplication that would have occurred had we worked alone.
You can also view both the Usability Index and Quality Assurance checklists in table format at Usability Checklists on the Web. Or you can download them in Microsoft Excel format and use them online or as printed forms. If you use the online version, Excel calculates the usability index based on your answers. You can also add and delete questions to better fit the requirements of your own user guides. The checklists are "living" documents and are updated to reflect the latest academic and industry research.
This glossary defines terms and abbreviations that are used in this paper and that may be unfamiliar to the reader.
ACM - Association for Computing Machinery.
browser - A software application for searching the World Wide Web.
checklist - A list of usability and quality assurance questions (for example, "Does each chapter have a clearly defined goal?") that require a yes or no answer.
desktop - The background on your screen that contains the objects for a software application, such as icons, menus, and windows.
dialog box - A box in a software user interface that asks for input from the user.
document - See Web document.
documentation - In this paper, part of the user support information and includes the printed user guide, online user guide, and quick-reference guide. See also user information.
download -To transfer a file from a host computer to a personal computer or workstation.
drag-and-drop - To place your mouse pointer on a screen object, such as a document, press and hold the mouse button, drag the mouse pointer to another screen object, and release the mouse button. With Tapestry software you can drag and drop to create, edit, or link documents. Also known as Drag Transfer.
Finder - An Apple Macintosh program that displays the desktop, opens and closes windows, and keeps track of your files and disks.
Frequently Asked Question (FAQ) - A list of questions and corresponding answers for a particular topic.
heuristic - A general principle or rule of thumb that can guide a design decision or be used to critique a decision that has already been made .
heuristic evaluation - A method for structuring the evaluation of a system using a set of simple and general rules.
hypertext - Text that is organized by links and jumps that move the reader from one piece of online information to another in a nonlinear fashion.
Hypertext Markup Language (HTML) - A markup language that assigns attributes and links in a Web document.
icon - A small picture that represents a screen object, such as a disk, folder, program, or document.
Internet - A large network of computers made up of many smaller networks.
Internet Service Provider (ISP) - A business or service that provides Internet access to consumers.
link - A tag in an HTML document that allows users to jump from one topic to another.
Netscape Navigator - A software application that allows users to ³browse² the Web.
online information - The information stored in a computer system that can be displayed, used, and modified interactively without the need for printed copy. See also, user information.
Quality Assurance checklist - A list of questions that test the production quality of a document (for example, "On the cover is the company logo included and correct?").
QuickTime - A multimedia (video and sound) file format allowing Macintosh and Windows users to create and view QuickTime movies if they have the appropriate authoring or viewing software. You can configure Netscape Navigator (Version 2.0 or later) to play QuickTime movies.
SIGDOC - The ACM Special Interest Group on Systems Documentation reviews and examines the process of producing documentation for the computer industry, the process of producing documentation using computers (including hardware and software), and the end results of the documentation development process.
Tapestry - An Apple Macintosh application that allows users to create and edit Web documents without learning Hypertext Markup Language (HTML).
Uniform Resource Locator (URL) - An address that a Web browser uses to locate, retrieve, and display a document.
upload -To transfer a file from a personal computer or workstation to a host computer.
usability - The quality of a system, program, document or device that enables it to be easily understood and conveniently employed by a user.
usability test - In system development, a test to determine that an implemented system fulfills its functional purpose as determined by its end users.
Usability Index checklist - A list of questions that test the usability of a document (for example, "Does each chapter have a clearly defined goal?").
user - A person who requires the services of a computing system.
user guide - An online or printed book that describes how to use a software application.
user information - The information that helps you to use a software application. It includes the documentation (printed user guide, online user guide, and quick-reference guide), online help information, and support information on the Web, such as the Frequently Asked Questions and QuickTime movies that illustrate how the software works.
user interface - The hardware, software, or both that allows a user to interact and perform operations on a system, program, or device.
World Wide Web (WWW) - An Internet service that allows users to browse linked documents. Also called the Web.
Web document - A file or set of related files that can be transferred from a Web server to a Web client. The document can contain text, graphics, sound, video, or links to other documents.
QuickTime, Finder, and Macintosh are trademarks of Apple Computer, Inc.
Netscape Navigator is a registered trademark of Netscape Communications Corporation.
The authors would like to thank Jane Aronovitch, Joan Cherry, Nancy Hall-Chapman, Roy Hartshorn, Craig Henkle, Jamie Mason, Venicio Melo, Ed Patrick, Keith Soltys, and Joanna Tumminello, whose assistance helped to make this project successful.
1. Cherry, J. M. (1992). Tools for Evaluating the Usability of Human-Computer Interfaces. Seminar presented to the Toronto chapter of the Society for Technical Communication (STC).
2. Cherry, J. M. and Jackson, S. (1992). Online Help: Effects of Content and Writing Style on User Performance and Attitudes. IEEE Transactions on Professional Communication, Vol. 32, No. 4, December 1989.
3. Chignell, M. H. and Valdez, J. F. (1992). Methods for Assessing the Usage and Usability of Documentation. Third Conference on Quality in Documentation, at the Centre for Professional Writing, University of Waterloo, Waterloo, Ontario, Canada, pages 5-27, ISBN-0-9393527-2-7. (Also available on the Internet at URL http://220.127.116.11/ENGL/courses/engl210e/210e/tutorial/usability/chignell/chignell.htm)
4. Hartshorn, R. (1995). Judging Guidelines for User Guides. A talk presented to the Toronto chapter of the Society for Technical Communication (STC).
5. Henkle, C. (1994). Quality Assurance checklist. Personal communication.
6. Horton, W. (1994). Designing and Writing Online Documentation. Second Edition, John Wiley and Sons, Inc., New York, NY. ISBN 0-471-30635-5.
7. Lewis, Clayton and Rieman, John (1993). Task-Centered User Interface Design: A practical introduction. A shareware book. Original files for the book are available by FTP (File Transfer Protocol) from ftp.cs.colorado.edu. The book files are in the directory /pub/cs/distribs/clewis/HCI-Design-Book.
8. Mehlenbacher, B. (1993). Software Usability: Choosing Appropriate Methods for Evaluating Online Systems and Documentation. SIGDOC'93: The 11th Annual International Conference Proceedings. New York, NY: The Association for Computing Machinery (ACM), Special Interest Group on Documentation, 209-222 (paper named the "most innovative of the refereed materials" in the Proceedings).
9. Nielson, J. (1990). Heuristic Evaluation of User Interfaces. ACMSIGCHI'90: The Conference Proceedings. The Association for Computing Machinery (ACM), Special Interest Group on Computer Human Interaction, (Seattle, Washington, April 1 to 5, 1990) ACM New York, pages 249-256.
10. Ravden, S. J. and Johnson, G. I. (1989). Evaluating the Usability of Human-Computer Interfaces: A Practical Method. Ellis Horwood Limited, Market Cross House, Cooper Street, Chichester, West Sussex, P019 1EB, England. ISBN 0-7458-0614-7.
11. Rockley, A., Hutchinson, M., and numerous competition managers in the Toronto and Southwestern Ontario chapters of the Society for Technical Communication (1985 to 1994). Technical Publications Competition Evaluation Forms. A summary of judging forms and procedures prepared for the Toronto chapter of the Society for Technical Communication (STC).
12. Wiklund, M. E. (1994). Usability in Practice, How Companies Develop User-Friendly Products. Academic Press, Inc., Cambridge, MA. ISBN 0-12-751250-0.
Mark Chignell is President of Concept 1 Communications of Markham, Ontario, Canada. Concept 1 is a multimedia software development company which in June 1996, released the Tapestry Macintosh software that allows you to create and edit Web pages without learning HTML (HyperText Markup Language).
Note that as at August 2000, Concept 1 Communications is not open for business.
Mark is also an Associate Professor (on sabbatical until July 1997) in the Human Factors and Information Systems groups of Industrial Engineering at the University of Toronto, where he has been since 1990. From 1980 to 1982, he taught Psychology at Monash University in Melbourne, Australia, and was an Assistant Professor of Industrial and Systems Engineering at the University of Southern California from 1984 to 1990. Mark has a Ph.D in Psychology (University of Canterbury, New Zealand, 1981), an M.S. in Industrial and Systems Engineering (Ohio State, 1984) and was a postdoctoral fellow at the Ohio State Human Performance Laboratory. His research has been funding by a variety government funding agencies, and by companies such as Apple Computer and Ricoh Corporation.
He has co-authored books on expert systems and intelligent databases (published by John Wiley and Sons) and has written many papers on hypermedia and information retrieval. He recently co-authored the chapters on multimedia for two handbooks.
His current professional interests include usability engineering, hypermedia and information retrieval, and computer-based training.
For information on his university research, see Mark Chignell's home page on the Internet.
Benjamin Keevil firstname.lastname@example.org is Manager of the Technical Information Centre of Keevil & Associates, a small part-time technical writing business centered in Toronto, Ontario, Canada, with a network of associates throughout North America.
A member of SIGDOC since 1992, he attended the 1992 conference in Ottawa, Ontario, the 1996 conference in Savannah, Georgia, and presented a paper "Developing Usable Online Information for a Web Authoring Tool" at the 1996 conference in Research Triangle Park, North Carolina. He is also a member of the 1996 program committee for the Toronto chapter of the STC (Society for Technical Communication) and the Professional Engineers of Ontario.
Benjamin grew up in mid-town Toronto. After graduating from the University of Waterloo, he moved to Ottawa, Ontario, where he conducted oil pollution research on the "Behavior of Oil Spilled Under Ice" and helped set up a cable television research institute to develop new cable services. In 1985, he recycled himself and graduated from the Technical Writing Program at Algonquin College, worked as a technical writer at a small computer company, and in 1990 moved back to Toronto. During the past 10 years, he has worked for small, medium, and large telecommunications companies developing well-structured, fully tested hardware and software user guides in printed and online formats.
His current professional interests are usability testing of software user guides and the convergence of computers and communications on the World Wide Web. His hobbies are classic cars, wooden boats, microcomputers, and reading.
For more information, see Benjamin Keevil's home page.
Copyright © 1996 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to re-publish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the Publications Department, ACM Inc., at facsimile (212) 869-0481, or e-mail (email@example.com).
This paper was presented at the ACM SIGDOC96 (Special Interest Group on Systems Documentation) Conference in Research Triangle Park, North Carolina, USA, October 20 to 23, 1996. The theme of the conference was - Marshalling New Technological Forces: Building a Corporate, Academic, and User-Oriented Triangle. Reprinted by permission, © ACM, Inc.
[Contents] [Back to Usability of the Tapestry Software]