aifia-metrics AT lists.ibiblio.org

Subject: Towards standard methods and metrics for evaluating IA

Description: This list is for discussions of evaluation methods and metrics for information-rich websites and non web-based applications. The goal of the project is described below. Please join in the conversation!

Project Goals:
Currently, there are no standard methods and materials available for evaluation of information architecture. Usability questionnaires such as WAMMI (Website Analysis and Measurement Inventory) or QUIS (Questionnaire for User Interaction Satisfaction) are not suited to information-rich domains. Moreover, neither of the both can be used only after payment of a fee. Our goal is to propose some standard methods and questionnaires suitable for the use by Information Architects.

What can open and free standards do for us? Many of the evaluations conducted by IA's are designed to answer the same core set of questions: can user complete task/find information? are users satisfied with site? number and type of errors, usability problems etc. However, it remains difficult to compare results of different evaluations because of the lack of standardization of evaluation methods.

Our goal in creating free and open standards is to allow IA's to benefit from the experience of others who have conducted similar exercises before. We will encourage IA's to both share their own evaluation methods and materials, and also to use the methods shared by others. This will encourage reuse, reducing the reduplication of effort.

Creation of Benchmarks: We hope that over a period of time, this project will lead to the creation of benchmarks for task success and satisfaction with various types of information-rich websites.