Poster

An Evaluation Scheme for Hierarchical Information Browsing Structures

Megan Richardson

There is no widely accepted means of evaluating category systems for information search and browsing. The key to evaluation is provision of specific feedback. This presentation outlines an evaluation scheme and an evaluation method that combine the best features of intrinsic and extrinsic, automatic and empirical evaluations. The scheme delineates features broadly classified under comprehensiveness, coherence, and correctness. The study method given requires minimal resources and effort, is easily done remotely, and is easily modified. The method evaluates the category system through an interactive survey distributed among subject domain experts. Inferences are then drawn about the method of generating the system. This study technique is but one possible application of the scheme. The overall approach finds the over- and under-sensitivities of the method of generating the system. A case study has demonstrated the usefulness of the method, and inter-rater reliability in the data suggests that the evaluation scheme is meaningful.