Answer:
Introduction - Evaluation plays an important role in users' information searching and retrieving processes. While previous research mainly focuses on applied criteria, less research is on other dimensions of evaluation. This study explores the dimensions of evaluation activities including criteria applied, elements examined, an activity engaged in, and time spent in evaluating individual documents.
Method - Conducting their own tasks, thirty-one participants with different demographic backgrounds, participated in the study. Multiple data collection methods were used: pre-questionnaires, interaction diaries, think-aloud protocols, transaction logs, and post-questionnaires.
Analysis - Types of evaluation criteria, elements, evaluation activities, and their associated pre- and post-activities were analyzed by open-coding. Descriptive analysis was applied to analyze the most and least time spent on individual documents with associated factors.
Results - The findings of this study reveal that evaluation activities are complicated and dynamic. Eighteen types of evaluation criteria, seven types of evaluation elements, six types of evaluation activities, and their associated pre- and post-activities were identified. In addition, factors affecting the time spent in evaluating individual documents were analyzed.
Conclusions - The authors offer suggestions for information retrieval system design to support effective evaluation.
