HCI 6 (Evaluation: Collecting Data)

Assess the extent and accessibility of the systems functionality (i.e. the design of the system should enable users to perform their intended tasks more easily)

Assess users experience of the interaction and its impact upon the user (e.g. how easy the system is to learn, its usability and the users satisfaction with it; is the experience enjoyable)

Identifying any specific problems with the system (e.g. unexpected results, confusion amongst users, bugs)

• Who - is giving the feedback (expert, novice etc)
• When - are you getting the feedback with regard to the development cycle
• How - is the evaluation been arrived at? what approach is being used (walkthrough, comparison guidelines, realistic context?)
• Which - method for data gathering is used (Interview, observation etc.)
• What - is being used to measure (qualitative/quantitative, alternatives?, recommendations)
• Resources - what time, effort and money do you have

• - interviews
• - focus groups
• - questionnaires
• - surveys
• - Observation in the field
• - indirect observation
• - experiment

• -pre-design
• -early prototype
• -pilot
• -product

evaluation is iterative and adaptive. repeated/different to suit needs of stages

• Analytical evluation (by experts)
• - cognitive walkthrough
• - heuristic evaluation
• - model-based evaluation

Usability testing (controlled conditions) - hypothesis test

Field studies (natural conditions)

Ethnomethodology

• 1. set your goals
• 2. decide on participants
• 3. decide your approach
• 4. relationships with participants
• 5. trangulate/mix methods
• 6. use pilot studies

cognitive walkthrough involves stepping through an interaction sequence from the user's perspective

(look up slides, lec8_19)