Modality
on-site
Room: TBA
Target Audience
This course caters to attendees who are motivated to learn about, and use, empirical research methods in HCI research. Specifically, it is for those in academia or industry who evaluate interaction techniques using quantitative methods, or those who perform usability tests and, in particular, user studies following an experimental methodology.
Requirements for participants
Course participants should bring their own laptop or tablet
Abstract
Most attendees at HCI conferences will agree that an experiment (user study) is the hallmark of good research in human-computer interaction. But what constitutes an experiment? And how does one go from an experiment to an HCI research paper? This course will teach how to pose testable research questions, how to make and measure observations, and how to design and conduct an HCI experiment. Specifically, attendees will participate in a real experiment to gain experience as both an investigator and as a participant. The second session covers (and simplifies!) the statistical tests typically used to analyze data. Most notably, attendees will learn how to organize experiment results and write a publication-ready HCI research paper.
Benefits for attendees
Participants will learn the basics of research that is empirical and experimental. They will learn how to design and conduct a user study, collect data, analyze the data, and write it all up in a research paper that can be submitted for publication to a conference such as Human-Computer Interaction International.
Course Content
This course presents selected topics from the first presenter's book titled Human-Computer Interaction: An Empirical Research Perspective. Topics are drawn from Chapter 4 (Scientific Foundations), Chapter 5 (Designing HCI Experiments), Chapter 6 (Hypothesis Testing), and Chapter 8 (Writing and Publishing a Research Paper).
Session 1 topics:
- What is research, what is empirical research, and what is the scientific method?
- Formulating "testable" research questions
- How to design an experiment (broadly speaking) to answer research questions
- Parts of an experiment (independent variables, dependent variables, counterbalancing, ethics approval, etc.)
- Group participation in a real experiment
Session 2 topics:
- Results and discussion of the experiment from session 1 (this affords a strong opportunity to revisit and expand on the elements of empirical research)
- Experiment design issues (within-subjects vs. between-subjects factors, internal vs. external validity, counterbalancing test conditions, etc.)
- Data analyses (main effects and interaction effects, requirements to establish cause and effect relationships, etc.)
- How to organize and write a successful HCI paper (including suggestions for style and approach, as per typical HCI conference submissions)
Hands-on part:
Early in session 1, participants are divided into groups of two and participate in an experiment. A hand-out is distributed for the in-class experiment.
Following brief instructions, the in-class experiment proceeds. During the experiment, attendees take turns acting as "participant" and as "investigator." The participant does an experimental task – entering a text phrase five times with a non-marking stylus on the image of a soft keyboard – while the investigator measures the time to enter each phrase. This is done twice, once for keyboard layout "A" and once for keyboard layout "B". The data are entered into a log sheet. When finished, the participant and investigator switch roles with the process is repeated. This time the order of using the keyboard layouts is reversed, "B" first, then "A". This is an example of counterbalancing, as explained during the course.
As well as performance data, demographic information is entered on the log sheet. The in-class experiment takes about 20 minutes. Two of the course instructors collect the hand-out sheets, leave the room, and transcribe the data from the handout sheets into a boilerplate spreadsheet. This is done as the course continues. Transcribing the data takes about 20-30 minutes with two people; that is, one reads out the data while the other inputs the data. This procedure has proved successful in previous offerings of this course.
During session 2, the course continues but now uses the methodology and results of the in-class user study to reinforce topics in the course. It is extremely useful from a pedagogical perspective that the results discussed are from an experiment in which the course attendees have just participated. Results of an analysis of variance are also presented.
Bio Sketches of Course instructors
Scott MacKenzie's research is in human-computer interaction with an emphasis on human performance, experimental methods and evaluation, interaction devices and techniques, etc. He has more than 200 peer-reviewed publications in the field of Human-Computer Interaction.
Dr Maria Francesca Roig-Maimó is an associate professor of the Department of Mathematics and Computer Science at the University of the Balearic Islands (Spain). She is member of the "Computer Graphics and Vision and AI Group" (UGIVIA) with research interests that cover several topics such as Human-Computer Interaction, mainly focused on mobile devices and performance evaluation.
Dr Ramon Mas-Sansó is an associate professor of the Department of Mathematics and Computer Science at the University of the Balearic Islands (Spain). He is member of the "Computer Graphics and Vision and AI Group" (UGIVIA) with research interests in Computer Graphics and Human-Computer Interaction, mainly focused on mobile devices and performance evaluation. He is currently the Director of the Master’s Degree in Intelligent Systems and Secretary of the Doctoral School of the Balearic Islands University.