Human-Computer Interaction Best Paper Award

Paper Abstract
Conversational AI systems often elicit anthropomorphic responses. However, much HCI research still relies on self-report measures that primarily capture reflective beliefs and may overlook cue-driven, interaction-level framing effects. We address this measurement gap by introducing and validating the Anthropomorphic Language Index (ALI), a transparent LIWC-22 composite designed to quantify implicit linguistic anthropomorphism in users’ prompts. Grounded in the AugCog-LLM framework, ALI operationalizes anthropomorphic framing as the degree to which prompting language treats the system as a social and intentional agent rather than a neutral instrumental tool. ALI is constructed from five interpretable dictionary-based LIWC-22 indicators: second-person pronouns (you), affect, social behavior, prosocial language, and politeness. We validate ALI using a within-subject pretest comparison between two interaction contexts with ChatGPT-4o: socio-affective chitchat and goal-directed information search, based on aggregated prompt logs with sufficient text length in both tasks (WC>=500; N = 52). Pooled standardized ALI robustly distinguished contexts, with substantially higher ALI in chitchat than information search (mean paired difference = 0.53, 95% CI [0.40, 0.66]; t(51) = 8.28; dz = 1.15), converging across parametric, nonparametric, and bootstrap inference. Word count was negatively associated with ALI within chitchat, but the task effect remained large and significant after WC control via within-subject difference score regression. We discuss how ALI complements self-report and model-based approaches (e.g., AnthroScore) by prioritizing auditability and psychological interpretability, and outline future work on convergent validation, broader construct coverage, and reliability modeling at the turn level.

The full paper is available through SpringerLink, provided that you have proper access rights.