Publications

2025

1. Examining Spanish Counseling with MIDAS: a Motivational Interviewing Dataset in Spanish

   Aylin Ece Gunal*, Bowen Yi*, John D. Piette, and 2 more authors

   In NAACL, 2025

   * = Equal Contribution

   Abs PDF

   Cultural and language factors significantly impact counseling, yet NLP studies have not explored the applicability of English counseling findings to other languages, which is crucial for multicultural countries like the US. Seeking to contribute to this line of work, this paper introduces MIDAS, a new Motivational Interviewing Dataset in Spanish created from public video sources and expert annotations. It includes annotations for counseling reflections and questions. We use it to explore language-based differences in counselor behavior in English and Spanish MI, and develop classifiers in monolingual and multilingual contexts to demonstrate the dataset’s value for behavioral coding tasks.

2. Causally Modeling the Linguistic and Social Factors that Predict Email ResponseCausally Modeling the Linguistic and Social Factors that Predict Email Response

   Yinuo Xu*,  ..., Bowen Yi*, and 2 more authors

   In NAACL, 2025

   * = Equal Contribution

   Abs PDF

   Email is a vital conduit for human communication across businesses, organizations, and broader societal contexts. In this study, we aim to model the intents, expectations, and responsiveness in email exchanges. To this end, we create SIZZLER, a new dataset containing 1800 emails annotated with nuanced types of intents and expectations. We benchmark models ranging from feature-based logistic regression to zero-shot prompting of large language models. Leveraging the predictive model for intent, expectations, and 14 other categories of pragmatic features, we analyze 11.3M emails from GMANE to study the conversational dynamics in email exchanges. Through our causal analysis, we find that the email response rates are influenced by social status, argumentation, and in certain limited contexts, the strength of social connection. Our code and data are available upon acceptance.

3. Unveiling Behavioral Differences in Bilingual Information Operations: A Network-Based Approach

   Bowen Yi

   2025

   In Submission to IC2S2, Preprint on arXiv

   Abs

   Twitter has become a pivotal platform for conducting information operations (IOs), particularly during high-stakes political events. In this study, we analyze over a million tweets about the 2024 U.S. presidential election to explore an under-studied area: the behavioral differences of IO drivers from English- and Spanish-speaking communities. Using similarity graphs constructed from behavioral patterns, we identify IO drivers in both languages and evaluate the clustering quality of these graphs in an unsupervised setting. Our analysis demonstrates how different network dismantling strategies, such as node pruning and edge filtering, can impact clustering quality and the identification of coordinated IO drivers. We also reveal significant differences in the topics and political indicators between English and Spanish IO drivers. Additionally, we investigate bilingual users who post in both languages, systematically uncovering their distinct roles and behaviors compared to monolingual users. These findings underscore the importance of robust, culturally and linguistically adaptable IO detection methods to mitigate the risks of influence campaigns on social media. Our code and data are available on GitHub: this https URL.

2024

1. The Generation Gap: Exploring Age Bias in the Value Systems of Large Language Models

   Siyang Liu, Trisha Maturi, Bowen Yi, and 2 more authors

   In EMNLP 2024, Nov 2024

   Abs

   We explore the alignment of values in Large Language Models (LLMs) with specific age groups, leveraging data from the World Value Survey across thirteen categories. Through a diverse set of prompts tailored to ensure response robustness, we find a general inclination of LLM values towards younger demographics, especially when compared to the US population. Although a general inclination can be observed, we also found that this inclination toward younger groups can be different across different value categories. Additionally, we explore the impact of incorporating age identity information in prompts and observe challenges in mitigating value discrepancies with different age cohorts. Our findings highlight the age bias in LLMs and provide insights for future work. Materials for our analysis will be available via \urlhttps://github.com/anonymous

2. Uncovering the Impact of Intervention Messages on Diverse Population Groups

   Bowen Yi, Rada Mihalcea, Fang Yu, and 4 more authors

   Nov 2024

   Working Paper

   Abs

   Medical interns usually face mental health challenges due to the demanding nature of their working environments. In this context, intervention messages are delivered to their mobile devices to support them in maintaining healthy behavior. This study investigates the effectiveness of various intervention messages by analyzing their conditional average causal effects on health outcomes such as steps, mood, and sleep. We identify linguistic features, including LIWC, and factors encoded by medical experts, such as cognitive strategies, that contribute to the efficacy of these interventions. To handle the high dimensionality of our treatment variables, we develop methods that balance the tradeoff between overlapping and positivity assumptions of causal inference. In addition, we introduce a decision tree-based approach to identify subgroups that respond differently to interventions. It shows higher stability than previous approaches against small data variations. Our research, validated on a 4-year dataset involving over 6,000 medical interns from 250 U.S. medical programs, offers valuable insights for designing more effective, subgroup-aware intervention strategies in healthcare.

3. Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue

   Johnathan Ivey*,  ..., Bowen Yi*, and 2 more authors

   Nov 2024

   In Submission to ACL 2025. * denotes equal contribution with randomized order.

   Abs PDF

   Studying and building datasets for dialogue tasks is both expensive and time-consuming due to the need to recruit, train, and collect data from study participants. In response, much recent work has sought to use large language models (LLMs) to simulate both human-human and human-LLM interactions, as they have been shown to generate convincingly human-like text in many settings. However, to what extent do LLM-based simulations \textitactually reflect human dialogues? In this work, we answer this question by generating a large-scale dataset of 100,000 paired LLM-LLM and human-LLM dialogues from the WildChat dataset and quantifying how well the LLM simulations align with their human counterparts. Overall, we find relatively low alignment between simulations and human interactions, demonstrating a systematic divergence along the multiple textual properties, including style and content. Further, in comparisons of English, Chinese, and Russian dialogues, we find that models perform similarly. Our results suggest that LLMs generally perform better when the human themself writes in a way that is more similar to the LLM’s own style.