I gave an 1-hour presentation on “Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies” in the first NLP Reading Group. This paper was published in 2023 at ICML. It’s well-written and doesn’t require much CS background to understand. Considering that many audience today had non-CS backgrounds (e.g. Psychology, Linguistis, Medicine), this paper was relevant to share as the authors used LLMs to simulate multiple classic studies in diverse subjects. In case interested, here are my annotated paper and presentation slides.

Due to technical issues, we started the event 10 minutes late. There were around 16 attendees in person and online, and I was grateful to their attendence and support. One attendee, Dr. Lisa Levinson, pointed out that the authors didn’t simulate the Garden-Path Sentence study properly as they oversimplied the experimental setup. According to her research as a linguist, LLMs are not able to faithfully simulate humans’ response in garden-path sentences. I appreciate such viewpoints from experts in other domains: NLP researchers will thrive by hearing thoughts in other fields. Otherwise, we may not even notice our issues. In the original paper, authors didn’t seem to consult a linguist when simulating a linguistics study, which is something future researchers should avoid…

Overall, the first reading group was a good start: I presented with confidence and had positive interaction with audience. Sadly, we didn’t order food today for in-person attendees, but I tried my best to make their visits worth more than food :D

Look forward to future events and seeing more people participating!