Can AI be funny? A brand new study by psychology researchers from the University of Southern California tested ChatGPT’s comedic ability with surprising results.
ChatGPT can write code, coherent content, and even poetry. But can it write content that captures the hard-to-quantify concept of humor?
It’s easy to judge if text is grammatically correct or if code works, but humor is way more subjective. The researchers ran two experiments to see if people found ChatGPT’s content funny.
For the primary experiment, they hired 123 US-based laypeople (i.e. not skilled comedians) on Amazon Mechanical Turk. The participants were asked to finish 3 humor production tasks:
- Generate one recent, humorous phrase for every of the next three acronyms: “S.T.D.”, “C.L.A.P.”, and “C.O.W.”
- Fill within the blanks. Create one humorous answer for every of the three items: “A lesser talked about room within the White House: ___”, “A remarkable achievement you almost certainly wouldn’t list in your resume: ____”, “Worst first date activity: ____.”
- Roast joke: Create a humorous, conversational response to fictional scenarios. E.g. a friend asks for feedback on their singing. “To be honest, listening to that was like ____”)
The researchers gave ChatGPT 3.5 the identical tasks and directions because the participants and generated 180 comedic responses.
They then hired 200 US-based CloudResearch approved MTurk staff to rate the humor of a random subset consisting of an equal amount of human and AI-generated responses.
The participants needed to rate 54 responses based on a 7-point Likert scale (0 = Not Funny at All; 6 = Very Funny).
Funny results
Comparison of funniness rankings of AI-generated and human comedic responses. Source: Gorenz, D., & Schwarz, N. (2024). How funny is ChatGPT? A comparison of human- and A.I.-produced jokes. PLOS ONE, 19(7), e0305364. https://doi.org/10.1371/journal.pone.0305364
After evaluating the responses, the researchers said, “When we compare the funniness of the jokes generated by each of our human participants within the humor production task with the funniness of the jokes produced by ChatGPT 3.5, ChatGPT outperformed the vast majority of our human humor producers on each task.”
ChatGPT was competing with laypeople on this test. How wouldn’t it do when it got here up against professionals?
The researchers compared ChatGPT 3.5’s ability to provide humorous, satirical news headlines to that of skilled comedy writers at The Onion.
They recruited 217 students from the University of Southern California and asked each of them to rate the funniness of 10 satirical headlines (5 AI, 5 human).
The results showed that ChatGPT’s satirical headlines were rated on average as funny as those written by The Onion’s skilled comedic writers.
Of the 4 top-rated headlines, two were generated by skilled writers and two by ChatGPT.
ChatGPT wrote the highest-rated entry (“Local Man Discovers New Emotion, Still Can’t Describe It Properly”) and the fourth-highest-rated entry (“Man Achieves Personal Best in Avoiding Eye Contact With Neighbors During Awkward Elevator Ride”).
Those are pretty funny and lift some interesting questions and conclusions.
If you enjoy telling jokes but you’re not a professional, having ChatGPT write your material will get you more laughs than writing your personal jokes.
If you’re knowledgeable comedian, ChatGPT’s output is probably going on par with what you currently write. Will an AI-generated comedian like George Carlin be coming on your job?
Probably not yet. Writing a joke is one thing. Getting the timing, delivery, and intonation right requires skills that few people have, never mind an AI.
AI doesn’t feel emotion so how does it know what’s funny? The researchers got here to a disconcerting conclusion:
“Our studies suggest that the subjective experience of humor is probably not required for the production of excellent humor–merely knowing the patterns that make up comedy may suffice.”
So ChatGPT can write funny jokes but still needs a human to deliver them. At least, for now.
