Chatbot or Humanaut? How the Source of
Advice Impacts Behavior in One-shot Social
Dilemmas

doi:10.21203/rs.3.pex-2653/v1

Method Article

Chatbot or Humanaut? How the Source of
Advice Impacts Behavior in One-shot Social
Dilemmas

https://doi.org/10.21203/rs.3.pex-2653/v1

This work is licensed under a CC BY 4.0 License

This protocol has been posted on Protocol Exchange, an open repository of community-contributed protocols sponsored by Nature Portfolio. These protocols are posted directly on the Protocol Exchange by authors and are made freely available to the scientific community for use and comment.

Version 1

posted

You are reading this latest protocol version

This paper explores how the source of advice (human vs. AI chatbot) impacts behavior in three classic social dilemmas. Utilizing a novel experimental design, we show that advice enhances welfare, but when it comes to prosociality, the source matters. Players exhibited a preference for human advice over that from generative AI (genAI) and were more willing to pay for it. Prosocial behavior was more prevalent when players received human advice compared to genAI, increasing the probability of the cooperative stag hunt strategy and significantly driving contributions in public goods and dictator games. Potential explanations include genAI's limitations in being truly empathetic and the possibility of moral wiggle room. Leveraging natural language processing (NLP) advances through large language models (LLMs) topic modeling, we demonstrate that the human and genAI advice differ significantly. Human advice was more objective, specific, intuitive, and norm-oriented, while AI advice tended to offer reasoning and targeted concepts of risk. We conclude that the adoption of genAI technologies should be balanced with an investment in human connectivity and organizational input with a goal of the constrained agency. Preregistered at Aspredicted.org #128933

advice

bargaining

text analysis

chatbots

genAI

LLMs

Methods
Our study design and hypotheses were pre-registered (Aspredicted.com #128933) and IRB-
approved, #23-09. The primary investigation involves players’ preferences over the source of
advice and how advice impacts prosocial behavior. To operationalize our inquiry, we conducted
a 1x4 between-subjects experiment. Players interacted asynchronously (strategy method) in a
one-shot 1) stag hunt (to capture coordination/cooperation), 2) public goods game/VCM with a
contribution return rate α = 1.5 (to measure other-regarding behavior in the face of incentives
to free ride, hereafter PGG), and 3) a dictator game (a common measure of altruism). We
presented these interactions in random order. Instructions framed the games as tasks but oth-
erwise employed neutral language (details and payoff structures depicted in Figure 4, panel a)
and omitted words like ‘cooperate’ or ‘free ride.’
Data were collected from workers on the Prolific.co platform. One upside to using Prolific
is a pre-populated demographic profile for each subject, allowing us to minimize the number of
questions in the instrument. Subjects earned a $1 participation fee and could earn bonuses up
to $2.50 in virtual currency (tokens at the rate of US $1 = 100 tokens). Each game involved
stakes of roughly 100 tokens, and the game paying final bonus earnings was randomly deter-
mined.
Subjects were randomly assigned to one of the following experimental conditions with com-
prehension checks (see Fig. 4, panel b for design flow).
• Control (c1): Players played each interaction unadvised using the strategy method and
composed around three sentences of advice to future players for each game.
• Advice choice treatment (t1): Players played each interaction advised. Before committing
to actions in each game, they chose whether to receive a randomly selected piece of
advice from a human or a genAI chatbot.
• Advice choice treatment with payments (t2): Players played each interaction advised.
Before committing to actions in each game, they chose whether to receive between zero
and three randomly selected pieces of advice from a human or a genAI chatbot. They
paid a small fee following a price menu to receive the advice.
• Random advice treatment (t3): Players played each interaction advised. Before com-
mitting to actions in each game, they were randomly allocated a piece of advice from a
human or a genAI chatbot, but the source was not revealed to them. We incentivized
them to guess the source.
Control subjects composed advice to be forwarded to treatment subjects. We used the
Javascript editing features in Qualtrics to insert a corpus of human-composed advice to the
treatment questions. Then, we set the balance counts for each condition to preload human-
created content and ensure a flow of advice as requested. Thus, the instrument allowed ran-
domization, compiled a sizable corpus of advice, and responded to subjects’ selections in real-
time. While composing advice in the control condition required a certain amount of (compen-
sated) effort, there was no incentive for strategic misguidance because the advice was not

One shot Social dilemmas: stag hunt (to capture coordination/cooperation), public goods
game/VCM with a contribution return rate α = 1.5 (prosociality in the face of incentives to free
ride), and dictator (altruism) Players play all games (random order).

meant for the player’s counterpart [contrasting with advice in Sobel, 2013].
A corpus of AI-generated advice regarding the three games was preloaded into our instru-
ment. Players assigned to the advice treatments were advised with genAI content from Chat-
GPT4 and Google Bard – LLMs we deemed the most capable after a comparison of several
models on sdk.vercel.ai. While we assumed players had some familiarity with genAI chatbots,
we provided the following definition (generated by ChatGPT4): “An AI chatbot is a computer
program that uses artificial intelligence (AI) and natural language processing (NLP) to commu-
nicate with humans through chat interfaces, such as messaging apps, websites, or mobile apps.
Chatbots (such as ChatGPT4 or Google Bard) can simulate human-like conversations and help
users with various tasks, such as answering questions, providing information, or executing sim-
ple commands.”
After playing the games and executing advice choices, all players provided their beliefs
about existing injunctive norms through an incentivized coordination game [following Krupka
and Weber, 2013], commented on their perception of social norms, and decided over risky lot-
teries [via the Eckel and Grossman, 2002, method].
Sample A pre-collection power test suggested a minimum sample size of 140 observations
for the advice-giving control and 230 per treatment. We ultimately included 147 subjects in the
control and 247 in each of the three treatments. A subject was 32.26 years on average, and the
sample was 50% female (both uniform across treatments); these demographics are consistent
with most studies using samples of Internet-savvy subjects. Because the study involves atti-
tudes toward technology, we confirmed no meaningful age differences across treatments using
Kolmogorov-Smirnov tests. The ethnic background of the sample was 67.65% White, 13.55%
Black, 5.92% Asian, 4.44% Hispanic or Other, and 8.31% Mixed. Thirty-three percent of the
subjects were from the US, and the mean of 712 approvals suggests they were fairly experi-
enced on Prolific. The average completion time was about nine minutes. Players’ final bonus
payoffs were determined by randomly choosing one of the three games, lottery results, and
incentivized coordination over norms. The mean compensation rate amounted to roughly US
$9.25/hour, and the study paid out $3105.
Advice The authors debated several potential approaches to compiling the advice corpora
before settling on the following methods. We note that “prompt engineering” is as much an art
as a science; a slight variation in a prompt can lead to drastically different results. While we
were mindful to collect meaningful and authentic advice from both genAI and human subjects,
we make no claims about the “correctness” of either form nor their consistency with theory (in
fact, advice was often obtuse). GenAI advice results from an LLM being trained on game theory
and behavioral economics articles, among other inputs. We did not pre-screen or evaluate the
advice for factual or theoretical correctness, nor did we suggest that we endorsed the advice
they would receive. Adulterating or cherry-picking advice to send players in the treatments more
closely approached deception than providing raw advice, warts and all. Following Schotter
[2023, p. 1], advice does not have to derive from experts to be meaningful. Thus, we are
confident that the advice corpora are suitable for capturing the effects we explore. Previous
studies propose that there is value in eliciting advice. Nyarko et al. [2006] set up a market for
advice and allow subjects to bid for advice based on the expertise of the source; in their setup,
advice is binary, whereas in ours, advice is free-form text.
GenAI Advice Using two prompt styles, we elicited advice from ChatGPT4 and Google
Bard on three games. We used neutral language and entered the subject instructions verbatim,
avoiding game names. For practical genAI advice, specificity was required in prompts, such as
equating the ‘stag’ strategy with ‘Blue’ in our context. An alternative approach would have been
allowing subjects to create their prompts. However, we recognized that, despite understand-
ing the games, they might lack the genAI experience to elicit useful responses, and individual
prompts would render between-subject comparisons invalid.
The following text depicts our prompts to ChatGPT4 and Google Bard. The text is essentially
the same instructions we gave human subjects; prompt engineering for genAI requires some
clarifying statements and examples meant to exclude meaningless output).
• Mega Prompt 1: Stag Hunt
Suppose I will randomly match you with another participant in this task. You and your
counterpart will be asked the same question. You will play once only and have no com-
munication. You each have to decide between one of two strategies: Blue or Yellow. You
will make this choice once and must choose without observing your counterpart’s choice.
Anytime either of you chooses to Yellow, that person is guaranteed a payoff of 50 tokens
regardless of what the other chooses. If each of you chooses Blue, each of you will re-
ceive a bonus payoff of 100 tokens. If either player chooses Blue when the other chooses
Yellow, they receive no bonus payoff and zero tokens. The player choosing Yellow gets
the guaranteed payoff of 50 tokens. In no fewer than three sentences and in your own
words, please provide some thoughtful advice to a future player about how best to play
in this interaction.
• Mega Prompt 1: PGG
Now, suppose I will randomly match you with another participant in this task. You and
your counterpart will be asked the same question. Each of you will have 100 tokens. You
each have to decide how many tokens to transfer to a public fund. All transfers will grow
by 50%. Each player gets an equal share of the public fund. You play once, and there is
no communication. Your final payoff will be the sum of 1) what you did not transfer plus 2)
your share of the public fund. In no fewer than three sentences and in your own words,
please provide some thoughtful advice to a future player about how best to play in the
same interaction that you just had.
• Mega Prompt 1: Dictator
We will randomly match you with another participant in this task. You and your counter-
part will be asked the same question. There is an endowment of 100 tokens among you.
You play once, and there is no communication. There will be two roles: Role A and Role
B. You will be randomly assigned one of the two roles. Role A decides how many tokens
from the endowment to allocate to Role B. Role B has no action and must accept Role
A’s allocation. Your final payoff will depend on the role assigned to you: If you are in Role
A, your payoff is 100 tokens minus what you allocate to Role B, If you are in Role B, your
payoff is what Role A allocates to you. In no fewer than three sentences and in your own
words, please provide some thoughtful advice to a future player about how best to play
in the same interaction that you just had. You will make this choice once in Role A and
must choose without observing your counterpart’s choice.
The resulting corpus contains advice for each interaction, with prompts crafted from the
instructions for usefulness and accuracy. Without specifying the one-shot nature of the games,
genAI occasionally gave irrelevant advice on repeated interactions. We didn’t alter AI-generated
content; we only formatted it into bullet points.
Human Advice In the control condition, subjects composed advice for future players imme-
diately after committing to action in each game but before recording their beliefs about existing
norms (to avoid priming). The following text depicts our prompt to the subjects. Please be thoughtful, complete, and mind grammar in this part of the task. We will review all
responses before compensation. AI-generated responses are not allowed.
In no fewer than three sentences, more than 40 words. And in your own words, please provide
some thoughtful advice to future player about how best to play in the same interaction that you
just had. Note, you’re not writing to your counterpart. "
Subjects composed whatever advice they saw fit; we pre-screened but did not edit or spell-
check those responses before providing them to players in the treatment. To ensure data quality
and protect participant anonymity, it may be necessary to exclude messages from a research
study [see Brandts et al., 2019]. We removed nonsensical responses and identified potentially
AI-generated content with GPTZero.com. We acknowledge that some players may have be-
lieved they were communicating with their counterparts. The resulting human corpus included
eighty-nine pieces of advice for each interaction.
One significant upside of the method is a corpus large enough to conduct the text analysis
key to understanding why sources of advice may have value to players. We conducted a deep
dive into natural language processing to achieve this goal.
NLP and Text Analysis After an extensive literature review, we consider Gentzkow et al.
[2019] the current standard reference for using text as economic data. Many papers have
adopted text-based data to explore economic or social sciences problems [for example, Ru-
binstein, 1996, Eckel and Wilson, 2004a, Charness and Dufwenberg, 2006, Houser and Xiao,
2011, Chen and Chen, 2011, Rand et al., 2012, Lucas et al., 2015, Ding and Schotter, 2017,
Saltzman and Yung, 2018, Tingley et al., 2018, Babin and Chauhan, 2023, Kalabikhina et al.,
2023]. Historically, such text analysis in studies like ours has been limited to coding chat-
logs/classification techniques, word frequency analysis, and dictionary-based ‘bag of words’
approaches. The rapid advance of machine learning algorithms and LLMs has rendered that
paper somewhat dated.
Our analysis utilized the most sophisticated methods available to us at the time of writing.
We recognize that the tools we reference in the following paragraphs are likely to be unfamiliar
to most outside of the computer and information sciences; thus, we provide a brief description
of each method accompanying our results. Our corpora were cleaned and preprocessed using
the following Natural Language Processing (NLP) methods:
• Tokenizing: Splitting text into smaller units (tokens), which can be words, phrases, or
symbols, allowing algorithms to work with individual components of the text. For example,
tokenizing the sentence "I enjoy analyzing text" might result in the tokens "I", "enjoy",
"analyzing", and "text".
• Stemming: This process involves reducing words to their base or root form. For example,
the stemming process would convert words like "analyzing" and "analyze" into their stem
analyz. This step makes it easier for algorithms to process text and humans to interpret
results.
• Stopwords: Some terms are commonly removed from the text before processing because
they carry minimal meaningful information for analysis. Examples include "and", "the",
and "is." Removing stopwords helps focus on the more significant words that carry the
core meaning of the text; thus, the process involves some understanding of the corpora
and trial and error. We removed words that are portions of the question used to elicit the
response (e.g., "advice" and "interaction"). On a few rare occasions, we removed what
we deemed as game-specific stopwords.

Vectorizing: This process involves transforming text data into a numerical vector format
suitable for algorithmic analysis and pattern recognition.
Sentiment Analysis TextBlob sentiment analysis, as described by Loria [2018] and im-
plemented in Python’s natural language toolkit (NLTK), was utilized for its simplicity and acces-
sibility in processing and analyzing textual data to determine polarity (i.e., negative, neutral, or
positive sentiment, on a -1 to 1 scale) and subjectivity (objective or subjective perspective, on a
0 to 1 scale). The terms are non-stemmed, and we applied no stopwords.
Term Weighting (TF-IDF) TF-IDF term weighting demonstrates significant differences
in the relative informational importance of terms in advice by source – the salience of a term.
We note that term/bigram frequency and ‘bag of words’ count models, [such as LIWC Tausczik
and Pennebaker, 2010], have been largely replaced with the TF-IDF statistic – widely used in
text mining, search engine scoring, and data preprocessing for machine learning models. TF-
IDF is significantly more beneficial for document classification and clustering tasks. Still, it has
limitations, such as being unable to capture context and semantics. The corpora terms are
stemmed, and we applied custom stopwords.
Topic Modeling One of our goals is to expose the underlying themes and content of each
piece of advice. Every document (a piece of advice) is a mixture of topics, as is a corpus. Our
method, topic modeling, extracts conceptual topics from unstructured text data. At the time of
Gentzkow et al. [2019], the most sophisticated topic modeling technique was Latent Dirichlet Al-
location (LDA), a Bayesian framework to reveal underlying topical patterns in text data. We refer
the reader to Jelodar et al. [2019] for a survey of LDA applications and Egger and Yu [2022] for
a cost-benefit comparison of contemporary topic modeling techniques. We also employed more
traditional methods, such as LDA, bigram frequency, and Kamada et al. [1989] co-occurrence
network analysis for robustness. While LDA remains a fundamental technique, it has several
limitations, such as assuming the independence of terms and being agnostic to textual context.
Recently, sophisticated transformer-based models like Google’s BERT have been developed,
which capture semantic relationships between s, words using the context-aware c-TF-IDF ap-
proach. BERTopic [Grootendorst, 2022] addresses the downsides of a model like LDA by ex-
ploiting a pre-trained large language model framework and package modularity to generate
topics to capture the intent and context of advice – what it talks about. BERTopic differs from
LDA by using the context-focused c-TF-IDF algorithm to compare term importance within clus-
ters and generate the optimal number of topic representations. Instead of treating a single term
as a cluster’s centroid, it regards all documents within a cluster as unique. Topic labels are de-
rived from keywords with the highest c-TF-IDF scores. These interpretations remain subjective
and depend on the specific data context. The method classifies tokenized and vectorized parts
of text into topic bins. The corpora terms are stemmed, and we applied game-specific custom
stopwords.

References
Eyal Aharoni, Sharlene Fernandes, Daniel J Brady, Caelan Alexander, Kara Queen, Javier
Rando, and Victor Crespo. Attributions toward artificial agents in a modified moral turing
test. Scientific Reports, 14(8458), 2024.
James Andreoni and Justin M Rao. The power of asking: How communication affects selfish-
ness, empathy, and altruism. Journal of Public Economics, 95(7-8):513–520, 2011.
J Jobu Babin and Haritima S Chauhan. Initiating free-flow communication in trust games. Fron-
tiers in Behavioral Economics, 2:1120448, 2023.
Daniel Balliet. Communication and cooperation in social dilemmas: A meta-analytic review.
Journal of Conflict Resolution, 54(1):39–57, 2010.
Avner Ben-Ner, Louis Putterman, and Ting Ren. Lavish returns on cheap talk: Two-way com-
munication in trust games. The Journal of Socio-Economics, 40(1):1–13, 2011.
Yochanan E Bigman and Kurt Gray. People are averse to machines making moral decisions.
Cognition, 181:21–34, 2018.
Samuel Bowles. The moral economy: Why good incentives are no substitute for good citizens.
Yale University Press, 2016.
Jordi Brandts, David J Cooper, and Christina Rott. 21. communication in laboratory experiments.
Handbook of research methods and applications in experimental economics, 401, 2019.
Joy Buchanan, Stephen Hill, and Olga Shapoval. Chatgpt hallucinates non-existent citations:
Evidence from economics. The American Economist, 69(1):80–87, 2024.
J Elliott Casal and Matthew Kessler. Can linguists distinguish between chatgpt/ai and human
writing?: A study of research ethics and academic publishing. Research Methods in Applied
Linguistics, 2(3):100068, 2023.
Gary Charness and Martin Dufwenberg. Promises and partnership. Econometrica, 74(6):1579–
1601, 2006.
Ananish Chaudhuri. Sustaining cooperation in laboratory public goods experiments: a selective
survey of the literature. Experimental economics, 14:47–83, 2011.
Ananish Chaudhuri, Sara Graziano, and Pushkar Maitra. Social learning and norms in a public
goods experiment with inter-generational advice. The Review of Economic Studies, 73(2):
357–380, 2006.
Roy Chen and Yan Chen. The potential of social identity for equilibrium selection. American
Economic Review, 101(6):2562–89, 2011.
David J Cooper and Roberto A Weber. Recent advances in experimental coordination games.
Handbook of experimental game theory, pages 149–183, 2020.
Russell Cooper, Douglas V DeJong, Robert Forsythe, and Thomas W Ross. Communication in
coordination games. The Quarterly Journal of Economics, 107(2):739–771, 1992.
Jason Dana, Roberto A Weber, and Jason Xi Kuang. Exploiting moral wiggle room: experiments
demonstrating an illusory preference for fairness. Economic Theory, 33:67–80, 2007.
Benedict GC Dellaert, Suzanne B Shu, Theo A Arentze, Tom Baker, Kristin Diehl, Bas Donkers,
Nathanael J Fast, Gerald Häubl, Heidi Johnson, Uma R Karmarkar, et al. Consumer deci-
sions with artificially intelligent voice assistants. Marketing Letters, 31:335–347, 2020.
Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. Algorithm aversion: people erro-
neously avoid algorithms after seeing them err. Journal of Experimental Psychology: Gen-
eral, 144(1):114, 2015.
Tingting Ding and Andrew Schotter. Matching and chatting: An experimental study of the im-
pact of network communication on school-matching mechanisms. Games and Economic
Behavior, 103:94–115, 2017.
Sidney I Dobrin. AI and writing. Broadview Press, 2023.
Anna Dreber, Tore Ellingsen, Magnus Johannesson, and David G Rand. Do people care about
social context? framing effects in dictator games. Experimental Economics, 16:349–371,
2013.
Catherine Eckel and Rick K Wilson. Is trust a risky decision? Journal of Economic Behavior &
Organization, 55(4):447–465, 2004a.
Catherine Eckel and Rick K Wilson. Whom to trust? choice of partner in a trust game. Depart-
ment of Economics, Virginia Tech, 2004b.
Catherine C Eckel and Philip J Grossman. Sex differences and statistical stereotyping in atti-
tudes toward financial risk. Evolution and human behavior, 23(4):281–295, 2002.
Roman Egger and Joanne Yu. A topic modeling comparison between lda, nmf, top2vec, and
bertopic to demystify twitter posts. Frontiers in sociology, 7:886498, 2022.
Tore Ellingsen and Robert Östling. When does communication improve coordination? American
Economic Review, 100(4):1695–1724, 2010.
Tore Ellingsen, Magnus Johannesson, Johanna Mollerstrom, and Sara Munkhammar. Social
framing effects: Preferences or beliefs? Games and Economic Behavior, 76(1):117–130,
2012.
Ernst Fehr and Klaus M Schmidt. The economics of fairness, reciprocity and altruism–
experimental evidence and new theories. Handbook of the economics of giving, altruism
and reciprocity, 1:615–691, 2006.
Emilio Ferrara. Genai against humanity: Nefarious applications of generative artificial intelli-
gence and large language models. Journal of Computational Social Science, pages 1–21,
2024.
Matthew Gentzkow, Bryan Kelly, and Matt Taddy. Text as data. Journal of Economic Literature,
57(3):535–574, 2019.
Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure.
arXiv preprint arXiv:2203.05794, 2022.
Pablo Hernandez-Lagos. Cooperative initiative through pre-play communication in simple
games. Journal of Behavioral and Experimental Economics, 80:108–120, 2019.
Christiaan Hetzner. Chatgpt’s creator sam altman was asked what humans would still do better
than ai—he didn’t have a convincing answer, 2024. URL https://finance.yahoo.com/
news/chatgpt-creator-sam-altman-asked-142037284.html.
Daniel Houser and Erte Xiao. Classification of natural language messages using a coordination
game. Experimental Economics, 14:1–14, 2011.
R Mark Isaac and James M Walker. Communication and free-riding behavior: The voluntary
contribution mechanism. Economic Inquiry, 26(4):585–608, 1988.
Hamed Jelodar, Yongli Wang, Chi Yuan, Xia Feng, Xiahui Jiang, Yanchao Li, and Liang Zhao.
Latent dirichlet allocation (lda) and topic modeling: models, applications, a survey. Multimedia
Tools and Applications, 78:15169–15211, 2019.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang,
Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation.
ACM Computing Surveys, 55(12):1–38, 2023.
Irina Kalabikhina, Ekaterina Zubova, Natalia Loukachevitch, Anthony Kolotusha, Zarina
Kazbekova, Evgeny Banin, and German Klimenko. Identifying reproductive behavior argu-
ments in social media content users’ opinions through natural language processing tech-
niques. Population and Economics, 7(2):40–59, 2023.
Tomihisa Kamada, Satoru Kawai, et al. An algorithm for drawing general undirected graphs.
Information processing letters, 31(1):7–15, 1989.
E. Krupka and R. Weber. Identifying social norms using coordination games: Why does dictator
game sharing vary? Journal of the European Economic Association, 11(3):495–524, 2013.
Margarita Leib, Nils Köbis, Rainer Michael Rilke, Marloes Hagens, and Bernd Irlenbusch. Cor-
rupted by algorithms? how ai-generated and human-written advice shape (dis) honesty. The
Economic Journal, 134(658):766–784, 2024.
Steven Loria. Textblob documentation. Release 0.15,
https://buildmedia.readthedocs.org/media/pdf/textblob/latest/textblob.pdf, 2(8):269, 2018.
Christopher Lucas, Richard A Nielsen, Margaret E Roberts, Brandon M Stewart, Alex Storer,
and Dustin Tingley. Computer-assisted text analysis for comparative politics. Political Analy-
sis, 23(2):254–277, 2015.
Qiaozhu Mei, Yutong Xie, Walter Yuan, and Matthew O Jackson. A turing test of whether ai chat-
bots are behaviorally similar to humans. Proceedings of the National Academy of Sciences,
121(9):e2313925121, 2024.
Peter G Moffatt. Experimetrics: Econometrics for Experimental Economics. Palgrave Macmillan,
2015.
Yaw Nyarko, Andrew Schotter, and Barry Sopher. On the informational content of advice: A
theoretical and experimental study. Economic Theory, 29(2):433–452, 2006.
OpenAI, 2023. URL https://openai.com/gpt-4. ChatGPT4.
Matthew Rabin. Psychology and economics. Journal of economic literature, 36(1):11–46, 1998.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Lan-
guage models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
David G Rand, Joshua D Greene, and Martin A Nowak. Spontaneous giving and calculated
greed. Nature, 489(7416):427–430, 2012.
Ariel Rubinstein. Why are certain properties of binary relations relatively more common in
natural language? Econometrica: Journal of the Econometric Society, pages 343–355,
1996.
Bennett Saltzman and Julieta Yung. A machine learning approach to identifying different types
of uncertainty. Economics Letters, 171:58–62, 2018.
Arvind Satyanarayan and Graham M Jones. Intelligence as agency: Evaluating the capacity of
generative ai to empower or constrain human action. 2024.
Andrew Schotter. Decision making with naive advice. American Economic Review, 93(2):196–
201, 2003.
Andrew Schotter. Advice, Social Learning and the Evolution of Conventions. Cambridge Uni-
versity Press, 2023.
Andrew Schotter and Barry Sopher. Trust and trustworthiness in games: An experimental study
of intergenerational advice. Experimental Economics, 9(2):123–145, 2006.
SN Sirimanne. How artificial intelligence chatbots could affect jobs, united nations conference
on trade and development. unctad, 2023.
Joel Sobel. Giving and receiving advice. Advances in economics and econometrics, 1:305–341,
2013.
Dan Sperber and Deirdre Wilson. Relevance: Communication and cognition, volume 142. Har-
vard University Press Cambridge, MA, 1986.
Kai Spiekermann and Arne Weiss. Objective and subjective compliance: A norm-based expla-
nation of ‘moral wiggle room’. Games and Economic Behavior, 96:170–183, 2016.
J Strong, K Hao, T Ryan-Mosley, and E Cillekens. Ai reads human emotions. should it, 2020.
Cass R Sunstein and Lucia Reisch. Do people like algorithms? a research strategy. A Research
Strategy (August 18, 2023), 2023.
Yla R Tausczik and James W Pennebaker. The psychological meaning of words: Liwc and
computerized text analysis methods. Journal of language and social psychology, 29(1):24–
54, 2010.
Dustin Tingley, Michael Yeomans, and Alejandro Kantor. The politeness package: Detecting
politeness in natural language. R Journal, 10(2), 2018.
Anna Wierzbicka. English: Meaning and culture. Oxford University Press, 2006.
Nanyin Yang, Marco Palma, and Andreas C Drichoutis. Humanization of virtual assistants and
delegation choices. 2023. URL https://mpra.ub.uni-muenchen.de/119275/.
Ilan Yaniv. The benefit of additional opinions. Current directions in psychological science, 13
(2):75–78, 2004.
23

The authors extend special thanks to Yoram Halevy, David J. Cooper,
Weiguo Fan, Caleb Cox, Paul Feldman, and Grace Melo Guerrero for the helpful comments and
observations. The authors also wish to thank the support from the Federal Bank of New York,
the BEEMA, and the attendees of the 2023 Economic Science Association meetings.

Chatbotinstrumentalltreatments.pdf
Instrument protocol
Designchatbot.png
Design

Download PDF

Version 1

posted

You are reading this latest protocol version

Chatbot or Humanaut? How the Source of
Advice Impacts Behavior in One-shot Social
Dilemmas

Status:

Version 1

Abstract

Procedure

References

Acknowledgements

Supplementary Files

Status:

Version 1

Privacy Policy

Terms of Service

Chatbot or Humanaut? How the Source of Advice Impacts Behavior in One-shot Social Dilemmas

Status:

Version 1

Abstract

Procedure

References

Acknowledgements

Supplementary Files

Status:

Version 1

Privacy Policy

Terms of Service

Manage Cookie Preferences

Chatbot or Humanaut? How the Source of
Advice Impacts Behavior in One-shot Social
Dilemmas