A recent study conducted by a team of political science and computer science professors and graduate students at BYU has examined the potential of using artificial intelligence (AI) as an alternative to human responders in survey-style research. The team tested the accuracy of programmed algorithms of a GPT-3 language model, which imitates the complex relationships between human ideas, attitudes, and sociocultural contexts of varied subpopulations.

Artificial Personas and Voting Patterns

In a single experiment, the researchers created artificial personas by assigning specific characteristics to the AI, equivalent to race, age, ideology, and religiosity. They then tested whether these artificial personas would vote the identical way as humans did within the 2012, 2016, and 2020 U.S. presidential elections. By utilizing the American National Election Studies (ANES) as their comparative human database, they found a high correspondence between AI and human voting patterns.

David Wingate, a BYU computer science professor and co-author of the study, expressed his surprise at the outcomes:

“It’s especially interesting since the model wasn’t trained to do political science — it was just trained on 100 billion words of text downloaded from the web. However the consistent information we got back was so connected to how people really voted.”

Interview-Style Surveys and Future Applications

In one other experiment, the researchers conditioned artificial personas to supply responses from an inventory of options in an interview-style survey, again using the ANES as their human sample. They found a high similarity between the nuanced patterns in human and AI responses.

The study’s findings offer exciting prospects for researchers, marketers, and pollsters. AI may very well be used to craft higher survey questions, refine them to be more accessible and representative, and even simulate populations which can be difficult to achieve. It will possibly even be used to check surveys, slogans, and taglines before conducting focus groups.

BYU political science professor Ethan Busby commented:

“It’s not replacing humans, nevertheless it helps us more effectively study people. It’s about augmenting our ability slightly than replacing it. It will possibly help us be more efficient in our work with people by allowing us to pre-test our surveys and our messaging.”

Ethical Questions and Future Research

As large language models proceed to advance, quite a few questions arise regarding their applications and implications. Which populations will profit from this technology, and which will probably be negatively impacted? How can we protect ourselves from scammers and fraudsters who may manipulate AI to create more sophisticated phishing scams?

While lots of these questions remain unanswered, the study provides a set of criteria that future researchers can use to find out the accuracy of AI models for various subject areas.

Wingate acknowledges the potential positive and negative consequences of AI development:

“We’re going to see positive advantages since it’s going to unlock recent capabilities. We’re also going to see negative things occur because sometimes computer models are inaccurate and sometimes they’re biased. It is going to proceed to churn society.”

Busby emphasizes that surveying artificial personas shouldn’t replace the necessity to survey real people, and calls for academics and experts to collaborate in defining the moral boundaries of AI surveying in social science research.