Abstract
Strategies aimed at reducing negative attitudes toward immigrants are at the core of integration policies. A large literature shows that misperceptions about the size and characteristics of immigrants are common. A few studies implemented interventions to correct innumeracy regarding the size of the immigrant population, but they did not detect any effects on attitudes. We study whether providing information not only about the size but also about the characteristics of the immigrant population can have stronger effects. We conduct two online experiments with samples from the United States, providing one-half of the participants with five statistics about immigration. This information bundle improves people’s attitudes toward current legal immigrants. Most effects are driven by Republicans and other groups with more negative initial attitudes toward immigrants. In our second experiment, we show that treatment effects persist one month later. Finally, we analyze a large cross-country survey experiment to provide external validity to the finding that information about the size of the foreign-born population is not enough to change policy views. We conclude that people with negative views on immigration before the intervention can become more supportive of immigration if their misperceptions about the characteristics of the foreign-born population are corrected.
Motivation
A large proportion of the populations of the United States and Western Europe view immigration as one of the most pressing issues facing their country. For instance, more than three-quarters of British citizens want to reduce immigration (Blinder 2015), and more than 40% of Americans are dissatisfied with the level of immigration in the United States (Gallup 2016). Political parties and politicians who have tapped into these concerns—such as the Front National in France, the Northern League in Italy, or President Donald Trump in the United States—have gained support in the last few years.
Even though immigration is a central issue in many national elections, voters remain highly misinformed about the size (Citrin and Sides 2008; Duffy and Frere-Smith 2014; Herda 2010) and characteristics of the immigrant population (Blinder 2015; Herda 2015, 2018).1 Indeed, some evidence suggests that these misperceptions have been recently growing in the United States, driven by politically conservative Americans (Herda 2019b).
Both the perceived size and characteristics of the immigrant group could be sources of perceived collective threat for natives, which may contribute to the formation of negative attitudes toward immigrants (Quillian 1995). Misperceptions about these attributes can thus exacerbate prejudice and hamper the integration of immigrants into society. Although beliefs and attitudes toward immigrants represent only one of the many forces driving assimilation dynamics, their importance cannot be disregarded (Drouhot and Nee 2019). Our work studies the effects of an information package that corrects people’s beliefs about the proportion and characteristics of immigrants in the United States. We test whether this information treatment can change people’s beliefs about immigrants and their policy preferences regarding immigration. Our findings indicate that this information package is effective at correcting misperceptions, and it affects the immigration policy preferences of Republicans and of those who generally opposed immigration before the intervention. We present evidence that information merely about the size of the immigrant group is not enough to generate significant effects, and we conclude that correcting misperceptions about the characteristics of immigrants is a more promising intervention.
Related Literature
A large iterature in economics, political science, and sociology has shown that people tend to overestimate the size of minority groups (Alba et al. 2005; Gallagher 2003; Herda 2010; Kunovich 2017; Laméris et al., 2018a; Lawrence and Sides 2014; Nadeau and Niemi 1995; Nadeau et al. 1993; Sides and Citrin 2007). Theories of intergroup threat posit that people feel more threatened when they perceive a larger size of the minority group (Blalock 1967; Blumer 1958; Bobo and Hutchings 1996; Quillian 1995), and several empirical papers have discussed whether size misperceptions could be correlated with negative attitudes toward minorities (Citrin and Sides 2008; Hjerm 2007; Hooghe and de Vroome 2015; Laméris et al., 2018b; Semyonov et al. 2004).
Based on this argument, we would expect that an intervention correcting people’s misperceptions about the size of the immigrant group will improve attitudes toward immigrants. However, in empirically testing this hypothesis, Sides and Citrin (2007) and Hopkins et al. (2019) found that correcting misperceptions about the size of the immigrant group has only limited effects on people’s attitudes toward immigrants.
Theories of social identification and social identity (Stets and Burke 2000) suggest that what matters for the formation of preferences about an out-group is not the perceived size of the group but instead the perceived characteristics of that group. This intuition can lead to an extension of intergroup threat theories, where the feeling of threat derives not only from the size but also from the perceived characteristics of the minority group. The argument is supported by recent studies finding that group size is not the only or even the most relevant source of misperception regarding minorities, and that there is a high degree of innumeracy for other dimensions related to the characteristics of the immigrant group (Alesina et al. 2018; Blinder 2015; Herda 2015, 2018). Therefore, correcting misperceptions about the characteristics of immigrants may be a more effective way to generate a change in attitudes.
Our study contributes to this literature by studying the effects of a comprehensive information package that corrects people’s beliefs about the proportion of immigrants and their characteristics. To select the relevant characteristics of immigrants that we include in our information package, we rely on work in political science (Bansak et al. 2016; Blinder 2015; Hainmueller and Hopkins 2014) and sociology (Flores and Schachter 2018; Schachter 2016). Blinder (2015) investigated who people have in mind when they think of immigrants, and Hainmueller and Hopkins (2014), Flores and Schachter (2018), and Schachter (2016) employed conjoint experiments and randomized profiles of hypothetical individuals with different characteristics to measure support for each particular attribute. The evidence indicates that non-Hispanic White Americans prefer immigrants who are employed, are documented, and speak English.2 These empirical findings are supported by theoretical work indicating that the native-born population subjectively picks a number of characteristics to define who is an insider and who is not. This subjective sense of similarity, or symbolic belonging (Schachter 2016), determines natives’ conception of the “deserving” immigrant.3
Main Hypotheses
Our intervention aims to correct misperceptions about the size and the characteristics of the immigrant population. Our first hypothesis is that these misperceptions exist and that we can correct them by providing credible information. This is the first step in our theory of change, which we test in two ways. First, we study whether our experimental group updates their beliefs about the statistics we provide in the short term and whether this update in beliefs persists one month later. Second, we estimate the treatment effects on beliefs about the size of the immigrant group and characteristics of immigrants that are directly linked to the information provided.
Our second hypothesis is that our intervention can also change beliefs about more general characteristics of immigrants that are not directly linked to the information provided. We hypothesize that people can develop more positive beliefs about immigrants if the information provided in the treatment makes them realize that immigrants living in the country are similar to the “deserving” immigrant category they have formed in their minds. This would be in line with the work on symbolic belonging.
Our third hypothesis is that our intervention might also affect immigration policy preferences. This third step in the theory of change will be observed if the change in people’s beliefs translates into a change in preferences regarding policy. In this case, we hypothesize that there could be important heterogeneity in results by political affiliation. On the one hand, there could be a ceiling effect given that Democrats in the United States have more positive views regarding immigrants to begin with, as our data confirm, leaving less room to change their policy preferences. On the other hand, the literature on motivated reasoning (Taber and Lodge 2006) posits that people who receive information that goes against their political convictions might be less willing to update their beliefs than people for whom the information is in line with their political orientation, which would indicate that Republicans will actually react less to positive information about immigrants.4
We expect that if there is a change in policy preferences, this will mainly happen for policies regarding legal immigrants and not for those regarding undocumented immigrants. Our hypothesis is that our experimental treatment makes people realize that immigrants are more similar to themselves than they originally thought. This will hold for a general immigrant category and would not apply directly to the particular subgroup of undocumented immigrants, which according to the previously cited evidence, is not considered part of the “deserving” immigrant category.
In the main analysis, we focus on three families of outcomes that allow us to test these hypotheses: people’s beliefs regarding the variables directly targeted by the intervention, their general beliefs about immigrants, and their policy preferences.5 We complement these families of outcomes with two behavioral measures: donations to a pro-immigrant charity, and willingness to sign a petition in favor of increasing the number of green cards. The use of real online petitions and donations is novel in the literature and can be widely applied by researchers to examine people’s support for various policy proposals.
We pre-registered the experimental design, our hypotheses, and our empirical specifications on the Social Science Registry before running our two online experiments. Almost all the analyses we present were pre-specified. We explicitly note which analyses were not part of the pre-analysis plan, which is available online. We conduct two experiments with identical design to test our hypotheses. Experiment 1 uses an online sample from the United States, which matches the U.S. population in terms of age, gender, and region of residence. Experiment 2 uses a voluntary response sample recruited on Amazon Mechanical Turk (MTurk) and includes the follow-up survey conducted four weeks after the main experiment that we use to measure persistence in beliefs.6
Experiment 1: TNS Global
Sample
We conducted Experiment 1 using a sample of the U.S. population, provided by TNS Global, a world-leading company in market research and political surveys. This sample of 1,193 people living in the United States was obtained as a nonprobability quota sample to match the U.S. population in terms of age, gender, and region of residence.7 All participants completed the survey online using a link provided by TNS Global.8
To participate in the experiment, people had to pass a standard attention screener at the start of the survey (Berinsky et al. 2014).9 The experiment was run at the beginning of September 2016. The characteristics of the whole sample are described in Table 1. Overall, 49% of participants are male, and the median age in our sample is 39, which is very close to the U.S. national average of 38 according to the American Community Survey. Similarly, 81% of our participants identify as White, and the percentage identifying as White in the United States is approximately 77.5%.10 The median household income in the TNS sample is $65,000, compared with $56,516 for the national estimate. Finally, 66% of the TNS sample reported being employed either part-time or full-time, which is close to the employment-population ratio for the United States (60%). Among our respondents in the TNS sample, 32% self-identify as Republicans, and 45% self-identify as Democrats, which is relatively similar to the shares for the United States. Participants in the TNS sample are more educated than the average American, which is very common in online samples. The randomization worked as expected, and our samples are balanced across the treatment and control group, as shown in Table 1.
Design
Pre-treatment Characteristics and Prior Beliefs
The experiment is structured as follows. First, all respondents were asked a few questions on how much they trust official statistics, how many petitions they have signed in the last 12 months, and how worried they are about immigration. Then, we asked them to estimate five statistics about immigration: the proportion of immigrants in the United States, the proportion of undocumented immigrants in the United States, the unemployment rate of immigrants, incarceration rate of immigrants, and proportion of immigrants who cannot speak English.11 To help participants give plausible estimates for the unemployment rate and the incarceration rate of immigrants, we told them what these rates are for U.S.-born citizens. This comparison reduces the concern that participants might not be able to translate their perception into numerical terms when asked about absolute shares (Alba et al. 2005). Both the treatment and the control group received this information, and the internal validity of our study is therefore not compromised.
Information Treatment
Only the treatment group was told the correct answers to these five questions. We reminded participants in the treatment group of their estimate before providing them with the correct answer. For instance, participants received the following feedback for the question on the unemployment rate of immigrants:
“You estimated that X percent of immigrants are unemployed. According to the American Community Survey, around 6 percent of immigrants are unemployed.”
Post-treatment Beliefs
We then asked all participants a series of questions on their perception of legal and undocumented immigrants. We first measured people’s agreement to the following three statements that are directly linked to the information provided on the characteristics of immigrants: (1) immigrants are more likely to commit crimes than U.S. citizens; (2) immigrants are more likely to be unemployed than U.S. citizens; and (3) immigrants generally learn English within a reasonable amount of time. We then focused on beliefs regarding the other two variables targeted by the intervention: views about the number of legal and undocumented immigrants. We asked participants to choose their agreement with the following statements: (1) there are currently too many immigrants in the United States, and (2) there are currently too many illegal immigrants in the United States. We also specifically measured people’s more general beliefs regarding immigrants. We asked them about the effects of removing undocumented immigrants from the United States and whether “over the last 10 years, immigrants have produced more disadvantages than advantages for the United States as a whole.”
Policy Views
We then obtained participants’ views on specific immigration policies that are at the core of the policy debate in the United States. First, we measured whether people think that the number of legal immigrants coming to the United States each year should be increased, reduced, or remain the same; we also assessed whether they think that the number of green cards available for immigrants coming to the United States each year should be increased, reduced, or remain the same.
On top of these questions on legal immigration, we also measured people’s views regarding unauthorized immigration. We measured people’s agreement to the statements, “The government should devote a larger share of its budget to find illegal immigrants, and to deport them” and “Congress should pass a bill to give some illegal immigrants living in the U.S. a path to legal status.” Finally, we measured people’s views on whether the government should “deport all illegal immigrants back to their home country, allow illegal immigrants to remain in the United States in order to work, but only for a limited amount of time, or allow illegal immigrants to remain in the United States and become U.S. citizens, but only if they meet certain requirements over a period of time.”
Behavioral Measures
We obtained two behavioral measures, introduced in a random order. First, we gave participants the option of signing an online petition in favor of facilitating legal immigration into the United States by increasing the number of green cards available for immigrants. We created two identical petitions on the White House website, and we gave different links to participants in the treatment and control groups.12 This is a credible measure of people’s support for immigration given that it requires some effort to sign the petition (i.e., people need to create an online profile and to sign with their initials). Furthermore, this behavioral measure involves a real petition with potentially concrete consequences, which attenuates concerns about its external validity.
Second, we told participants that 10% of them would receive $10, and that they must specify how much money they want to keep for themselves and how much they want to give to the American Immigration Council, a nonprofit organization that “promotes laws, policies, and attitudes that preserve [the United States’] proud history as a nation of immigrants” (American Immigration Council 2016) in case they receive the $10. Because people need to forgo some of their own money in order to support the pro-immigrant NGO, this behavioral measure may be deemed more credible than self-reported measures as a valid indicator of participant’s preferences (Bonica 2019).
After the behavioral measures, participants completed a standard attention check designed to assess how attentive participants were in the experiment. Then we asked participants in the treatment group to estimate again the same five statistics for which we had elicited prior beliefs so that we could test whether they updated their beliefs and how well they remembered the information. Finally, respondents completed a questionnaire on demographics, such as gender, age, education, and income.
Main Results: Experiment 1
We account for multiple hypothesis testing by adjusting the p values using the sharpened q value approach.15 For each table, we also create an index of the outcomes, which we regress on the treatment indicator.
Changes in Beliefs About Characteristics Targeted by the Intervention
In this section, we show that participants in the treatment group strongly updated their beliefs about the characteristics of immigrants targeted by the intervention, which is in line with our first hypothesis.
In Fig. 1, we show the average estimates that treated participants gave before and after receiving the correct information. It is clear that before the treatment, participants had biased beliefs about immigration. Their estimates were, on average, consistently higher than the actual values.16 For example, people overestimated the percentage of immigrants in the United States by more than 20 percentage points.
Similarly, panel A of Table 2 shows that compared with control group respondents, treated respondents are less likely to report that immigrants commit more crimes than U.S. citizens, that they take too much time to learn English, and that they have a higher unemployment rate than natives. All these results are statistically significant; the effect sizes are large and correspond to more than one-half of the gap between Democrats and Republicans.17 Furthermore, treated participants are less likely to state that there are too many legal and undocumented immigrants in the United States. These effects are statistically significant; their effect size is 0.10 and 0.24 standard deviations for legal and undocumented immigrants, respectively.
Changes in General Beliefs About Immigrants
In line with our second hypothesis, the information treatment also has an effect on how people perceive immigration generally, as shown in Table 3. People in the treatment group are less likely to say that immigrants have produced more disadvantages than advantages for the United States as a whole over the last 10 years. This result is significant at the 1% level, and the effect size is 0.14 standard deviations. Although treated respondents are also more likely to say that there would be no positive effects from removing undocumented immigrants, the coefficient (0.09 standard deviations) is only marginally significant.18
Policy Preferences
We next examine whether the information provision also affects people’s views about immigration policy, the last step in our theory of change.
In Table 4, we observe that treated respondents become more likely to be in favor of increasing the number of legal immigrants (0.13 standard deviations). However, we see no effects on their views about the number of green cards to issue every year or on the legalization of immigrants. Similarly, participants’ views on deporting undocumented immigrants and on the budget that should be devoted to it are not significantly affected by the treatment. Overall, the index of policy preferences is not affected for the average participant in our experiment.
Petition and Donations
Table 5 shows that there is no treatment effect on the probability of signing the online petition on the White House website in favor of increasing the number of green cards available for immigrants.19 Similarly, approximately the same fraction of people in the treatment and control group reported both intending to sign and having signed the petition.20
Finally, we find no statistically significant effects on people’s willingness to donate to a pro-immigration charity, the American Immigration Council. The effect is 0.07 standard deviations, with a 95% confidence interval that includes 0 (the confidence interval includes effects between –0.04 and 0.18).
The lack of significant effects on the two behavioral outcomes for the average respondent are in line with the lack of effects on policy preferences reported earlier.
Summary
Overall, our first experiment shows that when people are provided with information about the size and the characteristics of immigrants, they update their beliefs regarding the characteristics directly targeted by the intervention and more general beliefs regarding legal immigrants. However, there are no significant changes in policy preferences.
Experiment 2: MTurk Panel With Follow-up Survey
Sample
We replicated our first experiment on Amazon Mechanical Turk (MTurk), an online labor marketplace developed by Amazon.com that is commonly used by academics to recruit participants for online experiments. The pool of workers on MTurk is a voluntary response sample but is still more representative of the U.S. population than student samples typically used in laboratory experiments. Moreover, MTurk participants have been shown to be more attentive to instructions than college students (Hauser and Schwarz 2016). There is some concern about the MTurk sample related to the rising prevalence of bots as well as MTurkers’ high level of experience. However, our findings that the MTurk sample and the sample from TNS global yield very similar results reassure us of the data quality and alleviate concerns about the peculiarities of each of the samples.
The experiment was run in March 2016. In total, 802 participants completed it. Fewer than 10 people dropped out after the treatment section in this survey, implying an attrition rate of less than 2%. Table 1 summarizes the characteristics of the sample. Overall, 55% of participants are male. The median age in our sample is 35, compared with 38 for the United States. Moreover, the median income in our sample is $45,000, compared with $56,516 for the general population. Similarly, 78% of our participants identify as White, compared with 77.5% for the United States. The percentage of unemployed people in our sample (8%) is slightly higher than in the general population (5%), and the percentage of employed people (76%) is also larger than in the general population (60%). Participants in the MTurk sample are younger, more likely to be in the labor force (both the share employed and unemployed are larger than in the general population), and more likely to be Democrat.
Four weeks after our main experiment, we invited everyone who had completed the main experiment to complete a follow-up survey. The percentage of participants who completed both the main experiment and the follow-up is 88%. This high recontact rate indicates that it is possible to construct panels on MTurk with relatively low attrition. The recontact rates are very similar for treatment and control groups, and they are statistically indistinguishable (p value = .708). The randomization worked, and our samples are balanced across treatment and control groups for both the sample in the main experiment and the sample that completed the follow-up (Table 1).
Design
The design of Experiment 2 is almost identical to that of Experiment 1, with a few differences as noted.
Incentives and Attention Check
In Experiment 2, we incentivized the pre-treatment questions about immigrant characteristics. Participants received 10 cents for each question (8% of the participation fee) if their estimate was within 3 percentage points of the official value, which we obtained from the American Community Survey (ACS).
Moreover, to avoid having participants look up the answers online, we gave them only 25 seconds to answer each question. We did not include an attention check in Experiment 2. However, response times are similar to those of Experiment 1, indicating that respondents were not less attentive in this experiment.21
Follow-up Study
The key innovation in Experiment 2 is that we conducted a follow-up study four weeks after the main experiment, allowing us to examine whether the treatment effects persisted over time. We asked people the same set of self-reported questions on immigration as the ones they answered in the main experiment, and we also asked them to estimate the same five statistics about immigration, again providing the same incentives for each correct answer as in the main experiment. This allows us to see whether people in the treatment group remembered the information provided.22
Results: Experiment 2
Main Survey
For the main survey outcomes, the results are fairly similar to those of Experiment 1. We formally test the equality of treatment effects across the two samples, finding only a few cases where we reject the equality of coefficients. We find two main differences compared with the findings from Experiment 1. First, we find a larger and statistically significant effect of the treatment on donations (although the confidence interval overlaps with that of Experiment 1). MTurkers in the treatment group donated, on average, 36% more ($0.42 more) to the American Immigration Council than MTurkers in the control group. As shown in column 5 of Table 5, this represents an effect of 0.22 standard deviations. Second, we find a stronger effect on beliefs about characteristics directly targeted by the intervention, as illustrated in Table 2.
Follow-up Study
We leverage the follow-up study to shed light on the persistence of the effects of the information provision on beliefs and policy views. We first test the extent to which MTurkers in the treatment group remember the information four weeks after the main experiment. In Fig. 2, we show that estimates were still fairly accurate four weeks after the treatment. For instance, the average estimate of the proportion of immigrants is 15% in the follow-up, whereas the true value is 13%.23 Moreover, those respondents who were the most biased updated their beliefs the most, even in the follow-up. We observe a clear linear positive relationship between the revision of beliefs (the difference between priors in the main experiment and posteriors measured one month later) and the size of the initial bias in the treatment group.24
We also show in panel C of Table 2 that the effects on qualitative questions measuring beliefs about immigrants targeted by the intervention persist four weeks after the treatment, that they are statistically significant, and that they remain fairly large (about 0.2 of a standard deviation). We see slightly larger treatment effects on policy preferences (mostly around 0.1 of a standard deviation) in the follow-up. However, they are not statistically different from those in the main experiment.
Summary
Our findings in Experiment 2 confirm, with a different sample, what we find in Experiment 1. More interestingly, the follow-up experiment shows that the average effects of the information package on beliefs about immigrants are not ephemeral. We now explore whether the null average effects on policy preferences are masking important heterogeneity across different groups of participants.
Heterogeneous Treatment Effects: Experiments 1 and 2
In this section, we study heterogeneity in treatment effects across different subgroups. We focus on the indices for our main five families of outcomes. To increase statistical power and because results are similar for our two experiments, we pool both samples and present effects for the pooled sample (see Table A1 in the online appendix for disaggregated results). We estimate the following equation, where interactioni refers to the pre-specified group of interest, Xi is a vector of predetermined characteristics, π1 captures the magnitude of the heterogeneity in treatment effects, π2 measures the effect for the omitted group, and π1 + π2 gives the treatment effect for the studied group:
yi = π0 + π1Treatmenti × interactioni + π2Treatmenti + π3interactioni + ΠTXi + εi.
Political Affiliation
Panel A of Table 6 shows that people who self-identify as Republican or people who identify as neither Democrat nor Republican respond more strongly to the information treatment than people who identify as Democrat.25 At the bottom of the table, we report the p value from the test for the null hypothesis that there is no treatment effect for Republicans. We reject the null hypothesis at the 5% level in all cases: Republicans exhibit statistically significant improvements in all our main families of outcomes. The heterogeneity result can be partly explained by the fact that Republicans have more negative values for all outcomes to begin with, which implies that the information treatment is actually stronger for them.26
Initial Attitudes Toward Immigrants
As shown in panel B of Table 6, participants from the TNS and MTurk samples who are particularly worried about immigration tend to respond more strongly to the treatment. Although not all interaction coefficients are statistically significant in this case, we see a consistent pattern of larger effects.
Overall, we do not find evidence in favor of motivated reasoning theories or self-confirmation bias. Republicans and participants who initially have more negative views on immigrants update their beliefs and policy preferences more than people who have more positive attitudes toward immigrants.
Other Sources of Heterogeneity
In panel C of Table 6, we examine whether participants who have a high level of trust in official statistics respond more strongly to information. Overall, we find no consistent evidence in this direction.27
Persistence of Heterogeneous Effects
We find a consistent pattern of heterogeneous treatment effects in the follow-up (see panel C of Table A1, online appendix). Even four weeks after the treatment, the effects are stronger for Republicans, especially regarding their policy preferences. Our results are unlikely to be driven by experimenter demand effects for two reasons: First, demand effects have been shown to be quantitatively small (de Quidt et al., 2018; Mummolo and Peterson 2019) and second, both the heterogeneity of treatment effects and the persistence of effects over time suggest that demand effects are unlikely to be causing the patterns in our data.
Experiment 3: Cross-country Experiment
Evidence presented so far suggests that a package including information on both the size of the immigrant group and on the characteristics of immigrants can affect both beliefs for the average respondent and policy preferences for those with more negative views on immigration. In a third experiment, we show that information on size alone is not enough to generate significant effects. Consistent with the previous literature, we find evidence from the United States in this direction, and we provide external validity for these results by showing similar effects for other countries where the same experiment was conducted.
Description of the Data Set
We use data from the Transatlantic Trends Survey (TATS), which is a large representative survey on political attitudes conducted every year in the United States and in many other countries around the world. In particular, we focus on two waves of the survey, the 2010 and 2014 waves, which included an experiment on the effect of information about the size of the immigrant group.28
The 2010 wave of the TATS was conducted in the United States, Canada, Germany, France, Italy, the United Kingdom, the Netherlands, and Spain; in each country, participants were randomly drawn from the adult population with access to a landline. The 2014 wave added Greece, Portugal, Sweden, Russia, and Poland, but it did not include Canada; in most countries, participants were randomly drawn from the adult population with access to a landline or a mobile phone.29 Importantly, more than 94% of those who started the survey answered the main questions of interest.30
Information Treatment
At the start of the survey, participants were asked which issues they think are the most important ones facing their country, and how closely they follow news on immigration. Then only participants in the treatment group were informed about the true proportion of immigrants in their country, before being asked whether they think that there are too many immigrants in their country. Thereafter, all respondents were asked a series of questions on their level of concern regarding immigration, their perception of immigrants, and the legalization of immigrants. For example, people were asked whether they are worried about legal and undocumented immigration, and whether undocumented immigrants should be given the opportunity to obtain legal status.
Results
In parallel with our first hypothesis, we first check whether the information experiment embedded in the TATS affects beliefs regarding the size of the immigrant group, the only variable directly targeted by the intervention. Column 1 of Table 7 shows that this is the case. People who receive information about the share of immigrants in the United States become much less likely to say that there are too many immigrants in the country (an effect of 0.33 standard deviations). We find a similar effect for the average respondent across all other countries included in the survey (column 4, Table 7).31 This finding confirms that the information treatment is effective at correcting misperceptions about the size of the immigrant group.
However, we expected that this light information treatment, correcting only misperceptions about the size of immigrant group, would not meaningfully shift people’s general beliefs about immigrants or their immigration policy preferences. In line with our expectation and the findings of the existing literature (Hopkins et al. 2019), columns 2–3 and 5–6 of Table 7 show that being informed about the proportion of immigrants in the country does not make people (in the United States or in the other countries) less worried about immigration. Moreover, Table 8 confirms that this treatment does not change people’s immigration policy preferences.
Finally, as shown in panel B of Tables 7 and 8, the effects on general beliefs and policy preferences are not statistically larger for right-wing respondents. This finding stands in contrast to the heterogeneous effects we found after providing information about both the size and the characteristics of the immigrant group, which significantly affected policy preferences for respondents with more negative views on immigration.
Conclusion
The main substantive contribution of our study is to provide novel causal evidence on the effects of misinformation about immigrants’ characteristics on people’s policy preferences. We show that providing a package of information that includes not only the size of the immigrant group but also the characteristics of immigrants improves people’s general beliefs about legal immigrants. We also see significant effects on policy preferences for those groups with more negative attitudes toward immigration before the intervention. Based on motivated reasoning theory, previous studies have hypothesized that a stronger perception of threat can lead to more negative attitudes and generate larger misperceptions. However, our findings provide evidence for the reverse causal mechanism: innumeracy can cause negative attitudes toward immigrants. The result that a reduction in misperceptions leads to less negative attitudes is consistent with a version of group-threat theories that view the perceived characteristics of immigrants as the source of the threat.
Our findings have high policy relevance for at least two reasons. First, we show that targeting relevant subgroups can be essential for successful information campaigns. People with negative views on immigration (e.g., Republicans) become more supportive of legal immigration if their misperceptions about the characteristics of the foreign-born population are corrected. Second, we show that the type of information provided makes a difference and that including objective statistics about the characteristics of immigrants can reduce social distance and thereby increase support for immigration. Interventions focused on information about only the size of the immigrant group have not been effective at affecting policy preferences.
The effects of information on beliefs persist after one month, which indicates that information campaigns can have an effect that is not ephemeral. However, we do see an important reduction in the size of the treatment effect after one month. Therefore, we believe that over a longer time horizon, the effects on beliefs could disappear as a result of imperfect memory and the impact of competing pieces of information. To persistently shift beliefs, political organizations would need to run information campaigns repeatedly.
Future research should complement our work in at least two ways. First, it is important to grasp whether the effects of information on political attitudes depend on the credibility of the agent who provides the information (e.g., the government or the media). Second, it is crucial to understand how people process factual information compared with emotionally loaded content.32 Answering these questions will be necessary to find the most effective ways of fighting people’s misinformation on important issues, such as immigration.
Acknowledgments
We thank all the participants of the NBER political economy summer institute (2017); ESA conference in Bergen; the applied coffee at Bonn; the conference on Managing Migration in Siracusa; the EPEC Workshop in Political Economy; and seminars at Bocconi University, Columbia University, IIES Stockholm, the University of Oxford, and Uppsala University. Financial support from IGIER (Bocconi) and the Oxford Economic Papers fund is gratefully acknowledged. Christopher Roth acknowledges funding under the grant Policy Design and Evaluation Research in Developing Countries Initial Training Network (PODER), which is funded under the Marie Curie Actions of the EU Seventh Framework Programme (Contract Number: 608109). Alexis Grigorieff would like to thank the Economic and Social Research Council for their financial support (Grant No. SSD/2/2/16). Diego Ubfal acknowledges funding from the Bocconi Young Researcher Grant. We also thank the Centre for Experimental Social Sciences at the University of Oxford for their help as well as Ornella Bissoli for administrative assistance.
Authors’ Contributions
All authors contributed equally to the study conception and design, material preparation, data collection, and analysis. The first draft of the manuscript was co-written by all authors. All authors read and approved the final manuscript.
Data Availability
The data are available on the Open Science Framework on the following link: https://osf.io/rhz8n/.
Compliance With Ethical Standards
Ethics and Consent
IRB approval was obtained at the University of Oxford and Bocconi University.
Conflict of Interest
The authors declare that they have no conflict of interest.
Notes
Throughout the text, we define immigrants as people living in the country but who were not born in that country.
A large literature has studied the determinants of people’s attitudes toward immigrants (Alba et al. 2005; Hainmueller et al. 2015; Scheve and Slaughter 2001). Previous studies have focused on characteristics such as age, media exposure, competition in the labor market, exposure to immigrants, education, or income that are correlated with people’s attitudes toward immigrants (Card et al. 2012; Citrin et al. 1997; Facchini and Mayda 2009; Haaland and Roth 2018; Mayda 2006). Others have included the real or the perceived size of the immigrant group as a key correlate (Gallagher 2003; Hjerm 2007; Hooghe and de Vroome 2015; Semyonov et al. 2004).
Although our results are in line with the notion of a “deserving” immigrant category, our interventions do nothing to encourage moralizing classifications or to advocate support for only one category of immigrants. Andrews (2018) studied how the combination of expanded immigration enforcement and good/bad moralizing classifications can affect undocumented immigrants. Menjívar and Lakhani (2016) showed how the process of applying for legal status can trigger enduring changes by which immigrants try to behave according to the “deserving” immigrant profile.
An influential paper has documented the existence of backfire effects (Nyhan and Reifler 2010), where people’s beliefs actually are reinforced in the face of contradictory evidence. However, recent evidence indicates that these types of backfire effects might not be so common (Guess and Coppock 2018; Wood and Porter 2019).
A precise definition of these families of outcomes can be found in the online appendix.
Previous literature has cast doubt on whether interventions can have persistent effects on beliefs. For example, Flores (2018) found that the effect of anti-immigrant rhetoric by political elites does not persist more than two weeks and attributed them to social desirability bias. In contrast, Herda (2017, 2019a) showed that a classroom activity can correct misperceptions among students, with effects persisting five weeks later; Herda did not, however, examine whether the correction generated changes in attitudes or policy preferences.
Because the sample is not drawn from a probability-based sample, it is not representative in terms of variables not targeted by the quota.
TNS provided us with 1,193 observations rather than 1,000 as we had specified in the pre-analysis plan because they made an error in a count variable, which meant that they underestimated the number of observations and therefore accidentally provided us with a larger sample.
The attrition rate was very low (smaller than 2%). We find no evidence of differential attrition across treatment arms.
Our survey question includes mutually exclusive options for White, Hispanic, Black, Asian, or other ethnicity. Therefore, our White category includes those identifying as Whites, and we cannot distinguish between Hispanic and non-Hispanic Whites.
We chose these statistics for two main reasons. First, as we described in the related literature section, evidence shows that people are particularly concerned about these issues. Americans prefer immigrants who are employed, speak English, and are documented. Second, census data are available on these issues, which increases the reliability of the information we provide.
Only participants with a link can see the petition until at least 150 people sign it, after which it becomes public. Moreover, if the petition reaches 100,000 signatures in 30 days, it is entitled to receive an official reply from the White House.
Robust standard errors are used throughout the analysis.
Among pre-specified covariates, we include measures of prior beliefs, which are missing for less than 2% of our respondents. We impute their values using the set of pre-specified controls displayed in the balance table. In an earlier working paper (Grigorieff et al. 2016), we showed that results were very similar when we did not include the covariates Xi in the regression.
For each family of outcomes, we control for a false discovery rate of 5% (Anderson 2008).
We found that more educated people, males, and people who live in zip code areas with a small share of immigrants tend to have less biased beliefs about the share and characteristics of immigrants (Grigorieff et al. 2016). These findings are consistent with previous research on the determinants of innumeracy in the United States (Alba et al. 2005; Laméris et al., 2018a; Nadeau et al. 1993) and Europe (Herda 2010).
On average, Republicans have a significantly more negative view of immigrants than Democrats for all our outcomes. This is in line with evidence that immigration enforcement is higher in states with a larger share of Republican constituents (Moinester 2018).
We asked respondents some additional questions on the respective contributions of legal and undocumented immigrants, for which we find consistent effects. These estimates are reported in Grigorieff et al. (2016).
About 10% of our sample signed the petition, suggesting that we had sufficient variation to detect treatment effects.
The number of people who reported having signed the petition (25%) is higher than the number of signatures, which can partly be explained by the fact that signing the petition was a multistage process. People who signed the petition received a confirmation email containing a link that they had to click to confirm their signature. If they did not complete this second step, their signature was not counted. People’s intention to sign the petition and their self-reported signature are strongly correlated with their self-reported support for increasing the number of green cards for immigrants.
Another piece of evidence indicating that MTurk data are of high quality is a very high correlation (of around .80) between responses in the follow-up and in the main survey among control group participants.
We randomized the order of the sections in the survey. Half of the sample estimated the five statistics first and then answered the set of self-reported questions on immigration, and the other half answered the self-reported questions first. We find no significant order effects.
People in the control group did not update their beliefs in the follow-up, indicating that they did not make the effort to look up the information we provided to the treatment group.
Our measures of beliefs and attitudes toward immigrants are strongly correlated with people’s self-reported policy preferences regarding immigration. These results were not pre-specified and are available upon request.
Republicans represent 28% of the pooled sample (32% of the TNS sample and 23% of the MTurk sample); the share of Democrats is 45% in the TNS sample and 58% in the MTurk sample. Both the share and observable characteristics of Republicans and Democrats in treatment and control groups are well balanced.
In Grigorieff et al. (2016), we employed a machine learning algorithm to identify the most significant sources of heterogeneous treatment effects (Athey and Imbens 2016). The algorithm confirmed that political affiliation is the factor that most strongly predicts heterogeneous responses to the treatment.
In an additional pre-specified analysis, we examine heterogeneous treatment effects by participants’ biases in beliefs using three different definitions of biases. We find that people with larger biases in beliefs seem to respond more strongly to information. However, this effect is not statistically significant for most families of outcomes, which could be due to measurement error given that we do not know how people weigh the biases for the five statistics we measure. Results are available upon request.
The experiment was designed by the German Marshall Fund of the United States, and the main results were graphically reported in Wunderlich et al. (2010) and Stelzenmueller et al. (2015); those reports did not include any regression or heterogeneity analysis.
In Germany and in the United Kingdom, only people with access to a landline were surveyed. In Poland and Russia, participants were randomly selected from the general population, and face-to-face interviews were conducted. Response rates for phone interviews ranged from 4% in France, the United Kingdom, and the Netherlands, to 27% in the United States. Face-to-face interviews had higher response rates: 49% in Russia and 40% in Poland (Stelzenmueller et al. 2015; Wunderlich et al. 2010).
To obtain a sample that is as representative as possible for each country, we use the probability weights constructed by the Transatlantic Trends Survey. Our results are not affected by the use of these weights.
Results are robust to the inclusion or exclusion of control variables, and wave and country fixed effects. The sample is well balanced across the treatment and control group, as is highlighted in Table A2 in the online appendix.
Flores (2018) showed that altering the source of negative statements about immigrants does not have differential effects on attitudes, but changing the polarity of the message does: only negative messages have an effect.
References
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
