Human rights 19/02/26 Chinese censors get tough with Gallup - Poll company struggles to get reliable data as authorities block political questions

[성기화 요셉]

Chinese censors get tough with Gallup

Poll company struggles to get reliable data as authorities block political questions

A newspaper featuring a story about a meeting between U.S. President Donald Trump and Chinese President Xi Jinping is seen at a news stand in Beijing on Dec. 3, 2018. China's heavily censored state-run media hailed the trade war truce with the United States as ‘momentous’ but warned of complex negotiations ahead. (Photo by Greg Baker/AFP)

Anders Corr
China
March 12, 2019

Gallup’s Rating World Leaders 2019 is a perception study that polls people in 130 countries on their opinions about the leaders of China, Russia, the United States and Germany. The 2019 study was released on Feb. 26 with a surprising omission — there were no responses listed from Chinese citizens. Jerry Hansen, a Gallup consultant with years of experience on Asian matters, told ucanews.com that the analytics company had no data because China forbids Gallup from asking such political questions. He said in an email: “I’d love to have sent along some trended Chinese opinions about U.S. leadership, but we are not allowed to ask ‘political’ questions like that in China. We push the edge as far as we can to get interesting data there, but there is only so far that we can go and still be allowed to conduct our random population polling there.” Despite the lack of data from China, President Xi Jinping did very well in the most recent poll. The global approval rating of China’s leadership, according to the study, rose to 34 percent, while approval of U.S. leadership fell to 31 percent. This confirm‎s an earlier Pew Spring 2018 Global Attitudes Survey of 25 countries that found a confidence rating for Xi of 34 percent but only 27 percent for U.S. President Donald Trump. But can we really trust this data? The results are troubling because China, the world’s largest economy when measured by purchasing power parity, is able to control perception in its own country (18 percent of the world’s population), plus use its resources to try and control perception in the rest of the world.

Sign up to receive UCAN Daily Full Bulletin

Gallup must submit questions to Chinese authorities prior to conducting surveys. They need to be screened and approved by the State Statistics Bureau. Hansen said that while Gallup has never been asked to add questions to polls on behalf of China, “we have been directed to not ask certain questions, which we will oblige in order to conduct our work there.” Although Gallup does not self-censor its global poll or non-China questions, he added: “We do have a pretty good sense of where the line is drawn, though.” The political topics that must be avoided to get past Chinese censors are anything to do with government. “It is very clear that no questions about the perception of government are really allowed in China. No eval‎uation of institutions, leaders, leadership or corruption perception in the government,” Hansen said. Gallup dances with the censor Gallup has conducted the World Poll survey for 13 years. “We presented a full slate of questions in our first effort to poll in China with the World Poll and had many, many items quashed by the authorities,” Hansen said. “In the ensuing years, we had fewer items questioned. We always attempt to add items and include more. We know that overtly political topics will be line-itemed out of the questionnaire by the authorities. It is a bit of a dance as to how hard to push things; the risk is the inability to field at all with any questions.” Hansen likens Gallup’s role in the censorship process to a dance with the authorities rather than self-censorship. This dance allows Gallup to increase the likelihood that borderline questions will be allowed. “We do not self-censor; it is the Chinese government that has the authority to grant us permission to do our work in their country,” he said. “We are obligated to follow the process they have laid down. It is a dance every year that we have to go through. Push too hard and you risk everything. Don’t push hard enough and you are viewed as self-censoring I suppose.” Regarding the Rating World Leaders 2019 report, Hansen explained: “For this report there would have been no Chinese attitudes represented. You would have other countries’ views of Chinese leadership represented, but not Chinese attitudes.” Take survey statistics from China with a pinch of salt Even when allowed to ask questions in China, it is hard for researchers to get unbiased data due to fear of foreign and Chinese questioners and government posters explaining how to respond. Respondents who have negative opinions of the government in an authoritarian society will be less willing to participate in surveys, leading to sampling bias. A business consultant who has commissioned multiple surveys in China told ucanews.com that sample bias is a greater problem in China than in other countries. “This is a country where NBS employees face huge challenges performing something as universal and basic as a population census. The trust barrier when a stranger calls you or knocks on your door to ask you questions about your family and views on life is pretty massive, even if they produce some official-looking badge or document, given the amount of scamming that goes on,” he said. “Note that when local governments here know a survey is coming, they will also post notices around the city with guidelines for answering questions. "In pretty much all the cross-country survey data I have used, including the ones I have commissioned, Pew data, World Values Survey data, China somehow always comes out with superlative results. In many instances these results are completely in contradiction with expectations. For instance, when the country's trust deficit was a hot topic of public discussion following the Nanjing judge ruling and poor Little Yue Yue's fate, we looked up the scores on the World Values Survey trust questions. Lo and behold, China was among the most trusting societies in the world.” Polling methodology in China is hard to perfect As far as Hansen knows, Gallup is not monitored after approval of the questionnaire. But there is always the possibility that the Chinese Communist Party (CCP) is monitoring Gallup without its knowledge. Gallup does not use non-Chinese interviewers or question respondents in Tibet or Xinjiang, where there is significant instability and where it might well experience an increased level of surveillance. In those places, the government often requires approved guides and translators, and it sends government cars to tail researchers and reporters. The presence of government guides and translators must seem threatening to respondents when researchers ask sensitive political questions. In Xinjiang, foreign researchers and reporters are not allowed except on government-sponsored group tours that are highly managed. Most foreign companies operating in China have embedded CCP representatives, and it is likely that if those representatives asked a Chinese national who was a Gallup employee or respondent to do something, he or she would feel compelled to do so. According to Hansen, there was no other organization, sponsor or partnership involved in the Rating World Leaders 2019 poll as an initiator, funder or formulator. This is not the case for all Gallup question series, some of which include outside funding. He said the Chinese government does not pay Gallup for special projects. “Our work there is predominantly with the business community.” Much of Gallup’s work is self-funded through general subscriptions to universities, NGOs, governments, and even businesses. According to 2017 and 2018 Gallup methodology documents seen by ucanews.com, polling in China in 2017 was only done in Beijing, Shanghai and Guangzhou. In 2018, that expanded to the entire country apart from the politically sensitive regions of Xinjiang and Tibet. The World Poll and Rating World Leaders poll are very much worthwhile. But for the former, Chinese citizens may ironically feel compelled to answer more positively about freedom in China’s society, the less freedom there is. And those who feel most strongly about a lack of freedom in China might not want to answer at all. That would be a form of sampling bias. In the Rating World Leaders poll, the general reader should reserve judgement due to selection bias from the lack of access to Chinese citizens. How would the numbers change if Chinese citizens were included? There are several additional issues with using this data to compare global opinions of Donald Trump and Xi Jinping. If asked their opinions about Trump, internationals, U.S. Democrats and many establishment U.S. Republicans will typically answer negatively, even if they prefer, for example, Trump to Xi. A more specific comparison such as “Who do you prefer, President Trump or President Xi?” would provide a more credible estimate. This question would, of course, be impossible to get past Chinese censors. Xi has an unfair advantage globally through the CCP's use of state media to promote him. Media in democratic countries, on the other hand, typically cleave to the truth more zealously and get more clicks and subscriptions through being critical of their own country's leadership. We should expect there to be anti-leader bias in the media of democratic countries. That is great for democracy and transparency but calls into question the validity of comparisons between Trump and Xi. Despite the difficulties of polling in China, Gallup bravely soldiers on. “We proceed with ‘non-political’ questions in China for the time being. There is a great deal that we can focus on, including perceptions of life, economy, employment, health, sense of community, etc.,” Hansen said. While most statistical projects are imperfect, the Rating World Leaders poll is a truly worthwhile endeavor and will hopefully improve its access in China over time. “This is something we hope to run for 100 years,” said Hansen. “We’ve just wrapped up our 13th year of data collection, so we are off to a very good start.”