Kirby Farrell Ph.D.

A Swim in Denial

Judging Your Worth

1,984 points of light

Posted Jun 12, 2017

China is developing a digital system to track and evaluate its population of 1.3 billion people. The system excites comparisons to Orwell’s 1984 and dystopian films, but it is only one use of big data to manage people.  Kai Strittmatter, who has reported on Chinese culture for the German newspaper Sueddeutsche Zeitung, summarizes the new program in “Punkteregime” or “Points Regime” (May 19, 2017). 
Some apps for the system are already in trials. When you download "Honest Shanghai," the app scans your face as you register, and retrieves data about you from the Internet. Like a credit rating in the U.S., the app uses algorithms to evaluate your financial transactions (bills paid on time?) and rank your creditworthiness.

By 2020, the system is planned to include all Chinese in a "system for social trustworthiness." The idea is not only to facilitate more, and more secure, business transactions but also to improve individual behavior.  As in Orwell’s 1984, big data, social media, and a digital point system will use rewards and disincentives or outright punishments to create a new model person. An official in the town of Rongchen declares “We want to civilize people."

Zhang Zheng, Dean of Faculty of Economics at Beijing University, explains, "How do you treat your parents and your spouse, all your social actions, whether and how you comply with moral rules—does not that also tell you about your trustworthiness?"

According to the Director of the pilot project in Rongchen, the system will rank every company and citizen in China. In the pilot project everyone starts with 1000 points. Approved behavior improves your score. You “can be an AAA citizen ("model of honesty", more than 1050 points). But a slip to 849 points is the "warning level." Below 599 points, rated "dishonest," your name will be blacklisted, published, and you become the "object of significant monitoring." This is specified in the Rongcheng official handbook of the "Administrative Measures for the Reliability of Natural Persons.”

You can see some deep metaphors in the system. It resembles games based on scoring, combined with the standardized processes of a factory. Measuring worth by scores evokes trade and business, especially bookkeeping. Like most computer technology, it makes a trait theory and a decision tree more important than inner life.  

Enthusiasts make the system sound wholesome as a TV game show, with built-in safeguards for flexibility and fairness.  But if history is any guide, the Communist Party and big business will prefer a muscular system that enhances social control. It remains to be seen if any design can rule out incompetence or  corruption. 

In some ways the scheme echoes the corporate promotion of privileges in the U.S., where a certain level of spending or customer “loyalty” qualifies you for special treatment. But when Chinese dissenters disagree with the Party, they’re not usually disappeared into an airline’s Elite Club lounge.

The enthusiasts avoid the specter of punishment by suggesting that negative ratings might reshape the citizen by limiting social privileges, such as access to library books or travel. But no matter how gentle the euphemisms, influence over others is bound to have a coercive element.  Even utopia needs the protections of law and due process.

The dream of the New Man was the 20th century nightmare of totalitarianism.  Who will control the controllers? Who will police the police? Who will sort out the confusion of business practice with governance? The goal is to spur self-policing while disguising the controller.

In the U.S., as advertising and recent election cycles have shown, miners of big data envision algorithms that can predict people’s choices. The dream is that given enough information, a program will be able to tease out and control the consumer or voter’s intuitive and still-unconscious preferences.

Social media such as Facebook and academic endeavors such as The World Well-Being Project and myPersonality promise to enhance individual freedom. They assume that psychological machinery can actualize authentic values otherwise merely latent in us. But in all such efforts to help the butterfly out of the cocoon, the tools and assumptions of the project color the butterfly. 

And of course some butterfly hunters are frankly interested in perfecting the tools for sale to the highest bidder. In a YouTube presentation, the CEO of Cambridge Analytica claims to have combined personality test responses with data from social media to produce “psychographic profiles.” Supposedly his “models that predict personality traits for every adult in America” played a role in the last election.  But “it is important to remember that this much-discussed video is a sales pitch.” [1]  

Despite different emphases, Chinese and American interest in social control overlap.  As shown by the new hysteria about illegal immigrants and terrorism, and massive government investment in surveillance, the U.S. shares the Chinese anxiety that the scale of life exceeds traditional constraints.  U.S. immigration officials are combing records looking for even minor infractions that could justify expulsion.

 At  the same time both countries nurture ambitions that look for a payoff from new tools of control.  Some of the tools are crude propaganda such as the ballyhooed Mexican wall, but others are exploring the depths of electronic data technology and human nature. Why the hysteria? For the moment the scale of life has reached a tipping point. Big numbers challenge the brain, whether they’re population, trade, or environmental figures. And competition makes high-strung humans nervous, since the deep metaphor is combat.  You see the hysteria in the hoarding of power and money at the top, a gun under every pillow, and shameful attacks on the working poor and labor law.

The Chinese have a thousand-point surveillance system. The US has the smarmy slogan “a thousand points of light.” Both cultures are trying to devise narratives that control rambunctious reality without leaving unsightly scars. It’s an old project. Let’s see how it works out this time.

In his Notes from Underground, Dostoyevsky vowed that some humans are defiantly perverse and therefore will be defiantly free. Skeptics anticipate that some Chinese will find ways around the ratings system and its likely corruptions. We are social animals, but also competitive and devious creatures.  The same mentality that enables traders to intuit what others value may also be able to imagine what fools them.  As we see around us today, we can deplore deception even as the crowd is applauding a hoodwinking magic show.

Resources used in this essay:

1. Tamsin Shaw, "Invisible Manipulators of Your Mind," New York Review of Books  (April 20, 2017), 64.