Big Data and Evaluation

In the latest issue of the American Journal of Evaluation (Vol 41, No.2, June 2020) Robert Picciotto argues that the time has arrived for evaluation to make use of Big Data. In “Evaluation and the Big Data Challenge,” Picciotto observes that evaluators, although not yet well versed in Big Data’s use, should nonetheless actively engage Big Data, which is data composed of large data sets that are too large and complex for traditional data processing tools and that imply just-in-time information for decision-making, continuous storage and processing, and the extensive use of algorithms. The scale and seeming availability of Big Data, coupled with exponentially growing computer power, Picciotto tells us, increasingly makes the use of such data more attractive for evaluators. Big Data makes it possible for evaluators to identify patterns and to gain insights that arise from large data sets, insights that can’t be secured though limited and costly access to traditional data. Additionally, Big Data, if handled correctly, may improve the quality of evaluation. It may even allow evaluators to more effectively wrestle with the persistently thorny problem of discerning causality in complex social systems.

While Big Data offers a range of new opportunities to evaluators, Big Data (and big tech that privately owns and deploys this data) is not without its challenges and drawbacks. Governments, corporations, and interest groups are increasingly reliant on Big Data to manipulate public opinion, shape consumer behavior though predatory advertising, and in some cases to manipulatively intervene in the civic and political lives of nations. (See our previous article, “Everybody Lies”) Picciotto acknowledges the often pernicious uses of private data, and points to the lamentably under-regulated use of Big Data to monitor and influence the behavior of citizens and consumers. He also notes that the algorithms now used to analyze the volumes of data are neither objective nor universally accurate. Despite these substantial challenges, Picciotto—somewhat sanguinely I think—believes that evaluators and greater governmental regulation of big tech may ameliorate some of the more egregious dimensions and uses of Big Data. “Big Data has let lose a host of social threats: oppressive surveillance, loss of privacy, reduced autonomy, digital addiction, spread of disinformation, social polarization, and so on.” Whether evaluators and the public are capable of taming an enterprise that is now overarchingly global, under-regulated, ethically questionable, and resistant to national constraints, will need to be seen.

Resources: