In a recent article “Against Metrics: How Measuring Performance by Numbers Backfires,” Jerry Z Muller argues that companies, educational institutions, government agencies, and philanthropies are now in the grip of what he calls “metric fixation,” “…the belief that it is possible – and desirable – to replace professional judgment (acquired through personal experience and talent) with numerical indicators of comparative performance based upon standardized data (metrics).”
In this brief and important article, Muller critiques the growing phenomenon of paying employees for performance. He points out that such schemes often lead to a narrowing measure of what is desirable for the organization, leads members of an organization to “game the system”, often undermines organizations ability to think more broadly about their purposes, and most importantly, impedes innovation.
Looking at the unintended outcomes of metric fixation, he writes:
“When reward is tied to measured performance, metric fixation invites just this sort of gaming. But metric fixation also leads to a variety of more subtle unintended negative consequences. These include goal displacement, which comes in many varieties: when performance is judged by a few measures, and the stakes are high (keeping one’s job, getting a pay rise or raising the stock price at the time that stock options are vested), people focus on satisfying those measures – often at the expense of other, more important organizational goals that are not measured. The best-known example is ‘teaching to the test’, a widespread phenomenon that has distorted primary and secondary education in the United States since the adoption of the No Child Left Behind Act of 2001.”
Pay for performance schemes, however, are not alone in eliciting a narrowing of goals, and a tendency to game the system. Metric fixation (or what I term the “tyranny of measurement”) can be a risk for a range of non-profit organizations and educational institutions who often feel that demands for accountability can be addressed by merely counting the number of participants who receive services, or the number of students who score well on reading tests. While it is important to have clear goals, and to be able to indicate if these goals are met, organizations, in their rush to address demands from funders and other stakeholders for accountability, must be careful not to reduce their goals—indeed their organizations’ vision— to only a few countable variables. “What can and does get measured is not always worth measuring, may not be what we really want to know, and may draw effort away from the things we care about” (Muller). As Albert Einstein observed, “Not everything that counts can be counted, and not everything that can be counted, counts.”
“Against Metrics: How Measuring Performance by Numbers Backfires,” Aeon, April 24, 2018
The Tyranny of Metrics, Jerry Z. Muller, Princeton University Press, 2018
Brad recently presented an introductory seminar on program evaluation at Nonprofit Net, a Lexington, Massachusetts-based organization. Nonprofit Net is a forum for Massachusetts nonprofit leaders and nonprofit consultants which offers seminars on topics of importance to the nonprofit community. Brad’s presentation provided an introduction to program evaluation, outlined the benefits of evaluating outcomes in order to demonstrate programs’ achievements and challenges, introduced the use of logic models, and reviewed the key questions that nonprofit leaders should consider as they approach a program evaluation.
The seminar was based on the materials in the following white papers, which you can download for free:
Recent revelations about Facebook and Cambridge Analytica’s use of personal data have raised serious concerns about internet privacy. It would appear that we inhabit a world in which privacy is increasingly under assault—not just from leering governments, but also from panoptic corporations.
Although the right to privacy in the US is not explicitly protected by the Constitution, constitutional amendments and case law have provided some protections to what has become a foundational assumption of American citizens. The “right to privacy” (what Supreme Court Justice Louis Brandis once called “the right to be left alone,”) is a widely held value, both in the U.S. and throughout the world. But why is privacy important?
In “Ten Reasons Why Privacy Matters” Danial Solove, Professor of Law at George Washington University Law School, lists ten important reasons including: limiting the power of government and corporations over individuals; the need to establish important social boundaries; creating trust; and as a precondition for freedom of speech and thought. Solove also notes, “Privacy enables people to manage their reputations. How we are judged by others affects our opportunities, friendships, and overall well-being.”
Julie E. Cohen, of Georgetown argues that privacy is not just a protection, but an irreducible environment in which individuals are free to develop who they are and who they will be. “Privacy is shorthand for breathing room to engage in the process of … self-development. What Cohen means is that since life and contexts are always changing, privacy cannot be reductively conceived as one specific type of thing. It is better understood as an important buffer that gives us space to develop an identity that is somewhat separate from the surveillance, judgment, and values of our society and culture.” (See “Why Does Privacy Matter? One Scholar’s Answer” Jathan Sadowski, The Atlantic, Feb 26) In the Harvard Law Review, (“What Privacy Is For” Julie E. Cohen, Harvard Law Review, Vol. 126, 2013) Cohen writes, “Privacy shelters dynamic, emergent subjectivity from the efforts of commercial and government actors to render individuals and communities fixed, transparent, and predictable. It protects the situated practices of boundary management through which self-definition and the capacity for self-reflection develop.”
Cohen’s argument that privacy is a pre-condition for the development of an autonomous and thriving self is a critical and often overlooked point. If individuals are to develop, individuate, and thrive, they need room to do so, without interference or unwanted surveillance. Such conditions are also necessary for the maintenance of individual freedom vs. slavery. As Orlando Patterson argued in his book, Freedom Vol.1 Freedom in the Making of Western Culture (Basic Books, 1991) freedom historically developed in the West as a long struggle against chattel slavery. Slavery, of course, entails the subjugation of the individual/person, and depends upon the thwarting of autonomy. While slavery may not fully eradicate the full and healthy development of the “self,” it may deform and distort that development. Autonomous selves are both the product of and the condition of social freedom.
Privacy, which is crucial to the development of a person’s autonomy and subjectivity, when reduced by surveillance or restrictive interference—either by governments or corporations who gather and sell our private information—may interfere not just with social and political freedom, but with the development and sustenance of the self. “Transparency” (especially when applied to personal information) may seem like an important feature to those who gather “Big Data,” but it may also represent an intrusion and an attempt to whittle away the environment of privacy that the self depends upon for its full and healthy development. As Cohen observes, “Efforts to repackage pervasive surveillance as innovation — under the moniker “Big Data” — are better understood as efforts to enshrine the methods and values of the modulated society at the heart of our system of knowledge production. In short, privacy incursions harm individuals, but not only individuals. Privacy incursions in the name of progress, innovation, and ordered liberty jeopardize the continuing vitality of the political and intellectual culture that we say we value.” (See “What Privacy Is For” Julie E. Cohen, Harvard Law Review, Vol. 126, 2013)
Privacy is not just important to the protection of individuals from governments and commercial interests, it is also essential for the development of full, autonomous, and healthy selves.
“Ten Reasons Why Privacy Matters” Daniel Solove
“Why Does Privacy Matter? One Scholar’s Answer” Jathan Sadowski, The Atlantic, Feb 26, 2013
“What Privacy Is For” Julie E. Cohen, Harvard Law Review, Vol. 126, 2013
Orlando Patterson argued in his book, Freedom Vol.1 Freedom in the Making of Western Culture (Basic Books, 1991)
“Facebook and Cambridge Analytica, What You Need to Know as Fallout Widens” Kevin Granville, New York Times, Mar 19, 2018
“I Downloaded the Information that Facebook Has on Me. Yikes” Brian Chen, New York Times, Apr 11, 2018
“Right to Privacy: Constitutional Rights & Privacy Laws” Tim Sharp, Livescience, June 12, 2013
Surveillance Capitalism, Shoshone Zuboff
Listen to “Facebook and the Reign of Surveillance Capitalism” Radio Open Source
Read a review of Surveillance Capitalism
“How to Save Your Privacy from the Internet’s Clutches” Natasha Lomas, Romain Dillet, TC, Apr 14, 2018
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are, Seth Stephens-Davidowitz, Dey St., 2017
Rationalization and Bureaucracy
In the early years of the 20th century, the German sociologist Max Weber argued that modern society increasingly relies upon the “rationalization” of social organizations and institutions. He maintained that Western society is increasingly reliant upon reason, efficiency, predictability, and means/ends calculation. He further believed that modern society is highly dependent upon both public and private bureaucracies (e.g., the nation state and the modern corporation) as a way to achieve important societal goals (education, social welfare, medical care, business administration, governance, etc.) Bureaucracies are, “Highly organized networks of hierarchy and command structures (which are) necessary to run any ordered society – especially ones large in scope.” (See “Max Weber’s Theory of Rationalization: What it Can Tell Us of Modernity,” ) As one form of social organization, bureaucracy is distinguished by its: (1) clear hierarchy of authority, (2) rigid division of labor, (3) written and inflexible rules, regulations, and procedures, and (4) impersonal relationships. (For this and additional definitions, see the BusinessDictionary)
Weber and subsequent social theorists saw the process of rationalization and bureaucratization as replacing traditional modes of life, traditional values, and religious orientations with a society characterized by growing calculability, pursuit of individuals’ self-interest, efficiency, and ordered control. (Weber termed the loss of tradition that accompanied the increasing rationalization of Western society as the “disenchantment” of society.) As modernity transforms traditional social forms and social values, rationality and bureaucracy come to dominate the various spheres of contemporary society. Moreover, as society becomes ever more rationalized, it increasingly depends upon bureaucratic regimes of governance and management by impersonal rules and the exercise of technical knowledge by experts. Today, various social arenas—ranging from government to corporate organizations, from healthcare to public education—have become suffused with the ethos of bureaucracy and rationality. “…rationalization means a historical drive towards a world in which ‘one can, in principle, master all things by calculation.’” (See, Max Weber, in Stanford Encyclopedia of Philosophy)
Advantages and Disadvantages of Bureaucracy
Although “bureaucracy’ is often thought of as a pejorative term, bureaucracy has some advantages over other forms of social organization. Bureaucracy creates and utilizes rules and laws (vs. fiat decisions by a powerful notable, such as a king), mobilizes the knowledge of educated experts, promotes meritocracy, delineates and sets boundaries for the exercise of social power, establishes a formal chain of command and specifies organizational authority, and provides a technically efficient form of organization for dealing with routine matters that concern large numbers of persons. These advantages however, are accompanied by what are thought by many to be substantial disadvantages, including, compelling officials to conform to fixed rules and detailed procedures, sponsoring bureaucrats’ focus on narrow objectives, and supporting bureaucrats to become defensive, rigid, and unresponsive to the urgent individual needs and concerns of private citizens. “…individual officials working under bureaucratic incentive systems frequently find it to be in their own best interests to adhere rigidly to internal rules and formalities in a ritualistic fashion, behaving as if “proper procedure” were more important than the larger goals for serving their clients or the general public that they are supposedly designed to accomplish (i.e., the “red tape” phenomenon).” (See Bureaucracy)
Education and Bureaucracy
If we look at public education in contemporary society, we see many features associated with bureaucracy. State education agencies, districts, and schools: 1) are run by trained experts (e.g., credentialed teachers and administrators), 2) feature rigid hierarchies of authority, 3) have a strict division of labor, 4) depend upon and are run by formal and impersonal rules of administration and control, and 5) credential students by relying on impersonal and standardized methods for assessing student achievement. Additionally, students are routinely segregated into age-specified categories (classes) and are subjected not to individually tailored curricula, but to routine and standardized curricula that attempt to teach students en masse.
Does Standardized Testing Support Educational Bureaucracy?
Standardized testing of student achievement is one of the bureaucratic characteristics of modern public education. Although assessment is thought to be a necessary means for measuring student learning, it is also a means by which educational organizations categorize students, assign them social statues, and allocate them to various social trajectories (“life chances” to use Weber’s terminology). Standardized testing regimes also assist the educational bureaucracy by creating different categories of clientele (i.e., students) who can then be served en masse by large-scale routinized educational programs and mass-produced textbooks. Some would even argue that students are made to fit school as much as schools are made to fit the student. (For a summary of the problems associated with standardized testing, see “What’s Wrong with Standardized Tests.”)
While students are subject to the rule of bureaucracy, so too are faculty and administrators. Like the students they teach and oversee, faculty and administrators are subject to formal structures of authority, adhere to a strict division of labor, follow formal rules and regulations, and must be credentialed and certified. Like their students, teachers are also subject to assessment and review. (See our previous blog post “Too Much Assessment in Higher Education,” for an example of the effects of assessment on higher education.) Schools are also reviewed and rated by State Departments of Education.
While some feel that the stultifying aspects of bureaucracy may be ameliorated, the original theorist of rationalization and bureaucracy, Max Weber, was pessimistic about the reform of bureaucracy. As he surveyed the early 20th century and considered the likely developmental direction of Western society, he said that citizens of society were likely to find themselves increasingly entrapped in what he termed the “iron cage of bondage,” which continued to be cemented by the growth of rationalization and bureaucracy. Whether this dark prognosis is generally true for Western society is still very much debatable. That said, it is difficult to imagine large-scale public education without many of the features of bureaucracy that Weber first described —including standardized student testing. (For examples of reform efforts as they apply to standardized tests, see The National Center for Fair & Open Testing)
“What’s Wrong with Standardized Tests,” The National Center for Fair & Open Testing