In a recent review of two books, Education and the Commercial Mindset and School Choice: The End of Public Education, which appears in the December 8, 2016 New York Review of Books, Diane Ravitch, the former Assistant Secretary of Education during the George HW Bush presidency, discusses the implications of corporate designs on public education. Ravitch begins her review by reminding us that, “Privatization means that a public service is taken over by for-profit business whose highest goal is profit.” In the name of market-driven efficiency, she argues, the “education industry” is likely to become increasingly similar to privatized hospitals and prisons. In these industries, as in many others, corporate owners, in their loyalty to investors’ desire for profits, tend to eliminate unions, reduce employee benefits, continually cut costs of operation, and orient to serving those who are least expensive to serve.
Ravitch sketches some of the challenges posed by charter schools, noting that “…they can admit the students they want, exclude those they do not want, and push out the ones who do not meet their academic or behavioral standards.” She says, that charters not only “drain away resources from public schools” but they also “leave the neediest, most expensive students to the public schools to educate.” Moreover, as Josh Moon recently noted in his article, “’School choice’ is an awful choice,” “If the “failing school” is indeed so terrible that we’re willing to reroute tax money from it to a private institution that’s not even accredited, then what makes it OK for some students to attend that failing school?”
While some argue that charter schools can “save children from failing public schools” research on student outcomes for charter school has shown mixed results. For example, The Education Law Center, in “Charter School Achievement: Hype vs Evidence” reports:
Research on charter schools paints a mixed picture. A number of recent national studies have reached the same conclusion: charter schools do not, on average, show greater levels of student achievement, typically measured by standardized test scores, than public schools, and may even perform worse.
The Center for Research on Education Outcomes (CREDO) at Stanford University found in a 2009 report that 17% of charter schools outperformed their public school equivalents, while 37% of charter schools performed worse than regular local schools, and the rest were about the same. A 2010 study by Mathematica Policy Research found that, on average, charter middle schools that held lotteries were neither more nor less successful than regular middle schools in improving student achievement, behavior, or school progress. Among the charter schools considered in the study, more had statistically significant negative effects on student achievement than statistically significant positive effects. These findings are echoed in a number of other studies.
In Michigan, Secretary of Education, Betsy Devos’s home state, 80 percent of charter schools operate as for-profit organizations. Ravitch says, “They perform no better than public schools, and according to the Detroit Free Press, they make up a publicly subsidized $1 billion per year industry with no accountability.”
Ravitch tells us that the privatization movement is largely composed of social conservatives, corporations, and business-friendly foundations. “These days, those who call themselves “education reformers” are likely to be hedge fund managers, entrepreneurs, and billionaires, not educators. The “reform” movement loudly proclaims the failure of American public education and seeks to turn public dollars over to entrepreneurs, corporate chains, mom-and-pop operations, religious organizations, and almost anyone else who wants to open a school.”
The Trump administration is likely to further advance a public-school privatization and school voucher agenda. The extent and results of such “reforms” are hard to predict. That said, as Ravitch argues “… there is no evidence for the superiority of privatization in education. Privatization divides communities and diminishes commitment to that which we call the common good. When there is a public school system, citizens are obligated to pay taxes to support the education of all children in the community, even if they have no children in the schools themselves. We invest in public education because it is an investment in the future of society.” How continued privatization of public k-12 education will affect an increasingly economically privatized and socially and politically divided society is not yet known.
One of the most important questions to consider before starting an evaluation is, “How will evaluation findings be used?” There are a number of ways the findings from an evaluation can be used. These include:
- to improve the program (formative evaluation)
- to make judgements about the ultimate value/benefits of the program to participants and stakeholders (summative evaluation)
- to sustain and/or expand the program
- to document and publicize the program’s achievements (outreach and marketing)
- to curtail the program
By being clear about the intended uses of evaluation findings, organizations can choose the most effective and stakeholder-relevant evaluation design. If, for example, a program funder wants to know whether their investment yielded desired outcomes for program participants, it may not be useful to focus the evaluation on gathering data that primarily indicate ways to improve the program’s delivery. On the other hand, if an organization is curious about how its programming is being implemented, and if such implementation is in fidelity with a research-based program design, the evaluation will want to focus more on implementation processes, rather than ultimate program outcomes.
Needless to say, stipulating the use of evaluation findings is closely related to the question of who are the major audiences for the evaluation. See our previous post, Identifying Evaluation Stakeholders. By clarifying potential audiences for findings, organizations and evaluators can identify the most useful evaluation strategy and design. In the next blog post we will discuss a related question, i.e., “What information will stakeholders find useful and valuable?”
- Who are the major stakeholders or interested parties for evaluation results?
- Who wants/needs to know?
- Who will benefit from knowing about the effectiveness of your program’s efforts?
- Make a list of key stakeholders and what they might want to know.
- Current funders
- Potential funders
- Community stakeholders
- Advocacy organizations
- Government agencies
- Colleagues/Internal organizational constituencies
- Peer organizations
- Program managers and program staff
- The “field”/general public
- Political allies
This is the first in a series of blog posts, that outlines the key elements of an evaluation.
There are many things in the world that can be evaluated. You will want to decide and define the specific object (i.e. the “evaluand”) of the evaluation. One useful approach is for program managers, and program implementers to speak with the evaluator, and together, consider these defining questions about what needs to be evaluated:
Whether you are managing a program, funding an initiative, or about to undertake a policy review, you will want to first consider:
What is the “it” that’s being evaluated?
- set of processes or relationships
Once you’ve identified what is being evaluated, it will be useful to consider these additional questions:
- What’s being done? What are the intentional actions that are being undertaken?
- Who does what to whom, and when do they do it?
- Why—for what set of reasons—are these things being done? Describe the need for the initiative or program?
- Who (which people, and which positions) are doing/carrying out the of the program, initiative, policy?
- What resources (i.e. inputs) are involved? (not just money and time, but knowledge, cooperation of others, networks of collaborators, etc.)
- Who (or what) are implementers working to change? What specifically is supposed to change, or be different as a result of the program or initiative doing what it does?
- Describe who benefits (e.g. program participants, beneficiaries, consumers, etc.) and what the benefits are.
By clearly identifying a determinate entity (i.e., “evaluand”) and the effects (i.e., outcomes) of the operation or implementation of that entity, you will help to ensure a successful evaluation. The clearer you can be in describing what the program, initiative, or policy is, the more effective the evaluation will be. Answering the above questions and sketching out a logic model of the program’s operation (which we’ll discuss in a subsequent blog post), will ensure an effective, accurate, and ultimately highly useful evaluation.
Also, for purposes of clarity, it may be useful for program managers and implementers to consider and discuss with the evaluator what isn’t being evaluated. This conversation can help to define the necessary boundaries around the program and will help to prevent evaluators expending unnecessary and potentially costly efforts to design evaluations of things that exist outside the boundary of the initiative, program or policy.
What is ‘Organization Development’?
Organization Development is an intentional set of processes and practices designed to enhance the ability of organizations to meet their goals. It entails “…a process of continuous diagnosis, action planning, implementation and evaluation, with the goal of transferring (or generating) knowledge and skills so that organizations can improve their capacity for solving-problems and managing future change.”(See: Organizational Development Theory) Organization Development deals with a range of features, including organizational climate, organizational culture (i.e., assumptions, values, norms/expectations, patterns of behavior) and organizational strategy. It seeks to strengthen and enhance the long-term “health” and performance of an organization, often by focusing on aligning organizations with their rapidly changing and complex environments through organizational learning, knowledge management and transformation of organizational norms and values.
What is an organization?
At the most abstract level, an organization is a collectivity of people, a social entity, which seeks to achieve specific aims and goals, and typically is characterized by a structure of designated roles, established rules, a system or structure of authority, a process for decision-making, and a division of labor among organizational members. Organizations are composed of a discreet “membership,” that is, a limited population of incumbents or personnel. Organizations exist in, affect, and are affected by, the larger social and economic environment. Formal, large-scale organizations may take the form of businesses, schools, the military, churches, prisons, foundations, and non-profits.
The Relationship Between Organizations and Evaluation
In seeking to achieve goals, organizations often design and implement discreet initiatives, policies, and programs. They mobilize resources to achieve specific ends. Non-profit organizations, for example, implement programs that mobilize resources in the form of activities, services, and products that are intended to improve the lives of program participants/recipients. “A program is a collection of resources in an organization and is geared to accomplish a certain goal or set of goals. Programs are one major aspect of the non-profit’s structure. The typical non-profit organizational structure is built around programs, that is, the non-profit provides certain major services, each of which is usually formalized into a program.”(See: Overview of Non-Profit Program Planning). In serving program participants, nonprofits strive to effectively and efficiently deploy program resources, including knowledge, activities, services, and materials, to positively affect the lives of those they serve.
Although evaluations are customarily aimed at gathering and analyzing data about discrete programs, the most useful evaluations often collect, synthesize, and report information that can be used to improve the broader operation and health of the organization that hosts the program. Program evaluation thus can contribute to organization development, the deliberately planned, organization-wide effort to increase an organization’s effectiveness and/or efficiency and to enable the organization to achieve its strategic goals.
Brad Rose Consulting Aids Organizational Development
Brad Rose Consulting works at the intersection of evaluation and organization development. While our projects often begin with a focus on discrete initiatives and programs, the questions that drive out evaluation research provide insights into the effectiveness of the organizations that host, design, and fund those programs. Findings from our evaluations often have important implications for the development and sustainability of the entire host organization. This is especially true in the case of small-to-medium sized nonprofit organizations and educational organizations, whose core programs often comprise the bulk of the organization’s structure and raison d’être. Information from our evaluations can be used to clarify the organization’s goals and objectives, to identify key organizational challenges and help to develop ways to address these, and to strengthen the overall effectiveness of the organization’s efforts. Additionally, Brad Rose Consulting’s evaluations offer an ideal opportunity for an organization to reflect on its practices and purposes, to rethink ways to achieve the organization’s mission, and to identify new data-based strategies for enhancing the organization’s long-term viability and well-being.
See our previous post: Helpful Resources: Program Evaluation Supports Strategic Planning
Overview of Nonprofit Program Planning by Carter McNamara