About us    Campaigns    Research    Support us    Publications    Media Room    Join Us    Contact us

Rating colleges is ‘like rating a blender’ – Education Department official

Education Loans, Outcomes


Washington Post

This is what  Jamienne Studley, a deputy under secretary at the Education Department, told a group of college presidents who were meeting to talk about President’s Obama’s plan to rate colleges with the apparent aim of driving out of business schools that don’t meet the administration’s definition of success, as reported by The New York Times:

“It’s like rating a blender. This is not so hard to get your mind around.”

And this is what Cecilia Muñoz, director of the White House Domestic Policy Council, said in the same article about whether it is possible for anybody to persuade the administration that their plan is a terrible one for many reasons, including the fact that rating a college is not really like rating a blender:

For those who are making the argument that we shouldn’t do this, I think those folks could fairly have the impression that we’re not listening. There is an element to this conversation which is, “We hope to God you don’t do this.” Our answer to that is: “This is happening.”

And there you have it. It doesn’t seem to matter what anybody else thinks. Though there are many definitions of success, the Obama administration is going to use its own to develop the rating system no matter how many people oppose it. They know better. Just ask them.

In this case, we are talking about a plan  to rate (not rank) colleges on criteria that could include average tuition and how much graduates earn even though many higher education leaders have said it is a terrible idea.

The administration says it will rate colleges by “mission” as well as institutional type, and wants to link federal student aid to the rankings, giving more to schools that score highly, and thus ultimately driving out schools that do poorly on the ratings system.  (The federal student aid piece involves congressional approval, which isn’t likely.) How much it is going to cost? Not known.

The administration thinks this will serve students well by revealing important data to families so they can better make college decisions. Critics say that all rating systems present a limited view of any institution and that the government already publishes a mountain of information on institutions of higher education. (See below for other problems with the plan.)

One of the critics is Janet Napolitano, president of the University of California system who had been Obama’s U.S. homeland security secretary; she said last December that she is “deeply skeptical that there are criteria that can be developed that are in the end meaningful.” The administration, apparently, doesn’t care much what its own former Cabinet member has to say.

This may seem painfully obvious but, for the record:  Blenders mix things together. That’s it. They may do it on different speeds, but mixing things is what they do. Colleges do countless things for students, and people go to them for many different reasons, with many different goals. The administration’s focus seems to be on financial rewards after college, but that’s not why everybody goes.

Yes, some students want to go to Wall Street and make a fortune. But some want to go to college to become teachers and not make a fortune. Some students want to be poets, engineers, sociologists, urban planners, nurses, etc. Some go for a religious education. Some go without knowing what they want to be but want to expand their understanding of the world and develop analytical thinking, which, incidentally, can be done in just about any area, not simply the sciences but also philosophy and music and the whole range of humanities.

Schools are highly complicated institutions with countless moving parts. Unlike a blender.

Besides, there are a host of problems associated with the plan.  The Education Department asked for public comments about its plan, and the National Association for College Admissions Counseling responded with some of the most interesting. I’ve published some of these comments before, but here they are again:

*Ratings and rankings can be skewed by the methodology used to create them.

* The federal government has major constraints in its ability to oversee data submission from colleges and “as a result, it may take years before institutions are held accountable for violating program integrity standards, including reporting false data to the Department of Education.”

At a minimum, a college ratings system in the current environment of program integrity enforcement would suffer from inaccurate and potentially misleading information if unscrupulous institutions are able to avoid accountability for reporting inaccurate information. At worst, decisions about the allocation of federal student aid will be made on information that has been manipulated to ensure continued eligibility for federal student aid programs, with little or significantly delayed corrective action.

* A rating system could create incentives for schools “to focus disproportionate resources on data elements that can change rankings without necessarily changing the quality of the institution.

* It is “virtually impossible to develop a ratings system that includes affordability as an input variable without also making an evaluation of the state funding mechanisms for higher education.”

* The administration suggested that  colleges and universities would be classified in its ratings system by “mission” as well as institutional type, but schools “differ widely within some of these categories.”

* Using as a data point the number of students from low-income families with Pell grants is problematic because “some institutions that enroll the largest number of Pell grants are also the institutions with the worst track record for serving students.”

Comments Off on Rating colleges is ‘like rating a blender’ – Education Department official

A defense of public education against ‘the wolves of Wall Street’

Charter Schools, Outcomes, Quality


Washington Post

As a lifelong educator who has worked 10 years in a Catholic high school and now 11 years as a public middle school librarian, I am highly invested in the current conversation surrounding public education and reform. Some of you have stopped reading, assuming I’m a union teacher with tenure. Disclaimer: I work in Texas, a right to work state, where we have no tenure or unions with collective bargaining rights. Still, my experience demands that I defend public education, which is often under assault.

I am disturbed when high-flying charter schools, such as Harlem Success Academy, brag about their student standardized test scores, not because I begrudge them but because they seem blithely unaware of selection bias. Just the very fact that a parent takes the initiative to apply to one of these schools makes a huge difference. In addition, both students and their parents have to sign contracts and agree to longer hours and high performance standards. If students do not live up to these standards, they are no longer able to attend. A fair comparison between test scores of high-profile charter students and regular public school children must include only the public school students who attend regularly, do their homework, and, along with their parents, are committed to school. Or compare them to magnet schools, which is what they essentially are.

Editorial pages of many newspapers often bemoan the expense of public schools. Yes, since the 1970s, we are spending more per student. Part of this is due to inflation (try comparing house values in the 1970′s to now), but part is also due to the 1975 Individuals with Disabilities Education Act (IDEA). This law has enabled so many children to access services, including, when necessary, one-on-one aides, who guide them throughout the school day. These services are absolutely the right thing to do, but they are expensive.

Charter schools have fewer children with disabilities as well as fewer English Language Learners. Studies cite a 3-4 percent gap in special education students at charter vs. regular public schools. But as someone who works in a public school with a Life Skills unit, I wondered: where are the Life Skills units at high profile charter schools? They may have special education students, but do they have middle school students with a mental age of 1 1/2 years old, colostomy bags and diapers, students with multiple and severe disabilities who are served in their own classrooms? No. For example, North Star Academy Charter schools in Newark have 36 percent fewer students with severe (high cost) disabilities than the Newark public schools in general. While in New York City, 41 percent of public school students speak a language other than English at home, only 5 percent do so at Harlem Success Academy.

Furthermore, the greatest expense for public schools is personnel. It seems that some of the recent animosity toward teachers is due to the simple fact that our salaries are paid from tax revenue. As Rupert Murdoch has remarked, public education is a $500 billion business, and some see that public money as a private business opportunity.

People ask: what’s wrong with “choice” and vouchers? First of all, vouchers transfer public money to private institutions. Public schools lose the annual, per dollar amount for each child who leaves and takes the money with her. If choice in the form of charters and vouchers continues to siphon involved families and students from public schools, then public schools will become dumping grounds for our children with the greatest material, physical, language, and emotional needs. We become a nation in which some children win while others lose. While that may be the way of the wolves on Wall Street, we public school teachers will not abandon the lambs in our charge. Rather than working to improve schools for all, market practices make it acceptable to continue leaving children behind.

In closing, great schools have these things in common: a strong principal who is an instructional leader, committed and involved families, adequate funding, and committed, prepared, well-compensated teachers. According to the latest Texas Tribune poll, even 65 percent of Republicans believe teachers should be paid more. Colleges and universities need to make teacher programs more competitive. We must raise the bar on getting a teaching certificate, and we must get rid of fly-by-night, six-week training programs.

We must partner with parents. The secret to success in high-achieving charter schools and private schools is parental commitment to the shared goal of educating children. Amanda Ripley, in her excellent book, The Smartest Kids in the World and How They Got That Way, cites research demonstrating that if parents do two things, their children will most likely succeed academically: 1) Read to them from a young age. 2) Model reading themselves.

We need to follow the common sense wisdom of Stephen Krashen’s “The Power of Reading.” Give students time to read in school, allow them to self-select their books, and provide them access to these books, and children will read. By reading, they practice and improve their literacy. Not only do readers do well on standardized tests, but according to research cited in “Reading in the Wild” by teacher Donalyn Miller, readers vote more, volunteer more, and are better informed citizens. Sadly, instead, many school districts are cutting librarians and library programs in order to spend more on testing.

We need to better support our children in the United States. Too many children live in poverty: 23 percent. In Texas, 60 percent of our public school students qualify for free or reduced lunch. Teachers will tell you of students living in cars, moving every other month because parents can’t pay the rent, coming to kindergarten not knowing their colors. Universal Pre-K will absolutely make a difference.

There are many things we, as a society, can do to improve our schools, but the current “reform” policies — which stress punitive testing, demonizing teachers as lazy leeches, advocating performance pay, and privatizing public schools — are taking us further from the noble traditions which made us great. Our nation was the first in the world to provide free, compulsory education. This mission is inscribed in most of our state constitutions. Our public schools are the centers of our neighborhoods and communities. They are run by democratically elected school board members. Public education as an institution needs to be nurtured and cherished for the common good.


Comments Off on A defense of public education against ‘the wolves of Wall Street’

School Choice, School Quality and Postsecondary Attainment

Outcomes, Quality, School Choice

David J. Deming, Justine S. Hastings, Thomas J. Kane, Douglas O. Staiger

National Bureau of Economic Research Working Paper Series

Working Paper 17438

September 2011

Abstract: We study the impact of a public school choice lottery in Charlotte-Mecklenburg schools on college enrollment and degree completion. We find a significant overall increase in college attainment among lottery winners who attend their first choice school. Using rich administrative data on peers, teachers, course offerings and other inputs, we show that the impacts of choice are strongly predicted by gains on several measures of school quality. Gains in attainment are concentrated among girls. Girls respond to attending a better school with higher grades and increases in college-preparatory course-taking, while boys do not.

The working paper can be accessed here.

Comments Off on School Choice, School Quality and Postsecondary Attainment

Subir Shukla: Does measuring outcomes improve accountability in education?

Education, Outcomes


Business Standard

In the last few years, the clamour for measuring learning outcomes and using that as a means to ensure accountability has grown louder. In fact the current Five-Year Plan insists that learning outcomes be measurable and be measured. Corporate houses funding various foundations and NGOs are big on learning assessment and look to it as a means of bringing about improvement. Many sensible people are voicing views to the effect that if a teacher is unable to generate learning outcomes, he/she should be shoved aside and replaced by someone better. And, of course, the feeling persists that we are not measuring the quality of learning enough.This is unfortunate. Not because measuring outcomes is not important or somehow wrong but because the present formulations of the issue are simplistic to the extent that they prevent underlying issues to be addressed. Here is how.

First, it is not as if the quality of learning is not being measured, or has not been measured in the last twenty years. The first all-India survey of learning levels was conducted by the NCERT in 1995, and there have been many since. Several large-scale independent studies of students’ learning levels have been run, including ASER and surveys of Education Initiatives. Small-scale learning assessments have been conducted for innumerable research studies (e.g. of 1 lakh children in Tamil Nadu to assess the state’s Activity-Based Learning Programme) or pilot projects (for instance, several states have piloted their textbooks and used learning achievement as a benchmark). And of course at least hundreds (if not thousands) of NGOs/NGO-run programmes (often in government schools) have incorporated assessment as an effectiveness measure.

There are, thus, any number of assessments available –and they’ve been telling us for the last twenty years that our children are not learning. Only, this doesn’t seem to have resulted in improved learning, thus questioning the assumption behind the clamour for measurement.

This is a little like weighing a child to assess the level of nutrition – unfortunately, merely weighing the child will not lead to better nutrition… Something else is clearly required, and that doesn’t seem to be happening.

Second, insisting on having ‘measurable’ outcomes is hugely misleading – the mere fact that you can measure something doesn’t make it more worthwhile (e.g. we do want students to be creative or considerate or civic, though there are no easy measures for these). Several of the assessments mentioned suffer from this. Thus an Adivasi child who displays great resourcefulness, knowledge of the environment and concern for others would be called poorly educated since the ‘tests’ measure only basic literacy and numeracy.

Measuring outcomes would be useful only when we measure what matters most to us. Not whether a child can read something aloud but whether he can form an opinion on it and give the reasons behind them. Not whether a child can do calculations but whether she can apply it in real-world contexts to solve problems or take a decision. Some of these may be hard to measure, but it would be useful to remember that it is not the purpose of education to be assessable, but, rather, the purpose of assessment is to measure what is considered most worth learning.

Third, measuring outcomes does not account for contexts and tends to disadvantage (and label) those facing adverse conditions,  which then makes it even more difficult for them to improve. There are many teachers who work very hard in difficult conditions – but don’t attain the kind of outcomes expected because the curriculum assumes children will be able to attend daily or speak the school language at home (and several other such notions), which don’t apply to the children they work with (some 60-70% in India). We’ll end up shoving these teachers out if we take the advice to replace them – instead of overhauling the system which has designed itself in such a way that marginalized children WILL fail.

Fourth, there is a danger that the present focus on outcomes is actually obfuscating – instead of increasing – accountability. India’s challenges now arise from its success in rapidly expanding the school system to bring in so many children. The consequence is that we now have students (at all levels) who traditionally never attended schools – working children, migrant groups, girls from various communities, children with disabilities, socially excluded communities…. the list is endless. What this means is that while the nature of our students has changed, the curriculum, pedagogy and assessment remain as they used to be and so, the DESIGN ITSELF leaves these learners out.

At a second level, when it comes to implementation, there is a tendency in those responsible to ignore laxity on the assumed ground that it is only happening to those who do not matter. (Just as it is easier to ask a poor person to push a stalled car rather than a well-dressed one, similar prejudices operate in all walks of our society, including government officials.) Even now, therefore, it is mainly those from better-resourced families who continue to succeed, and we continue to have poor education for the poor. So the accountability really needs to be demanded at the level of the system (NCERT, MHRD, Departments of Education) and state / district / block officials.

As long as people keep pointing fingers at teachers as the main villains, those  really responsible will continue to escape accountability. For instance, when the NCERT’s own national survey shows low levels of learning, why does nothing happen to anyone at any level, including the NCERT itself (whose curriculum has been taken by many states now performing poorly)? How come officials at various levels continue exactly as they have been for decades with impunity when every measure  brings out dismal levels of learning  under their watch? Recently, when our group, IgnusERG assessed class 9 students in a district we found 68% of them to be at class 4-6 levels, 7% below class 3 level, and only 4% at the class 9 level where they were expected to be. When this finding is shared, everyone finds a way to blame someone  else!

Finally, let me leave you with this – in the current form, knowledge of outcomes attained does not help bring about improvement. Most states will be implementing SLAS (State Learning Assessment Survey) in the coming months. But once a state finds out it is performing poorly, say, in mathematics, that finding will not inform it of the reasons why this is so. It could be the poor curriculum (e.g. overambitious expectations) or weak syllabus (less time allocated than required), or inappropriate pedagogy (no use of concrete materials at an early age) or bad textbooks (poorly sequenced or giving discrete rather than contextual examples) or de-motivated teachers or insufficient teaching time (because the state continues using teachers for non-teaching tasks even after RTE and court orders to this effect) or home vs school language issues or at least 10 other problems that can be named, each of which can seriously lead to poor outcomes. So where will the improvement begin?

The point, as mentioned earlier, is: do ask for outcomes, but don’t keep it simplistic, or we’ll continue to get the poor outcomes we’ve been documenting over the last 20 years.

SubirShukla is with IgnusERG, a guild of resource persons working to support teachers and enhance the quality of education, particularly in government schools. Contact: subirshukla@gmail.com

Comments Off on Subir Shukla: Does measuring outcomes improve accountability in education?

CCE has improved scores, not teaching

Continuous and Comprehensive Evaluation, Outcomes


The Times of India

NEW DELHI: School standards in the country remain fairly low with CBSE’s subject evaluators rating 49.8% as average and 9.11% in need of improvement . The analysis of schoollevel assessment doesn’t paint a rosy picture either. Nearly 35% of schools don’t do their summative assessment evaluation strictly as per the board’s marking schemes while 38.1% don’t use sufficient tasks and tools in co-scholastic assessment.

CBSE’s first report on Continuous and Comprehensive Evaluation (CCE) since its introduction in 2009 shows that the performance of schools has been a mixed bag. There has been a marked improvement in student scores and the overall pass percentage since 2010 has improved by 9.48 percentage points to reach a record 98.76% in 2013.

But on the flip side, CBSE officials have found the general classroom teaching methodology wanting. Tasks given to students in 54.6% of the schools are of average quality and the difficulty level is also average or below average in 86% of schools. Nearly 8% of the schools did not adhere to the marking schemes and had inflated marks or grades in the summative assessments.

Parents and teachers, though, seem happy with CCE. TOI was the first to report in November 2013 that students who skipped the class X Boards in 2011 (the year Boards were made optional) fared better in their class XII exams than those who wrote the external exam that year. The new analysis is based on evidence collected from 5,552 schools.

Studies commissioned by CBSE; Management Development Institute, Gurgaon; National University of Educational Planning and Administration , and National Council for Educational Research and Training have shown that 60% of the parents and 90% of the teachers are happy with the new scheme of assessment and the related reforms.

CCE has been around in CBSE schools since 2000 but it was extended to the secondary level in 2009. At present, it is being followed across 14,647 schools, including government and private unaided schools. The scheme covers more than 22 lakh students in classes IX and X. The Board has also trained more than 3,000 mentors since 2010 in India and abroad to train teachers. While the results have improved significantly , CBSE was forced to introduce many changes in the scheme, such as mandatory appearance in both the summative assessments (SA) and also a minimum requirement of 25% cumulative scores to qualify the examination. Earlier, students were promoted to a higher class on scoring 33% in the complete assessment , which includes four formative assessments and two SAs. There used to be no minimum pass marks for the SA, which is the written examination conducted at the end of the two semesters.

Comments Off on CCE has improved scores, not teaching

NCERT releases survey on students’ learning ability

Education, Outcomes


Live Mint & The Wall Street Journal

New Delhi: Almost two-thirds of the students in Class 3 can read and understand simple text, and do basic math such as addition and subtraction, according to the findings of the national achievement survey (NAS) 2014 by the National Council of Educational Research and Training (NCERT).

The survey attempts to gauge the improvement in learning outcomes by assessing student abilities in language and mathematics. The survey’s results should be compared with the findings of NCERT’s Mid-term Achievement Survey for Class 3 conducted between 2005 and 2008, but the agency didn’t release those numbers and Mint couldn’t immediately access them.

In general, though, NCERT’s outcome measures have always shown India faring better than non-government organization Pratham’s Annual Status of Education Report (ASER) studies. While ASER is conducted on households, NCERT’s surveys are conducted in schools.

In NCERT’s surveys, in language, students are tested on their ability to read and understand text and to listen to and recognize words. In mathematics, students are required to perform basic functions.

The survey was based on information gathered from a sample of more than 104,000 students in 7,046 schools across 34 states and union territories.

Two in three students were able to listen to a passage and understand it but only three in five students were able to read a passage and comprehend it.

And while two in three students were able to solve problems relating to simple addition, subtraction, multiplication, geometry and measurement, the number of students who could perform division was just slightly more than one in two.

In most states, there was no significant difference in the quality of learning between boys and girls. Madhya Pradesh was the only Indian state where girls lagged boys in the learning outcome in language. In Kerala, girls outperformed boys in both language and mathematics.

The rural-urban divide also seems to have been bridged with most of the Indian states showing no significant disparity between rural and urban students.

“Overall, Class 3 children in 34 states/UTs were able to answer 64% of language items correctly and 66% of mathematics questions correctly,” the report said. “This National Achievement Survey for class 3 reveals learning outcome trends to be encouraging but still some way to go. Persistence pays,” tweeted Human Resource Development minister M.M. Pallam Raju.

ASER, prepared by the non-profit Pratham Education Foundation, has been highlighting the worsening quality of the Indian education system. The report, released last month, had pointed out that the quality of learning, measured by reading, writing, and arithmetic, had either shown no improvement or actually worsened in the nine years of the United Progressive Alliance government’s rule.

The ASER report showed that the proportion of all children in Class 5 who can read a Class 2 level text has declined by almost 15 percentage points since 2005. Similarly, the portion of students in Class 8 who can do divisions has declined by almost 23 percentage points during the same period.

“Till now, most of the focus has been to ensure access to education. But we have now reached a tipping point where outcomes have also become equally important”, said Yamini Aiyar, director, Accountability Initiative, Centre for Policy Research.

The fund allocation for better outcomes is still small. Out of the total funds allocated for Sarva Shiksha Abhiyan (SSA), the amount spent on two quality-related items—learning enhancement programmes and innovation—is meagre, she said.

The NAS survey also revealed significant disparities across states. States like Bihar, Chattisgarh, Rajasthan, Haryana and Uttarakhand lagged the national average score in both language and mathematics. Puducherry, Tamil Nadu, Kerala and Karnataka were some of the states that outperformed the national average.

Narayanan Ramaswamy , partner and head of education practice at consulting company KPMG, said: “We are not getting the basics right. Enough focus is not going on our curriculum. If the input is not right, then how will the output be good?”, he said.


Comments Off on NCERT releases survey on students’ learning ability

  Disclaimer: The copyright of the contents of this blog remains with the original author / publisher.