Monday, July 16, 2018

RAVITCH ON ROBO-GRADING

EducationViews.org; Houston, Texas

Robo-Grading Student Essays is Garbage


July 15, 2018 by Diane Ravitch


—As someone who devotes her life to writing and trying to communicate meaning, I reject the idea of robo-grading as insulting to the craft. It demands garbage. It deserves to get what it asks for. Anyone complicit in this betrayal of educational values should be ashamed...

NPR ran an interesting story about robo-grading student essays. It didn’t use the headline that appears here, but there is no better way to describe the insanity or stupidity of asking a machine to judge a student essay. First, it shows disrespect for the student. Second, it diminishes the importance and value of language. Third, the machine can be easily fooled, as Les Perelmann of MIT has demonstrated conclusively with his studies of how easy it is to fool the machine. Basically, the machine can’t evaluate facts. Perelman showed that a student could write an essay declaring that the War of 1812 took place in 1945, and the machine would not recognize it as an error.


To demonstrate, he calls up a practice question for the GRE exam that’s graded with the same algorithms that actual tests are. He then enters three words related to the essay prompt into his Babel Generator, which instantly spits back a 500-word wonder, replete with a plethora of obscure multisyllabic synonyms:


History by mimic has not, and presumably never will be precipitously but blithely ensconced. Society will always encompass imaginativeness; many of scrutinizations but a few for an amanuensis. The perjured imaginativeness lies in the area of theory of knowledge but also the field of literature. Instead of enthralling the analysis, grounds constitutes both a disparaging quip and a diligent explanation.


“It makes absolutely no sense,” he says, shaking his head. “There is no meaning. It’s not real writing.”


But Perelman promises that won’t matter to the robo-grader. And sure enough, when he submits it to the GRE automated scoring system, it gets a perfect score: 6 out of 6, which according to the GRE, means it “presents a cogent, well-articulated analysis of the issue and conveys meaning skillfully.”


“It’s so scary that it works,” Perelman sighs. “Machines are very brilliant for certain things and very stupid on other things. This is a case where the machines are very, very stupid.”


Because computers can only count, and cannot actually understand meaning, he says, facts are irrelevant to the algorithm. “So you can write that the War of 1812 began in 1945, and that wouldn’t count against you at all,” he says. “In fact it would count for you because [the computer would consider it to be] good detail.”


Even human readers, who may have two minutes to read each essay, would not take the time to fact check those kind of details, he says. “But if the goal of the assessment is to test whether you are a good English writer, then the facts are secondary.” 
 
Perelman says his Babel Generator also proves how easy it is to game the system. While students are not going to walk into a standardized test with a Babel Generator in their back pocket, he says, they will quickly learn they can fool the algorithm by using lots of big words, complex sentences, and some key phrases—that make some English teachers cringe.


“For example, you will get a higher score just by [writing] “in conclusion,’” he says….


In places like Utah, where tests are graded by machines only, scampish students are giving the algorithm a run for its money.


“Students are geniuses, and they’re able to game the system,” notes Carter, the assessment official from Utah.


One year, she says, a student who wrote a whole page of the letter “b” ended up with a good score. Other students have figured out that they could do well writing one really good paragraph and just copying that four times to make a five-paragraph essay that scores well. Others have pulled one over on the computer by padding their essays with long quotes from the text they’re supposed to analyze, or from the question they’re supposed to answer.


But each time, Carter says, the computer code is tweaked to spot those tricks.


“We think we’re catching most things now,” Carter says, but students are “very creative” and the computer programs are continually being updated to flag different kinds of ruses.


—As someone who devotes her life to writing and trying to communicate meaning, I reject the idea of robo-grading as insulting to the craft.


It demands garbage. It deserves to get what it asks for.


Anyone complicit in this betrayal of educational values should be ashamed.


Source: Robo-Grading Student Essays is Garbage | 

Diane Ravitch’s blog

Monday, July 9, 2018

SHANKER COMMENTS

[Excerpt] Reflections on Forty Years in the Profession [1991] 

By Albert Shanker  [1928-1997]

Public Education and a Multicultural Society 

Why do I continue when so much of what I’ve worked for seems threatened? To a large extent because I believe that public education is the glue that has held this country together. Critics now say that the common school never really existed, that it's time to abandon this ideal in favor of schools that are designed to appeal to groups based on ethnicity, race, religion, class, or common interests of various kinds. But schools like these would foster division in our society; they would be like setting a time bomb. 

A Martian who happened to be visiting Earth soon after the United States was founded would not have given this country much chance of surviving. He would have predicted that this new nation, whose inhabitants were of different races, who spoke different languages, and who followed different religions, wouldn't remain one nation for long. They would end up fighting and killing each other. Then, what was left of each group would set up its own country, just as has happened many other times and in many other places. But that didn’t happen. Instead, we became a wealthy and powerful nation–the freest the world has ever known. Millions of people from around the world have risked their lives to come here, and they continue to do so today. 

Public schools played a big role in holding our nation together. They brought together children of different races, languages, religions, and cultures and gave them a common language and a sense of common purpose. We have not outgrown our need for this; far from it. Today, Americans come from more different countries and speak more different languages than ever before.

Whenever the problems connected with school reform seem especially tough, I think about this. I think about what public education gave me—a kid who couldn’t even speak English when I entered first grade. I think about what it has given me and can give to countless numbers of other kids like me. And I know that keeping public education together is worth whatever effort it takes.