Last week saw new charges and countercharges in the recent skirmish that threatens to break a long-lasting cease-fire and end a peaceful coexistence that dates back to the 1950s.
No, Ethical College Admissions has not expanded its beat to include the conflict between North and South Korea. Last week’s battle of words was between the College Board and ACT over the release of scores for the March SAT, the first administration of the new version of the test, and simultaneous release of both a score converter tool and a concordance for comparing scores with the old SAT and the ACT.
The need for the concordance comes because, as mentioned in the last post, the scores on the new test are 20-40 points higher on each section of the new test (Evidence-based Reading and Writing, Math) except at the extremes on the 200-800 scale. It leaves those of us on the secondary side of the desk to figure out how to tell students that their 1230 score isn’t as good as they think and to explain to parents that tools such as Naviance scattergrams and published Middle 50% ranges are no longer valid or predictive for students in the Class of 2017.
We are obviously not the only ones baffled, confused, and irked by the new concordance. Last Wednesday Marten Roorda, the Chief Executive of ACT, published a blog post criticizing the College Board for creating a concordance based on “equipercentiles” (which he described as an SAT word). The equipercentile method of equating test scores from one test to another assumes that a score at the 65th percentile on one test is comparable, or “concorded,” to a score at the same percentile on a different test. The College Board used the equipercentile method in developing the concordance released last week both between the old and new SAT and also between, using Roorda’s terminology, the “revamped” SAT and the “tried-and-true” ACT.
Roorda’s argument is that a true concordance requires time--and concord. That’s doesn’t refer to the grape or the supersonic jet or even the Massachusetts town where Henry David Thoreau lived and claimed to have travelled widely. It refers to agreement between the testing agencies. In 2006 College Board and ACT worked collaboratively on a concordance based on a year’s worth of data following the last introduction of a “new and improved” SAT. So what brought the two testing agencies together then? An organization more powerful than NATO or the United Nations. Yes, it was the NCAA that brought the parties together for that concordance, based on the need to determine eligibility for athletes.
Roorda suggests that any attempt to equate scores, either with the ACT or with the old SAT, is flawed until there is data from seniors taking the new version of the SAT next fall. Not surprisingly, the College Board immediately responded, rejecting Roorda’s argument as “misinformation.” The College Board says that its concordance is not based solely on the March test but on two concordance studies conducted in December 2014 and 2015 to serve members at colleges and universities needing concordance info to make admissions decisions in the coming year. The real motivation, however, may be found in this sentence buried at the end of a paragraph in the College Board statement: “Additionally, states and districts that have recently switched from administering the ACT to administering the SAT statewide need concordance information to maintain continuity of students’ historical data within their student information systems.”
So what are we to make of all this? In a post a couple of weeks ago, I wrote about the fact that College Board criticism of the ACT test-prep alliance with Kaplan was unprecedented, represented a clear break with a long-standing policy of benign neglect between the two organizations. ACT chose not to respond, which I thought was classy. Was this opportunity to throw a jab too good to pass up?
There is within Marten Roorda’s original blog post a certain tone that might best be described by another SAT word, “Snippy.” And yet I think he is right that any attempt to produce a concordance now is premature and any attempt to do so without ACT collaboration and input flawed. While I am neither a statistician nor did I stay at a Holiday Inn Express last night, I am skeptical that the College Board’s concordance truly exceeds industry standards.
I am even more skeptical of the College Board claim that the concordance was created to meet the needs of member colleges and universities. If true, it would be one of the few times recently that the Board has acted as if it were a membership organization rather than America’s most profitable non-profit organization. The College Board long ago became a dot.com masquerading as a dot.org. I think the development of the concordance is clearly to serve the burgeoning state testing market rather than the college admissions market.
There is one other issue that continues to confuse me (which doesn’t take much). The College Board has explained the need for the concordance by observing that temperatures on the Celsius scale have to be converted to Fahrenheit. Having taken science, I get that. But temperature conversion involves a formula, not a concordance.
SAT scores are scaled scores. A student’s raw score is converted to a score on the 200-800 scale. I’m unclear about why the College Board couldn’t scale the scores on the new SAT in such a way that a 550 is comparable to a 550 on the old test. That’s clearly been done when the test was changed before. If the argument is that the new SAT is a fundamentally different test, then why not introduce a new scale altogether? That was done to some extent when the SAT went to three sections and a student’s score was on a 2400 rather than 1600 scale. Why is the 200-800 scale sacrosanct?
In any case, we don’t need prolonged bickering between the College Board and ACT. It’s time to end the dance of discord. Neither the College Board nor the ACT has any future on “Dancing With the Stars.”