Placement exams are ineffective
High school grades better predict success
Tens of thousands of California community college students may be wrongly assigned to remedial English and math courses based on placement exams that are flawed. At a time of increasing state and national scrutiny on completion rates, two national studies from the Community College Research Center (CCRC) at Columbia University’s Teachers College found that the most common placement exams are poor predictors of college success.
In Predicting Success in College: The Importance of Placement Tests and High School Transcripts, researchers found “high ’severe’ error rates using the placement test cutoffs.” How bad? Three out of ten students were wrongly assigned in English. The numbers were lower in math, “but still not nontrivial,” wrote the authors.
A far better indicator of how students will do in college is how they performed in high school. When researchers compared success rates using only high school GPA or only placement exams,they found the “severe error rate” was cut in half for the GPA group.
Comparing success rates by high school grades and scores on the two main placement exams. (Source: CCRC 2012) Click to enlarge.
The results were slightly less conclusive in math in the second study, titled Do High-Stakes Placement Exams Predict College Success? Researcher Judith Scott-Clayton found that math placement exams were more accurate than English placement tests. But Scott-Clayton had this caveat: “Placement test scores are better at predicting who is likely to do well in the college-level course than predicting who is likely to fail.”
None of this surprises Robert Gabriner, director emeritus of the Center for Student Success at the Research & Planning Group for California Community Colleges. He said researchers have been skeptical about the validity of placement tests for years.
“Some of my colleagues said you could put a chart on the wall and have monkeys throw darts at it and it would be just as reliable,” Gabriner recalled.
“This is a real positive step,” he said. “Where this leads is that overnight we have increased the number of students who do not go into basic skills in English and math because they shouldn’t have been there in the first place.”
That could mean the difference between earning a degree or credential or giving up. Numerous studies have shown that students stuck for several semesters in remedial – also known as basic skills – courses are significantly more likely to drop out of community college. A student required to take three semesters of basic skills math has just a 16 percent chance of finishing those courses according to another CCRC study; for English it’s 22 percent.
In California, between 70 and 90 percent of first-time community college students place into remedial math, English, or both. They have to pay for the courses, but the credits earned don’t count toward an Associate degree or units needed to transfer to a four-year college.
“Our average, and we’re not unusual, there are students who typically take seven to eight semesters at City College before they’re able to get out; that’s four years,” explained San Francisco City College chancellor Don Griffin. “The question really is, if you were allowed to go to a college-level class based on high school performance would you do as well as students who had been placed there?”
Chancellor Griffin believes those students would do as well, and the studies back him up. CCRC director Thomas Bailey found that students who ignored the results of the placement exams and went directly into college-level classes had lower success rates than the students who placed into those courses; but, weighed against students who accepted the test results and enrolled in basic skills, they did significantly better in college-level courses, passing at a rate of 72 percent compared to 27 percent.
City College ran its own experiment on placement exams and the results reinforced Griffin’s lack of confidence in the tests. They retested students two weeks after they took the placement exam, and 40 percent of them scored well enough to move into a higher-level course.
Next up, he’s asked the academic senate to approve a pilot program to take 500 new students and not test them all, and instead place them based on their high school grades. Griffin hopes to start the program next fall, and he predicts that the results will be good.
“If you want to look at how competent a person is, you place them in the best class, give them competent instruction, and see how well they perform,” said Griffin. “And I think the expectation is that they will perform well.”
If, indeed, the true mission of the California Community College system is to develop and prepare the current and future workforce for the State of California, would it not make sense to use a norm-referenced assessment device that accurately measures individuals’ abilities to apply theoretical knowledge to real world problems?
Under the current system, we have 112 campuses that each administer their own placement exams, and depending on which campus, somewhere between 50 to 90% of all students require remediation in Math, English, or both, before they are allowed to take for-credit courses. Due to grade inflation over the past x number of years and the enormous variation to be found between courses and teachers in our K-12 systems statewide, using an injdivdual’s high school grades to establish relative academic skills levels is problematic, at best…
Not long ago, I seem to remember reading a discussion in these very pages about developing a standardized assessment for the Community College system to address the placement needs of prospective students. A worthy goal, but surely capable of absorbing untold millions of taxpayer dollars and many thousands of hours of college administrators’ time in a series of seemingly endless committee meetings and discussions without end to devise a standardized placement tool that will be acceptable to the members of the committees.
At the risk of incurring the wrath of my esteemed colleagues and friends in the Community College system, may I offer an alternative suggestion? Rather than invest more time and precious resources in a quest that resembles a child’s game called “Let’s re-invent the wheel”, might we look at commercial produced products, currently available off the shelf, to address this problem?
In the San Joaquin Valley, the local Workforce Investment Boards have been using the “WorkKeys” assessment system, developed by ACT, for several years. It is standardized and norm-referenced, and has been given to several million individuals throughout the United States and around the world. The WorkKeys system even has a remediation software program that matches up to the assessment system, allowing self-paced skills acquistion. Employers and community college systems in other parts of the United States are using it successfully, why not try it here?
Chancellor Griffin’s proposal to place 500 students into classes without placement assessment sounds like a noble experiment, so long as you are not a tuition paying parent of one of the 500 who fails that initial course, and then has to go back to a remedial level course.
Report this comment for abusive language, hate speech and profanity
Keith,
The legislature did pass a common placement exam bill last year, it’s AB 124. Participating in the uniform assessment is voluntary, but the Chancellor’s office hopes that the lower cost from buying in bulk will entice districts to use it. The Student Success Task Force recommendations, which were approve by the community college Board of Governors, also call for a uniform assessment. However, the CCRC studies seem to show that these exams aren’t reliable predictors of how students will do in community college, and that high school grades, despite perceptions over grade inflation, are more valid. A number of community college administrators suggest that using GPA, perhaps coupled with a diagnostic test, would provide a much better indicator of where to place students.
Meantime, there is an as yet unpublished study of a community college in California looking at what happens when GPA is used. The results are reportedly strong enough that the faculty may be ready to sign off on a pilot program.
Report this comment for abusive language, hate speech and profanity
Report this comment for abusive language, hate speech and profanity
Standardized test only measure one dimension of a student’s ability. Some type of performance would give a more accurate measure of a student’s understanding and ability. However, it is not very likely for a performance based test to be administered…
Report this comment for abusive language, hate speech and profanity
So, why if placement exams are ineffective, did the SSTF, a member of whom is quoted in this article, argue so vigorously for a comprehensive placement process that would prohibit low-scoring students from taking any college-level courses?
Report this comment for abusive language, hate speech and profanity