California stutter steps on data
Data quality report shows a state of inertia
California’s various education databases bring to mind that philosophical puzzle about the tree falling in a forest. The state has the ability to link the data systems and make them robust, but since there’s no political will to do that, how useful are they? Okay, perhaps that’s overstating the existential question, but it does keep California from even having a thoughtful conversation about the pros and cons of data sharing.
So when the Data Quality Campaign yesterday released its seventh annual report on state education databases, California received mixed marks. The elements of a strong system are there, but they’re not backed up by action.
Increase in states meeting all elements of a good data system. (Source: Data Quality Campaign). Click to enlarge.
Since the organization’s first survey in 2005, California has gone from meeting just one of the 10 Essential Elements of a data system, to a full sweep. Thirty-six states have met all the elements, which include giving each student a unique identifier and having access to students’ college readiness test scores, graduation and dropout rates, and transcript information showing each student’s classes and grades.
But when it comes to action items the state has met only four of the ten. To put that in perspective, no states have implemented all ten actions; although more than half of them are at five or better. Here’s where California stands:
No – Link data systems
No – Create stable, sustained support
Yes - Develop governance structure
Yes - Build state data repositories
No – Implement systems to provide timely access to information
No – Create progress reports using individual student data to improve student performance
Yes – Create reports using longitudinal statistics to guide systemwide improvement efforts
No – Develop a P-20/workforce research agenda
No – Promote educator professional development and credentialing
Yes – Promote strategies to raise awareness of available data
The four Ts
Before Gov. Brown’s veto last summer of CALTIDES, the statewide data system for teachers – which required Brown to return a $6 million federal grant to help build the database – the state had also met the second action item of creating stable, sustainable support for longitudinal data. At that time, California was on a path to developing two huge databases that could be linked. The other one is CALPADS, the California Longitudinal Pupil Achievement Data System, which is up and running, and recently published the state’s most accurate dropout and graduation rates.
But it’s the link between the two systems that’s the big data kahuna. The Data Quality Campaign’s report now says that California “is not moving forward in a way that allows data to impact student learning, and political implications in the state are a detriment to this.”
The state’s reluctance to pursue a more expansive data system validates the four barriers to using data described by DQC’s report: Turf, trust, technical challenges, and time. Trust is the biggest hurdle.
Teachers unions foresaw a nefarious application of statewide data by using only student test scores to evaluate teachers, a concern that Gov. Brown shared. “Gov. Brown is very fixated that it’s all about test scores, and it’s not,” said Paige Kowalski, director of DQC’s state policy initiatives. “It’s only about test scores to the extent that the leadership makes it that way. As governor if he doesn’t like what it is, he can turn it into something that districts want.”
In the absence of a statewide strategy, some California districts have established their own data systems that teachers use to inform instruction. In Sacramento City Unified
Letter from CORE to Gov. Brown on the need for a statewide data system. (Click to enlarge)
School District, teachers meet weekly to review student work and test scores and use that information to modify lesson plans or provide more individualized instruction.
Sacramento City is a member of the California Office to Reform Education, or CORE, a group of seven unified districts, including Los Angeles, Fresno, and Long Beach, that are working together to build a shared data system. But, in a letter to Gov. Brown last June, urging him not to cut funding for CALPADS and CALTIDES, the superintendents said there’s a limit to what local districts can do without a statewide foundation to support them.
“CORE does not have the capacity to build a data system that tracks students longitudinally over time throughout the state,” they wrote. “The CALPADS infrastructure will provide a backbone for development of local data systems that can be used to promote and improve teaching and learning.”
Brad Strong, the senior director of education policy with the advocacy group Children Now, said it’s also not feasible to expect all the state’s 1,000+ school districts to have the resources and technical know-how to build their own systems.
What’s more, he said, local data systems alone couldn’t track a student who left a district to learn whether he dropped out or enrolled in another district, they couldn’t follow their graduates to see if they were prepared for college-level work, and they wouldn’t be able to analyze teacher preparation programs to determine which ones are doing the best job of developing highly qualified teachers.
“Some districts are going to have the capacity and they’re going to do fabulous work,” said Strong, adding that a statewide system is still required to give all teachers and administrators access to those top teaching tools. “In an ideal world, how do we capture that magic and give it to all districts?”
9 Comments
Trackbacks
- How should we measure our schools if not by API? | Thoughts on Public Education
- How Should the Success of Schools Be Measured? : Ed100
The grades on the action items must be self reports. The biggest and longest lasting problem with education data in California is the complete lack of a data governance system to provide long term support for the development and use of data in the state. It is disappointing to see that the DCQ missed this one so badly and graded the state as having a governance structure when they so clearly don’t. There are a couple of other actions and elements where it looks like the DQC has missed the mark suggesting that CA had data functionality that is questionable.
Report this comment for abusive language, hate speech and profanity
i dont understand what use all these data metrics are if they are never used. much of the data that is actually useful cant be published because it contradicts privacy laws. and when it could be used by districts they generally dont because it costs too much money to compile. and often, when they pay to have it done, too few people understand it to do anything worthwhile with it.
what is the hispanic api when exluding hispanic ell students that are not yet rfep?
what it the african american api when excluding swd? (especially for middle school)
what percentage of rfep (among the highest scoring subgroup) are not hispanics? by grade? how does their performance differ from hispanic rfep?
how can we make use of data for policy-making when its virtually impossible to answer simple questions like this? and why, with all the supposedly disaggregated data, do people never bother to ask questions like this?
Report this comment for abusive language, hate speech and profanity
Yeah, well …
http://www.insidehighered.com/news/2011/10/11/california_postsecondary_education_commission_closes
Report this comment for abusive language, hate speech and profanity
The national data quality campaign cannot know what has been happening recently to improve California data systems. PPIC hosted a meeting of all statewide data base organizations, and many key users and analysts. It is clear that we have a huge amount of data, but too much of it is in silos. The local districts data priorities for data use are much different from the statewide policy analysis needs. Calpads has a teacher and student identifier, so the Cal Tides issue is overblown, and not what Baron implies. I agree with Brad Strong’s concerns
. The National Student Clearinghouse has longitudinal data capacity for California with lots of possibilities, and found our college going rate from high school is 10% higher than we thought previously. SARC is underutilized as a qualitative data source and SBE is exploring other qualitative measures. Kowalski’s comments about Governor Brown’s view are wrong. His statements are clear that he is not just concerned about test scores. In sum, the situation is complex and fluid, and there is momentum to move forward.
Michael Kirst, President, California State Board of Education
Report this comment for abusive language, hate speech and profanity
I dont think SARCs are sufficient as a data source, though I would love to see them better-utilized (and improved).
I agree with your agreement with Brad. Districts who dont have this ability are sometimes de-facto left to agenda-driven (and media-influenced) policy. And worse, their communities are largely flying blind.
Report this comment for abusive language, hate speech and profanity
The headline from the Data Quality Campaign’s Data for Action 2011 is “States Could Empower Stakeholders to Make Education Decisions with Data….but They Haven’t Yet”. We know that states—including California — have done an excellent job in collecting student level data over the past seven years through our annual survey, but states continue to miss the policy opportunities to truly leverage this powerful information to improve student achievement.
I wanted to clarify a few things about the DQC, our annual report, and this article.
Through our annual survey, states voluntarily self-report to the DQC their status on the 10 Essential Elements and the 10 State Actions to Ensure Effective Data Use. DQC does not independently verify or research state submissions. Rather it is incumbent upon each state to have the necessary conversations across P20 sectors to accurately report out the state’s status around effective data use. DQC believes that while tremendous progress has been made across all fronts, we still have much work to do around the quality and depth of the work reported by states.
On numerous occasions in the past months, and during the call with this reporter, the DQC has clarified the confusion around the CALTIDES veto; the state is still currently linking teachers and students through CALPADS and does report through the DQC survey that it has operationalized DQC’s element 5 (the teacher student data link).
·In the statement attributed to me, it is unclear that the “it” I’m referring to is CALPADS. I’d like to clarify the statement and provide some context that the length of the article did not allow. For almost 7 years, DQC has sought to help state policymakers understand how statewide longitudinal data systems is can be powerful tools to support continuous improvement at all levels. The power of state longitudinal data systems is that they are able to provide a richer picture of system performance and student achievement because they collect and link multiple data points, including but not limited to, test scores. The collection of state assessment scores is one of DQC’s 10 Essential Elements…but there are 9 others (including student outcome data, enrollment/demographics/program participation data, transcript data, etc.). Every state now has systems containing rich data sets on their students and could be putting that data work for their districts and other local stakeholders. But as noted in our headline, they aren’t yet. The statement I made (but could have been clearer about) in this article is that CALPADS is more than test scores and the state’s leadership has the unique opportunity to use the bully pulpit to turn this system into something that districts want and can use. As evidenced by the CORE districts, we know there is demand. And, as evidenced by the majority of CA’s districts, we know the ability to do this at local levels is largely absent due to capacity challenges.
I echo Brad Strong’s comment and hope that CA can learn from other states who have started to “capture the magic and give it to all districts”. Colorado’s growth model has been adopted by over a dozen states to date and the state has served its 180 districts well (particularly its lowest capacity districts) by developing a powerful dashboard illustrating student growth data (http://www.schoolview.org). As a native Californian, I know that the state can and will figure out its own brand of magic in partnership with its districts. The DQC stands ready and willing to assist the state in this important endeavor.
Paige Kowalski
Director, State Policy Initiatives, Data Quality Campaign
Report this comment for abusive language, hate speech and profanity
War mir neu und finde ich gut. Der Blog ist scht eine gute Quelle für alles über Stutter. Danke, Patricia.
Report this comment for abusive language, hate speech and profanity