Databus Issue: 2007 3 09/17/2007
Student Performance Data and the Era of Accountability
Alan Grulich director, assessment and evaluation
PDF
In our era of accountability, data is
everywhere. Over the past decade, most
K-12 districts have invested significant
resources in mechanisms to manage the
vast array of student performance data
generated by a continually evolving alphabet
soup of accountability initiatives and
mandated assessment protocols.
We have come a long way since the
arrival of the first SAT9 data disks nearly
10 years ago when we struggled to make
sense of percentile ranks and labored to
parse data dictionaries with 3,000-plus
characters. In addition to creating new
infrastructure for storing and maintaining
student performance information, our
apparatus for collecting and reporting
basic student demographic information
has also improved. Six or seven years ago,
the idea of collecting parent educational
levels seemed burdensome and intrusive to
many districts. However, the CDE’s correlation
studies used to determine similar
schools ranks consistently show average
parent education level as a more powerful
predictor of API than any other demographic
consideration. Despite its onerous
requirements, No Child Left Behind
(NCLB) is a continual reminder of the
importance of the demographic context to
interpreting achievement results. Together,
http://www.cde.ca.gov/ta/ac/ap/documents/tdgreport0607.
pdf
these issues have dramatically changed
how we direct resources toward effective
instructional interventions.
Enhancement in the data-management
capabilities of most school districts has
come in many forms from FileMaker Pro
reports to full-blown academic data management
solutions such as Edusoft or—for
the most advanced districts—businessintelligence
solutions designed for the
singular purpose of analysis and reporting.
These systems have been developed
to accommodate the greatly expanded
audience for student achievement information
and provide the multiple perspectives
required by each type of stakeholder
(e.g. parent, student, teacher, site administrator,
etc.). They allow teachers the
ability to look deep into the sub scores
for a particular test while providing a site
administrator with a longitudinal vantage
point. Many districts have expanded these
systems to give parents a near real-time
perspective on how well their students
are progressing. CDE’s DataQuest allows
anyone with an interest and an Internet
connection the ability to explore the entire
performance legacy of any school or district
in the state.
As the technological tools for maintaining,
managing, and manipulating
student- performance data become more
sophisticated and more pervasive, many
of the fundamental limitations to the
“holy grail” of data-based decision making
are beginning to abate. As a result, the
remaining obstacles are increasingly likely
to be a function of how data is used.
An understandable, but unfortunate
reality in K-12 education is the tendency
to think that the job of data analysis is
complete once the tables and charts are
dutifully transcribed into the interminable
list of documents and reports (everything
from school accountability report cards to
single plans for student achievement) that
attempt to characterize how well schools
and districts are doing. Too frequently,
the zeal to create a comprehensive picture
of student performance robs the more
important work of assembling data into a
cohesive narrative that tells a story, or at
least offers a hypothesis, of what worked
and what didn’t. The process of taking a
mosaic of different performance indicators,
which often lead to contradictory
inferences when considered in isolation
and making sense of the larger pattern is
the new challenge.
