Rules tight on CRCT reviews

ALBANY -- The Governor's Office of Student Achievement released its "Procedure to Access to CRCT Answer Documents" Friday.

The two-page document provides school systems that had schools on the state's "severe concern" or "moderate concern" lists the requirements they will have to follow to view student test sheets from last spring's Criterion Referenced Competency Test.

Last month, the eight Dougherty County School System elementary schools were placed on the state's "severe" list and six other Dougherty schools landed on the "moderate" list after the governor's office conducted its first erasure analysis. The analysis, which it did with vendor CTB-McGraw Hill, investigated wrong-to-right answers on student answer sheets in reading, English/language arts and math.

Since Feb. 9 -- the day before the GOSA made its erasure analysis presentation to the state board of education -- DCSS officials have been requesting access to student test forms. The school system even filed a Georgia Open Records Act request Feb. 22 to collect the information that they believe is critical to their investigation. Calls made by The Albany Herald to the Governor's Office of Student Achievement on Thursday and Friday were not returned.

Early in its "Procedure to Access" release, the governor's office questioned the need for school systems' officials to view the test documents.

"OSA has provided school, classroom and student-level erasure data to all affected LEAs (local education agencies) and has posted to its Web site documentation on CTB's scanner processes," the Procedure to Access states. "This information is designed to aid LEAs in the audit of schools on the moderate and severe concern lists as required by the State Board of Education. It should be noted that response similarity analyses (i.e. pattern analyses) are done by optical scanners and sophisticated statistical software because it is not possible to do such work simply by eye. Such pattern analyses are not necessary to conduct the required investigations."

The governor;s office, however, appeared to contradict itself in a release that it placed on its Web site Thursday. In that statement, CTB-McGraw Hill officials said on Jan. 22 that "these types of analyses must be supported by additional, collateral information." It also stated, "This erasure analysis is considered a check for unusual numbers of erasures to student responses. Without additional layers added to the analysis, this kind of check only addresses the possibility, not the certainty, of teachers or administrators altering the responses of students."

In response to Dougherty County School System's request and to other school systems that had schools on "severe" or "moderate" concern lists, GOSA stated in its Procedure to Access that CTB-McGraw Hill had granted access to answer documents only in the third- through eighth-grade classes. This would only be provided to local education agencies that were required by the state to investigate their schools.

Access would not be given to first- and second-grade answer documents because some of those questions are being used again on next month's 2010 CRCT tests.

"More than half of our flagged test sections represent grades one and two, which the procedure states will be unavailable," DCSS Public Information Director R.D. Harter said. "We've requested outside professionals to review the test documents under supervised conditions where there would be little opportunity for compromise for 2010 test questions."

If school system officials choose to examine answer documents, they must do so between March 18 and 31. They also must pick between two options and follow strict conditions.

The first option is using a one-time sample viewing at the Office of Student Achievement in Atlanta.

"CTB has agreed to send a sample of answer documents to OSA for physical review by LEAs," the Procedure to Access states. "LEAs that select this option will be permitted up to two hours in which to examine a select sample of their flagged classrooms' grades 3-8 answer documents, not to exceed a total of 150 documents, under the supervision of OSA staff."

The Procedure to Access says that local education agencies will not be allowed to photocopy or photograph any of the answer documents and that limited note taking would be permitted, according to Georgia Department of Education assessment protocol. Also, answer keys and test booklets will not be available. It also states a maximum of four individuals who are conducting the audit can represent a school system, such as central office staff, a contracted auditor or a local board member. These individuals would also be required to sign confidentiality agreements.

"We don't understand who and how they will select the 150 test documents to be examined and question whether that is a sufficient sample to determine test behavior," said Harter, noting that that Dougherty system has about 12,000 students in first-to eighth-grade.

The second option would allow LEAs to have one eight-hour business day to look at documents in CTB-McGraw Hill's secure warehouse in Indianapolis, Ind. The same requirements of the first option also apply here in regard to copying, confidentiality and the number of individuals.

"One business day ... I don't know if that would be sufficient time to review the answer documents," Harter said of the school system utilizing donated time by Georgia Professional Standards Commission personnel. "The PSC has identified professionals willing to review our test documents, but we're not sure they would be willing to go to

Indiana to do it."

Officials with the Dougherty County School System and other affected school systems who want to see student test sheets must make their intentions known by noon Friday and fill out a form indicating which of the options they will use. Both options require that school systems to schedule a date on which they would like to view the answer sheets either in Atlanta or Indiana.