• Anchorage has undertaken a number of projects for 1997-98. These include:
• Completion of a grade 4 integrated Language Arts and Social Studies instructional unit on the founding of Jamestown which includes assessment activities in reading, writing, listening, map making, decision making, and categorization,*
• Initial development of a www based on-line assessment.*
• Redevelopment of "survey" CRTs covering grade 8 Science and Social Studies to include limited performance assessments.*

• Annual Profile of Performance including district and individual school reports of student performance and growth based on NRT scores.*
• Graduate Follow-up Survey*
• Review of Success of Full and Half Day Kindergarten Success.*
• Review of Success of new mathematics adoptions.*
• Community Surveys*

• Internet based assessment systems for K-12*
• Measures of "Independent Reading for Grade 3*
• Group measures to screen grade 6 students for grade 7 Algebra*.

For data disaggregation this year we focused more on showing growth over time within each group rather than comparisons across groups. This allowed us to emphasize progress rather than gaps.

Our K-2 performance assessment process for reading and writing has for two years now used a continuum of performance across all three grade levels. In our report this year we were able to follow the same set of students across both years to show the developmental progression of their language arts skills.

This year the six high needs elementary schools in the district which have received supplemental support services for the past two years made individual reports to the school board on the progress they have made with their individual plans. Our office provided extensive support in analyzing and reporting their results.

Our district has been involved this year in redistricting elementary and middle school attendance areas to address population shifts and new growth. As you might guess, this explains what we have been doing most of the rest of the year. Our office has been involved in analyzing community surveys, developing comparative analyses of plans, helping with planning sessions, etc.

We would appreciate feedback from anyone who has been successful in engaging school staff in the critical thinking and data-based decision making of evaluation without draining off an inordinate amount of instructional time.

Development of District Learner Goals that specify what students should know and be able to do along with critical indicators for determining when students have become excellent or proficient.

Learning to work with a Board that wants more involvement in curriculum than previous Boards.

Examining the alignment between the Iowa Tests of Basic Skills and our curriculum. *

Developing a database that enables analyses of cohort test data and conducting these analyses.

Helping select a new student information system that will give use much richer data, especially historical.

Beginning planning for development of an individual learning plan for each student.

Developing and implementing a continuum reporting system at grades K-3.

District standards and how they are being assessed.

Grade/progress reporting systems, especially at the elementary level.

Effectiveness of algebra at grade 8.

from Jan 1, 1998: Ottawa-Carleton District School Board

• Production of School Profiles
• The results of the Board-wide Evaluation projects. Annual Report, 1997
• Graduate Survey

*Literacy and Numeracy

*Reporting school results in context (school profiles)

*Large-scale assessment in senior English and mathematics

*Early Identification and Intervention Strategies

Development of special needs/inner city indices

Longitudinal studies of achievement; value-added analysis

Measures of literacy and numeracy

School audits

Changing girls' attitudes toward math

• We are evaluating the effects of a total re-constitution of two low achieving elementary schools.
• We are engaged in a study of effects of differential school funding levels on student achievement in two districts.
• We are engaged in two major evaluation studies.
• We are conducting a study of the equivalency of state assessment with our local IRT test.

• Standard Setting using IRT Achievement Scale
• Measuring and Reporting on 15% Achievement Gain Commitment to public*
• Relating State Assessment results to local IRT results
• Evaluating De-Tracking in High Schools*
• Literacy Assessment
• School profiles*
• Dealing with Re-Organization
• Strategic Plan Monitoring*
• Accreditation process Development
• Evaluating Re-Constituted Schools*

• Evaluating Restructured Schools
• Evaluating School Effectiveness
• Evaluating Technology Implementation
• Setting Cut Scores

Scores on the MAT7 and the State criterion-referenced test continue to show moderate gains for most subject areas.

Several schools' staff received monetary awards for meeting district benchmarks in student performance.

I help schools plan, collect and analyze data to meet requirements (ever-changing) for Title 1 evaluation.

Prepare summary reports for MAT7 and state criterion-referenced test results: grades 2-11

Prepare district benchmark accountability report for each school used to distribute rewards to schools.

Help design and tabulate surveys as needed.

Implement a new system to use local tests to monitor student achievement of the district curriculum. We are using Abacus for NCS for this purpose.

How districts deal with accountability systems that reward or punish schools for student achievement, especially how to handle equity concerns?

Fostering a concern for decision-making based on data rather than anecdote?

1. More than 75% of the DoDDS students at grades 5, 8, and 10 are proficient writers as measured by the DoDEA Writing Assessment.

2. DoDDS schools had the 8th ranked scale score among the 44 states and jurisdictions reported for the 8th grade NAEP Science Assessment.

3. Results of an evaluation of the School-Home partnership program indicated that DoDEA Schools demonstrated visible progress towards school home partnership.

• Reading Recovery Evaluation
• Implementation of Curriculum
• School Improvement Evaluation
• Best Practices for Curriculum, Evaluation and Instruction
• Evaluation of Advancement via Individual Determination

• School Effectiveness Measures
• Using Standards as a basis for Evaluation
• School Improvement Best Practices

• Development of district content standards and benchmarks aligned with state standards
• Implementation of School Improvement Teams to focus on academic achievement
• Evaluation of headstart/Preschool Child Care Program

• Evaluation of District's Kindergarten Program/Development of Comprehensive Recommendations
• Study of Everyday Mathematics Program
• Administration of District Assessment Program

• Assessments to be used with district-developed standards
• Meeting the assessment requirements of IDEA
• Evaluating School Improvement Plans
• Setting Cut Scores

Title 1 Program Improvement Model developed & implemented

Assisted in developing Accountability System.

Because of state department directives, we are administering the third norm-referenced test in as many years.

*Evaluation of Limited-English Proficient Programs

*Evaluation of $21 million in National Science Foundation grants to improve math and science instructional programs.

Development of reporting system to capture teacher ratings of Reading/Language Arts and Mathematics standards.

Assessments in Southeast Asian languages such as Hmong, Kmer, Vietnamese, and Lao.

Methods for combining multiple measures as called for in Title 1 Program Improvement guidelines.

Development of a results-based evaluation system for schools using benchmarks for performance indicators.

Participation in the Educational Benchmarking Network--a consortium of large successful school systems to compare results and best practices.

Graduate follow-up study of 1994, '95, & '96 graduates.

All of the above.

Report available for Graduate Follow-up Study.

Any form of evaluation of schools that is based on measured performance.

Note: I will not be at the meeting, but I am very interested in this group.)

• The Evaluation Team participated in a series of Elementary Achievement conferences conducted by the Associate Superintendent of Instruction (one for each elementary school in the district). The conferences, which were a response to declining test scores, focused on how reading and math were being taught and what needs to be done to improve instruction. The Evaluation Team provided data packets to each school and was represented on the visitation team.
• Resurgence of interest in program evaluation. Thirteen of twenty-six temporary projects are program evaluations (including the evaluation of both grants and district funded programs of interest).
• Publication of a three year assessment plan

NEW PROJECTS:

§ Development of District Assessment Standards (Leslie Lukin)

§ Evaluation of the Alternative Calendar at Belmont Elementary School (Bob Reineke)

§ Evaluation of Science Focus Program at Folsom Children's Zoo (Marj Willeke)

§ Study of How Principals Monitor Achievement (Leslie Lukin/Karen Stanley)

§ Follow-up Study of the Class of 1991 (Bill Walker)

CONTINUING PROJECTS:

§ Administration of District Assessment Program* (Nick Novak/Leslie Lukin)

§ Climate Assessment* (Bill Walker)

§ Development of Graduation Proficiency Demonstrations* (Leslie Lukin)

§ Evaluation of Early Childhood Grant* (Bob Reineke)

§ Evaluation of Even Start* (Marj Willeke)

§ Evaluation of Grant to Improve Skills of First-Year Teachers* (Bill Walker)

§ Evaluation of Inclusive Practices Grant* (Marj Willeke)

§ Evaluation of Library Power Grant (Marj Willeke)

§ Evaluation of Reading Recovery* (Bill Walker)

§ Evaluation of School to Work Grant* (Bill Walker)

§ Evaluation of Title 1* (Marj Willeke)

§ Improving Classroom Assessment Practices Grant (Leslie Lukin)

§ Multicultural Education Program* (Nick Novak)

§ Publication of District Statistical Handbook* (Nick Novak)

§ Support of School Staffing Conferences (Nick Novak)

Use of standards and the development of standards based assessments.

*Success of the MPS School Accountability Plan coupled with the School Education Plan to monitor student achievement.

*Increased participation in Data Driven Decision Making Seminars by "school teams."

One of the major areas of focus throughout this school year has been in the area of special education participation in State and district assessments. Involved in this is the identification of appropriate accommodations, the development of a data collection process, and the creation of an alternate assessment process for severely disabled students.

*Special education students participation in the assessment process.

*Research in "good" practices.

*Data collection processes for the calculation of "dropouts."

1. We are in the midst of a major reorganization of the district.

A major initiative involving over 100 people in the community to develop district priorities and a district plan.

Standard setting for the district's criterion referenced test

New ideas about how research and assessment are organized

1. Assessment design and training in the standards-driven classroom. (Re-design and training for classroom teachers.)
2. Baseline year for new state assessment of Sunshine State Standards (Florida Comprehensive Assessment Tests in reading and mathematics - grades 4, 5, 8 and 10).
3. Preparation for new standardized test adoption (March-July, 1998).

1. Design of training for classroom teachers on assessment techniques aligned with characteristics of intended learning outcome.
2. Evaluating the impact of CCC Labs at elementary, middle, and high school levels.
3. Validation of the running record as a formal measure of developmental reading progress.

• Methods for dissemination and follow-up on initial training with teachers on assessment methods
• Use of value-added and HLM designs in evaluating effects of school and district programs.

1. Provided school program practices and studies for court case considering our district's desegregation efforts. Judge to issue ruling June 1998.

2. Developed comprehensive school climate program to assess community, staff, student, and PTA opinions of 180 schools annually.

3. Performed Value-Added analysis of school and classroom determinants of student achievement, controlling for student poverty and mobility.

*Value-added assessment of classroom variables, controlling for student poverty and mobility.

*Enacted practices evaluation of Creative and Performing Arts magnet program.

*Annual community Satisfaction Survey.

*Annual School Climate Survey.

*Annual Graduate Survey Report.

*Annual Accountability Report

*Disaggregation of statewide performance assessment results by special education status.

*Development of departmental Operating Manual.

Evaluation of magnet school recruitment efforts (phone survey).

Annual Maintenance of data mart (i.e., relational database evaluation support system consisting of 132 tables and 1.7 gigabytes) to incorporate 1996-97 end-of-year data.

Enhancement of departmental data mart:

• include elementary student-classroom teacher links,
• NRT scores,
• audit of actual graduates from student records office,
• redesigned course hierarchy map and rebuilt highest course per student per subject area back to SY93
• developed procedures to prepare data for warehouse,
• prepared SQL scripts to automatically verify data just prior to loading into warehouse.

Monitored third-party research authorization.

Initiated research into 13 new indicators identified by desegregation court panel, including student bussing distance/time, participation in sports and school clubs, teacher assignments to Advanced Placement courses.

Provided data to outside agencies:

OCR/Special Ed Compliance

CPMSA/NSF

Equity 2000/The College Board

Presented at various conferences:

CREATE

• Enacted practices Evaluation Model

NCES

• Data Warehousing for Student Cohort and Multilevel Analysis
• Ethical and Control Issues for Collecting, Managing and Reporting School System Data
• Measuring School Effects: From the Naked Eye to the Microscope

Maryland Assessment Group (MAG)

• Assessing School Effectiveness with the Maryland School Performance Assessment Program

Current school district uses of SEM methodology.

Professional development: measurable definitions and evaluations of program impact.

Under a $184 million dollar bond referendum approved in September, 1996, Richland County School District One is building a new middle school, six new elementary schools, renovating two other schools, and bringing all elementary schools up to District standards. Middle and schools will be brought into compliance with all environmental regulations and have some repairs, as well.

A five-year $4 million-dollar technology grant called Richland Clicks! is being implemented. The two major goals are to increase student achievement and to improve family and child well-being through technology. The project has four main components: accessibility of the entire curriculum for Child Development through grade 12 on the Richland Clicks! Web site; development of interdisciplinary units in five lifelong learning strands; community involvement; and professional development.

1. Evaluation of High School Block Scheduling Programs

2. Designing support materials to help school-based projects build evaluation into their proposal plans.

3. Training and support for development of District assessments in various subject areas.

District assessments

Supporting multiple evaluation projects simultaneously

Controlling and reporting evaluation costs

New State Assessment

Implementation of Standards

Title 1 Program Improvement

Class Size Reduction Fallout

Implementation of Multiple Measures and Standards-based accountability reporting in core subjects.

Class Size Reduction and Evaluation of results.

Public relations with new STAR program and standards-based reporting. (State assessment and Title 1)

Quality performance assessments in core subjects for grades 4-11.

Standards reporting formats for parents and teachers.

Report cards that are standard-based.

Procedures for combining multiple measures for Title 1 reporting.

* District initiates the Accountability System by identifying and reviewing 20 low performing schools.

* Piloted Literacy Portfolios in grades 3, 4, and 8 and will centrally score on census basis by August 1998.

Data based instructional decision-making facilitated by Executive Profile (16 Expectations), School Profiles, and Accountability Data Sheets.

* Single-year drop-out rate at 2.3%.

* Fluent English speaking students tend to score at or above national norms on standardized testing.

* Developing site Accountability Data profiles for use with the District Accountability Systems.

* Responding to data requests from the 20 identified low-performing schools.

* Developing a data retrieval system for English Learners.

* Developing data reporting system for the District Literacy Portfolio.

Data retrieval systems for English Learners.

Synthesizing multiple data indicators into a two to three paragraph narrative.

Accountability system target-setting.

*Development and implementation of Achievement Level Test of Stockton

*Development and implementation of English Language Development Profile (K-3, 4-6, 7-12)

*Development and implementation of kindergarten and first grade assessment rubrics

See above.

Using multiple measures to assess students: combining multiple measures

Setting benchmarks on assessment scales

Test score improvement, especially for intervention schools

(I'm new, so this is sparse.)

black scheduling

evaluation of reading recovery

Steve--Sorry this is late.

• Major reworking of state content standards and assessments; state abandons state level performance assessments but requires districts to do them.
• State and district working to implement graduation standards.
• District working to implement skill standards for promotion.

• School Profile Documents*.
• Implementing a system-wide (21 schools) instructional management system at the elementary level(Abacus).
• Implementing Curriculum-Based Measurement in Reading & Math grades 1-5 for all students fall and spring and three additional end of quarter assessments for students performing poorly in reading. CBM probes have equivalent forms reliability exceeding .90 and a single one minute reading fluency probe correlated .72 with MAT Reading comprehension at 3^rd grade. Monthly measures for reading and math for all students grades 1-5 are being planned for next year. Have established reading fluency performance standards for grades 1-5.
• Building a comprehensive School-to-Career system.

• Meeting the assessment requirements of IDEA.
• Who is working with CBM and might want to collaborate on R&D projects, grants, presentations?
• Who is implementing a computer-based instructional management system?
• Value-added models for looking at instructional impact.

We are implementing a school self review process in 22 middle level schools. School reports will be completed in June 1998. We are also developing an external audit to be conducted in a small sample of schools.

Mathematics achievement tests are continuing to be administered to students in Grades y, 8 and 10.

Assessments of student writing are continuing to be conducted in Grades 5, 7, 9, and 12.

A new reading and writing assessment has been developed for Grade 11 and will be administered in Spring 1998.

Assessment Reports are available for mathematics and writing assessments conducted in 1997*

School Profiles were prepared for all schools for the fourth time.*

We are preparing for the administration of perception surveys and the conduct of school self reviews in elementary schools. Perception surveys will be administered to parents, teachers, support staff and students (Grades 4-6).

We are developing the external audit for the school review process

School improvement initiatives, particularly the use of data and the application of Total Quality Management principals.

Conduct of school reviews by external teams, particularly as a follow-up to the conduct of a self review by a school team.

• Report on Achievement Patterns (students grouped by initial levels of achievement were followed over time to determine what happens to their achievement)
• State of the System newsletter
• New statewide, high-stakes accountability program

*Implementing a sunset review process of various programs, and developing a cost-effectiveness model.

*Compiling a complete list of instructional services and costs in one document.

• Merging Internal Audit Office into our department

• Cost-benefit or cost-effectiveness reports on programs.
• Documents showing all programs and costs in an easy-to-read format.
• "Take-over" provisions or effects of high-stakes testing on schools