Mathematics Instructional Materials Review Advisory Group ReportDRAFT - 4/29/2008

Mathematics Instructional Materials Review Advisory Group Report

DRAFT - 4/29/2008

Contents

Executive Summary 2
Expected Outcomes 2
Best Practices Overview 3
Research 3
State Comparisons for Mathematics Curriculum Reviews 4
Proposed Categories 6
Proposed Measurement Criteria 7
Content/Standards Alignment 7
Structure & Organization (Scope & Sequence) 7
Instructional Design 7
Student Experience/Nature of Tasks 7
Assessment 8
Professional Development 8
Equity and Access 8
Next Steps 10



Executive Summary

A statewide group of mathematics educators, curriculum directors and administrators met to identify general categories for evaluating mathematics instructional materials. After reviewing best practice examples and current research on effective teaching and learning, the group decided upon 7 general criteria. The group then developed ideas for measuring those criteria and offered additional guidance about proposed processes for the instructional materials review.

The purpose of this report is to document the outcomes of the Mathematics Instructional Materials Review (IMR) Advisory Group and to outline the expected process for reviewing mathematics instructional materials.

The work of the Advisory Group is crucial to the success of the instructional materials review project. OSPI has committed to an inclusive process that actively solicits information and advice from many stakeholder groups. It is essential that the review methodology and process measures the appropriate factors, and takes into account a broad range of instructional materials-related criteria that contribute to effective teaching and learning. Ultimately, OSPI will recommend three mathematics texts at the elementary, middle and high school level. It is imperative that the process, evaluation and final recommendations support the success of all students in the Washington K-12 system.

Expected Outcomes

The Mathematics IMR Advisory Group adopted the following expectations for their work. They were successful in accomplishing the first three bulleted outcomes, and will continue to refine their work in an additional work session.

• Examine criteria and processes used by other states (OR, NC, CT) and districts within Washington to review mathematics textbooks.
• Identify 5-7 general criteria for use in our curriculum review that are specific, measurable, efficient and valuable to help us determine which three texts to recommend at the elementary, middle and high school levels.
• Identify why the criteria is important to measure, and develop ideas on how to measure those criteria.
• Recommend relative weights for selected criteria.


Best Practices Overview

Research

One of the goals of the Mathematics Instructional Materials Review Project is to design a review process and methodology that is grounded in current research and utilizes best practices gleaned from other states and districts which have recently completed a successful review process.

The Mathematics IMR Advisory Group reviewed the following research and publications before attending an all-day work session to develop the proposed criteria.

• National Mathematics Advisory Panel Foundations for Success
• Math Educators Summary of Effective Programs
• NCTM Principles and Standards for School Mathematics
• Choosing a Standards-Based Mathematics Curriculum
– Chapter 6: -- Developing and Applying Selection Criteria
– Appendix: Sample Selection Criteria
• Washington Revised Mathematics Standards (4/08)

In addition to the research review, participants examined mathematics curriculum review tools from three states (OR, NC, CT) and two Washington school districts (Edmonds, Vancouver). They identified good ideas to adopt from these sample tools. A summary of the state comparisons is shown below.


State Comparisons for Mathematics Curriculum Reviews:
(This appeared in Table form - which is changed to the following for blog readability)

Washington(new k-8 standards), Oregon, North Carolina, Connecticut

General Process:

WA:
• Curriculum review committee establishes criteria
• Educators evaluate texts based on standards alignment and other criteria (some qualitative, some quantitative)
• Review group selects top three recommendations at each grade range (E,M,H)

OR:
• Criteria developed in one day session by educators
• Publishers do alignment and criteria comparison
• Publishers pay a fee to be included in evaluation process
• Publishers give presentation during week-long review process
• Educators evaluate publisher’s response and presentation
• Textbooks must meet basal requirements. If met, then those that meet 80% of other criteria are listed in the textbook adoption list
• Spend about 3-4 hours per publisher for review, per grade band

NC:
• Curriculum review committee establishes criteria
• Publishers rate texts
• Educators evaluate response
• Textbooks added to approved list for SD purchase

CT
• Local control state
• They review curriculum documents from districts, not individual publisher texts.
• Had at least 4 reviewers for each district curriculum.
• Each individual did ranking.
• If individual rankings were off, they had a group discussion
• Used Choosing a Standards-Based Mathematics Curriculum to created evaluation tools for curriculum documents. (www.edc.org)

Standards Alignment

WA:
• Detailed check by standard and grade level

OR:
• Comprehensive
• Supportive
• Minimal

NC:
• Materials support a minimum of 80% of state standards at grade level

CT:
• Exemplary
• Some
• None

Other Curriculum Evaluation Criteria:

from Washington:
• Structure & Organization
• Instructional Design
• Student Experience/ Nature of Tasks
• Assessment
• Professional Development
• Equity and Access

from Oregon:
• Equity
• Accessibility (meets NIMAS)
• Utilizes best practices for teaching/learning math
• Teacher materials
• Online resources
• Supplemental/RTI materials
• Professional Development
• Readability

from North Carolina:
• Content-instructional approach alignment
• Utilizes best practices for teaching and learning
• Accuracy
• Appropriateness
• Scope
• Teacher resources
• Technology

From Connecticut:
• Alignment with NCTM PSSM process standards (problem solving, making connections, communication, reasoning/proof, representation)
• Identifies clear grade-level expectations
• Concept development across grade levels
• Balanced instructional approach
• Reflects current research about mathematics & learning
• Uses a variety of instructional strategies
• Provides varied assessments
• Integrates technology
• Extends to other disciplines
• Cites references
• Meets accessibility and readability standards

Publisher Involvement:
WA Low -- OR High -- North Carolina Medium Connecticut Low

Other Notes
-- Oregon delayed implementation of the HS curriculum review to fall 2008.
-- CT is developing a new tool that can be used across multiple subjects.

Oregon:
Contact Info Jonathan Wiens
(503) 947-5764
Jon.wiens@state.or.us

Drew Hinds
(503) 947-5799
Drew.hinds@state.or.us

NC:
Charlotte Hughes
chughes@dpi.state.nc.us

CT:
Charlene Tate-Nichols
(860) 713 6757
Charlene.tate.nichols@ct.gov

Marlene Lovanio
(860) 713-6786
Marlene.lovanio@ct.gov

Proposed Categories:

The Mathematics IMR Advisory Group developed the following general criteria during their first work session. The group considers this list to be preliminary; and expects to review the overall review criteria before making a final decision on their recommended categories. The order of the categories presented here has no bearing on the relative value or weights of the categories. The Advisory Group has not yet prioritized this list.

General Category Description
... Content/Standards Alignment
..=..Mathematical concepts, skills and processes are in alignment with revised state mathematics standards

... Structure & Organization (Scope & Sequence)
..=.. Overall program and design. Includes scope and sequence, appropriate use of technology.

... Instructional Design
..=.. Underlying assumptions of teaching and learning that reference research-based design. This means that the materials should be based on solid research about effective teaching and learning strategies, not that the material was used in a research study.

... Student Experience/Nature of Tasks
..=.. Tasks lead to the development of big ideas, and present opportunities for students to think about their thinking, develop both skills and understanding, and apply multiple strategies to solve real world problems.

Defined by “Making Sense” by Thomas Carpenter (Note to Porsche – get this info from Tony).

... Assessment
..=.. Tools for teachers and students to formally and informally evaluate learning and guide instruction.

... Professional Development
..=.. Support for teachers that is embedded in the instructional materials to assist them in teaching the content and standards. Instructional materials provide suggestions for teachers in initiating and orchestrating mathematical discourse.

... Equity and Access
..=.. Support for ELL, unbiased materials, support for gifted and talented students, differentiated instruction, diversity of role models, parent involvement, intervention strategies, quality website, community involvement ideas



Proposed Measurement Criteria

For each of the general categories listed above, subgroups identified proposed measurement criteria. They recommend a consistent measurement scale for each item, like a 4-point Likert scale.

Content/Standards Alignment

• Washington state math standards designated for the specific course and/or grade level/band are addressed
• Mathematics concepts are developed for conceptual understanding, computational fluency and problem solving ability
• The content has a coherent and well-developed sequence with sufficient practice
• Mathematics content is rigorous and accurate, with few errors of fact or interpretation
• Math concepts are addressed in a context-rich setting

Structure & Organization (Scope & Sequence)

• Does the scope and sequence of materials match the state standards?
• Builds from and extends concepts previously developed
• Intentional review & application of previously taught skills and concepts
• The program is organized into units, modules or other structure so that students have sufficient time to develop in-depth major mathematical ideas
• The instructional materials provide for the use of technology with reflects 21st century ideals for a future-ready student and school

Instructional Design

• Instructional model is clearly described in the materials so that teachers can make decisions about instruction consistent with the model
• The research base is evident in the materials

The subgroup working on Instructional Design identified other issues to consider. These are not formatted as evaluative criteria, but rather, they identify additional discussion that needs to occur. Instructional design is an “umbrella”. Teachers need to understand the design and research it is based upon so that they know what they may modify without “messing up” the research-based design.

It is important to identify the underlying assumptions that guided the development of the program by addressing these questions:

• What were the teaching and learning assumptions?
• Were these assumptions research-based?
• Is the development of the program consistent with the underlying assumptions?
• Are the assumptions based on philosophically underlying broad research?
• Upon what work were the models built?

Student Experience/Nature of Tasks
• Tasks lead to the development of both skills and understanding
• Tasks build upon prior knowledge
• Tasks lead to problem solving for real-world problems
• Tasks lead to the development of big ideas
• Tasks require students to think about their own thinking
• Tasks are of an open nature with multiple solutions where appropriate
• Tasks allow for differentiation based on student need (e.g. ELL, talented, struggling)
• Tasks promote classroom discourse
• Tasks support mathematical literacy

Assessment

• The assessments are frequent enough to guide student learning
• There are opportunities for student self-assessment of learning
• The assessments use a balanced approach, incorporating concepts, problem solving and procedures
• There are a variety of assessment types, including formative, summative and diagnostic
• Assessments are reliable and valid
• The assessments are bias-free and equitable
• Assessments match content goals and objectives
• High-stakes practice assessment formats (e.g. WASL format) are included
• The assessments engage the student in evaluating their own performance (communicates to students the learning target and gives them tools to let them know where they are in the sequence)
• Includes guidelines for performance levels necessary before “moving on”
• Includes screening/diagnostics tools that allow teacher to identify appropriate starting points
• Who is assessed? Note to Porsche – Follow up on these two to identify what the group meant
• Rubrics - quality

Professional Development

• The instructional materials provide suggestions to teachers so that in tasks and lessons teachers can help students to:
--o.. Provide suggestions on how to help students access prior learning as a foundation for further math learning
--o.. Learn to conjecture, reason, generalize and solve problems
--o.. Connect mathematics ideas and applications to other math topics, other disciplines and real world context
--o.. Develop a responsibility for learning and self confidence
--o.. Work together to make sense of mathematics

• Instructional materials help provide support to teachers in supplementary mathematical content knowledge
• Instructional materials help teachers anticipate common student misconceptions
• Background information is included so that the concept is explicit in the teacher guide
• The instructional materials identify typical misconceptions

Note to Porsche: Also address a way to measure what type and how much Professional Development a publisher offers to districts that adopt their materials. Ensure there is a reference check to verify publisher claims.

Equity and Access

• Materials have differentiated instruction
• Materials support intervention strategies
• Materials include support for students with learning disabilities
• Materials include support for gifted and talented students
• Includes examples of parent letters with explanations, key ideas and vocabulary for each unit
• Materials include free or inexpensive activities which can be done at home for each unit
• The program has a quality website for students and families
• Community involvement ideas for families are available
• The materials offer support for English Language Learners
• Materials are unbiased and relevant to diverse cultures
• There is a diversity of role models in problems and illustrations
• Materials are available in other languages


Next Steps

The Mathematics IMR Advisory Group intends to meet again in person to review the work from their first work session and to finalize their work. These steps remain:

• Distribute proposed categories and measurement criteria to other stakeholder groups to solicit input
• Make sure we have the right categories
• Review each proposed measurement criteria to ensure it is in the right category
• Identify areas where publishers can provide input comparing their materials to the proposed measurement criteria
• Clearly define the process for how OSPI will select three programs to recommend. Will it be based solely on the defined criteria, or will there be some level of review/decision making by a panel?
• Prioritize and weigh general categories
• Test the draft review tool