Public Disclosure ?? and the law..... OSPI still fails to follow the law You can read the Law here

pre-quell: we are still using the following narrow lame definition of math at OSPI:
----
What is Mathematics? - Mathematics is a language and science of patterns.

As a language of patterns, mathematics is a means for describing the world in which we live. In its symbols and vocabulary, the language of mathematics is a universal means of communication about relationships and patterns.

As a science of patterns, mathematics is a mode of inquiry that reveals fundamental understandings about order in our world. This mode of inquiry relies on logic and employs observation, simulation, and experimentation as means of challenging and extending our current understanding.
----
This extremely narrow poor definition is in part responsible for our current mess as Exploration and Inquiry describe the Cognitive model found in Project Follow Through to be the worst possible way to teach mathematics or for that matter much of anything. Check out the Cognitive Model of Exploration and Inquiry. Most of us will settle for math as a useful tool for living successful lives. Carpenters, Plumbers, Electricians, Home owners, Engineers,.. hey what is the job market like for Philosopher Kings these days... I mean other than at OSPI.
------------------------------

Before we start, it is important that all this Math Hub..Bub currently underway needs to have some connection to the LAW... OSPI has never been terribly keen on the LAW so it might be a good idea to refresh yourself with HB 1906 from 2007 and SB 6534 from 2008. I strongly recommend reading what should be happening.. do not just assume that what is happening would be funded by the laws or following the LAWS. The funding is provided by the legislature to follow the law.. do not assume that is what OSPI is doing. Check the BIG RED way below ... no surprise again the law is not being followed.

Here is the recently passed SB 6534 more relevant to the situation at hand
and here is HB 1906 from 2007 .. if OSPI had been able to follow HB 1906 there would have been no need for SB 6534..

My apologies as this is no where near finished.
Since there is as far as I know no public access to the current document under-construction by the Instructional Materials Review criteria panel, this will have to do for now. It is better than the public got for draft #1. [A sad something beats nothing] Draft #2 when polished will in large part be driving the recommended text selections. I am posting this in its current unfinished form... What is largely unfinished here are my responses in Blue.. I have little idea of how process produces improvement for anyone except those employed to process. Anyway here is what I can put up given formating difficulties of tables etc.

Executive Summary

The purpose of this report is to document the recommendations of the Mathematics Instructional Materials Review (IMR) Advisory Group and other key stakeholders, including the State Board of Education Math Panel; and to outline the expected process for reviewing mathematics instructional materials.

Does Draft #2 miss that the major task at hand is to select texts that will allow Washington Students to have the opportunity to become internationally competitive in Math?

The work of these key stakeholder groups is crucial to the success of the instructional materials review project. OSPI has committed to an inclusive process that actively solicits information and advice from many stakeholder groups. It is essential that the review methodology and process measures the appropriate factors, and takes into account a broad range of instructional materials-related criteria that contribute to effective teaching and learning. Ultimately, OSPI will recommend three core/comprehensive mathematics texts at the elementary, middle and high school level. It is imperative that the process, evaluation and final recommendations support the success of all students in the Washington K-12 system.

The process has clearly once again become the goal (so common with OSPI process trumps content again). The selection of the best math materials for Washington students is at best a secondary priority.

Expected Outcomes

The following expectations guided the development of the criteria.

• Examine criteria and processes used by other states and districts within Washington to review mathematics textbooks. Successful? .. please list the successful ones in Washington ... and the designation of successful is given by ???

• Identify 5-7 general categories for use in the OSPI Instructional Materials Review that are well defined and based upon accepted research, such as the National Mathematics Advisory Panel Foundations of Success Report.
What does the term accepted research mean?

• Identify specific, measurable, efficient and valuable criteria within each of the categories to help us determine which three texts to recommend at the elementary, middle and high school levels.
How will the determination of efficient and valuable be determined?

• Identify why the criteria is important to measure, and develop ideas on how to measure those criteria.
What criteria is of value and how is this determined?

• Recommend relative weights for selected categories. Recommend threshold categories that curricula must pass before being considered further.
How is the weighting determined?

• It is vital to have the selection criteria unambiguous and measureable, and define relative weightings for categories. The evaluation criteria should be straightforward, and where possible, mapped to approved standards (ex: WA Revised Math Standards, NMAP). The instructional materials review process should include math educators, curriculum specialists, mathematicians, parents and industry representatives.

If such a collection of individuals should comprise the IMR review process, why was there no such diversity present on the IMR criteria panel, which was selected privately and unilaterally by OSPI without even a public awareness of this team’s existence?

Figure 1. Content Alignment is the key consideration in the Washington Mathematics Instructional Materials Review.

This is really vague. Aligned with what? The three primary source documents? These do not even align with each other. Does alignment need to be at the specific grade level? A major concern is that although the WA Math Standards are referenced as a primary tool for alignment the secondary sources are strongly Standards-Based and those produced the defective programs in use currently.

Stakeholders ( Stakeholder generally implies a diverse group of individuals likely to be impacted by the decisions - but not here where the term refers to the fairly homogeneous group selected by OSPI ) recommend that the review process be structured in two parts.
• Part 1 is Content/Standards Alignment. All curricula must meet a certain (to be determined) When will this be determined? by what person or group? based on what alignment? threshold in order to be considered further as one of the three recommended core/comprehensive texts.
• Part 2 will consider other criteria (determined and weighted by what group?) in five categories. Note that the second stage categories have different weights, as shown in Figure 2.

Research Overview

Research
One of the goals of the Mathematics Instructional Materials Review Project is to design a review process and methodology that is grounded in current research and utilizes practices gleaned from other states and districts, which have recently completed successful mathematics instructional materials review projects. (successful as judged successful by what tool or data?)
Stakeholders used ( used as in past tense, who were these stakeholders that already used?... Oh that's right the homogeneous group of administrators selected by OSPI because of process expertise..) the following research and publications to finalize the category and criteria recommendations.

• Washington Revised Mathematics Standards (4/08)

• National Mathematics Advisory Panel (NMAP)
Foundations for Success

• NCTM Curriculum Focal Points .. The Focal Points were not even mentioned in draft#1 ..think that is odd .. the Focal Points are mentioned on page 2 of SB 6534 and on page 2 of HB 1906. Then again the law is for others not OSPI.
--- the only reason the Focal Points made draft#2 is because a few of us on the SBE MAP said WHAT??? ... A real testament to the content knowledge of the 22 IMR criteria panel members that there was no mention of the Focal Points in Draft #1. Does anyone read the law they are charged with fulfilling???

The following additional research and publications were used as secondary sources to inform the process.

• Math Educators Summary of Effective Programs

• Park City Mathematics Standards Study Group Report

• Framework for 21st Century Learning

• How People Learn: Brain, Mind, Experience and School

• How Students Learn: Mathematics in the Classroom

• NCTM Principles and Standards for School Mathematics

• Choosing a Standards-Based Mathematics Curriculum – Chapter 6: Developing and Applying Selection Criteria

• Choosing a Standards-Based Mathematics Curriculum – Appendix: Sample Selection Criteria

I have great concerns about the entire direction of this IMR process thus far. There is a lot of focus on process with inadequate reflection upon the eventual product. Thus far this bears a great resemblance to the processes that have produced more than a decade of substandard materials. These materials in both Washington State and nationally became more and more aligned with NCTM Standards and the results became more and more unsatisfactory on International tests.

Given that a goal is to produce an internationally competitive mathematics education in Washington State, why is such weight placed on using so many instruments that produced the current failing system? I put particular emphasis on the desire for a Standards-Based Curriculum. Most of the documents referenced in - Appendix: Sample Selection Criteria, produced particularly ineffective programs. It seems likely that the same failures will occur for us if these are used. In particular the Evaluation Criteria from the U.S. Department of Education’s Expert Panel on Mathematics and Science gave rise to the 1999 Exemplary and Promising Math Programs that have yet to produce evidence of positive academic results. The 1999 materials as selected were all NCTM Standards-Based and likely are at least partially responsible for the USA’s decline in mathematics internationally. As the US has slipped internationally, it is very apparent that we are the only nation advocating for the positions and practices promoted by the Standards Based thinking. The appendix also makes reference to the San Diego City Schools k-8 instrument. This produced another extremely poor text series. When many Schools in California switched curricula to meet the new California Standards adopted in 1998, San Diego city schools did not. In the Hook-Bishop-Hook study that compared various districts in California, it was very clear that San Diego and LAUSD, which stayed with their earlier Standards Aligned texts, performed poorly.

Standards Aligned is NOT synonymous with likely to achieve positive academic results. It appears that this IMR criteria document draft#2 has a mistaken belief that Standards-Based equates with effective in bringing about internationally competitive results.


In addition to the research review, participants examined mathematics instructional materials review processes tools from five states (OR, NC, CT, IN and CA) and two Washington school districts (Edmonds, Vancouver). They identified good ideas to adopt from these sample processes and tools.
How did an IMR criteria panel that had little math content knowledge (so little that draft#1 did not include the NCTM Focal Points) determine what constituted a good idea? A summary of a few of the state comparisons is shown below.

CHART SKIPPED

Proposed Categories
Part 1: Content/Standards Alignment
Part I is a review of the alignment of the core/comprehensive instructional materials to the revised Washington Mathematics standards. Materials that meet a to-be-determined threshold of alignment with state standards could be considered for inclusion in the list of three recommended mathematics curricula.
The Content/Standards Alignment part of the review process would determine to what degree the mathematical concepts, skills and processes are in alignment with revised state mathematics standards. Reviewers would look for evidence that each Washington state standard core process, content and additional key information was met in the expected grade level.

BIG RED ALERT!!!... the following is not mentioned in either HB 1906 or SB 6534 ... so how did to identify the necessary supplementation for existing materials to meet state standards become one of the goals .. did the Gov. and the legislature wish to pay for this sorry patch job ??? if so you would think it would be in the LAWS!!!

One of the goals of the Content/Standards Alignment process will be to identify the necessary supplementation for existing materials to meet state standards.

All Right we have finally arrived... at the real matter at hand. Under Dr Bergeson's guiding hand 80% of the students in the State are now saddled with the Connected Math Project materials at the middle school level. CMP and CMP2 have little if any focus on Authentic Algebra as described by NMAP.

Connected Math was far and away the OSPI most aligned text at the middle level no contest over second place.
See page 18
CMP = strongly aligned 63% adequately aligned 15%...
second place
MathThematics = strongly aligned 39%<> adequately aligned 24%...
Again little if anything other than Standards Based curricula were rated.

Again the criteria is being Standards-Based which is apparently in contradiction to MSSG 2004 and in contradiction to attaining success on the international math stage as well as becoming competent enough to enter a mathematically and academically challenging course of technical study at the university level....
So the job of the IMR is to fix and repair the defective Connected Math program...

It is now little wonder that Project Follow Through was not even mention as a source resource for although it is the guide to mathematical success ... it runs counter to the prejudices of the profession ... the education hierarchy wants to do what they want to do ... results do not matter..

If PFT had been a recommended source then the two elementary text series that are in use in 66%+ of Washington schools would be exposed as fakes.

Now it is the job of the IMR to fix and repair Everyday Math and TERC/Investigations... Did I miss something here ...wasn't the idea to select texts that would allow our students access to an internationally competitive course of study in math?

Guess not... The idea is to produce a patchwork cover-up for failed decisions from the last decade...

NMAP and the Focal Points both point to greatly restricting the number of topics at each grade level. Everyday Math is a terrible text series in that regard. It does not teach a few topics to mastery but instead does the exact opposite of what is recommended. Take a look at the following Everyday Math Topics and tell me about supplementation.

Little wonder that Dana Center Executive Director Uri Treisman, the big Everyday Math booster is against supplementation there is no room for it. He says so during minute five on the Video linked to his name.
I would greatly appreciate it if OSPI would stop spending Tax dollars to carry out tasks that are not mentioned in the law. Especially as unreasonable an undertaking as Supplementation of bloated defective incoherent materials.

We have exactly the right folks secretly selected by OSPI on the IMR criteria panel..
The majority of these 22 know not enough math content to make the judgments necessary to select internationally competitive materials .... but having been responsible for the incoherent defective math materials selections of the last decade they should have the motivation necessary to attempt turning the sow's ear into the silk purse...

My thoughts are Example Based Instructional Materials as used in the successful math nations. For more on this see the 2004 MSSG...from the introduction:

The essence of mathematical learning is the process of understanding each new layer of knowledge and thoroughly mastering that knowledge in order to be able to understand the next layer. ... You can forget that with the OSPI planned patch job!!!!



Part 2: Other Factors
Part 2 of the review process would examine the following categories:
Category Description
Program Organization and Design Overall program and design. Includes scope and sequence, appropriate use of technology. Content is presented in strands, with definitive beginnings and endings, and not in spirals. The material is logically organized, and includes text-based tools like tables of contents and indexes.
Student Learning Tasks lead to the development of core content and process understanding. They present opportunities for students to think about their thinking, develop both skills and understanding, and apply multiple strategies to solve real world problems. Opportunities exist for students to build computational fluency, number sense and operations.
Instructional Planning and Professional Support Support for teachers that is embedded in the instructional materials to assist them in teaching the content and standards. Instructional materials provide suggestions for teachers in initiating and orchestrating mathematical discourse. Includes key information about content knowledge to help teachers understand the underlying mathematics. Materials help demonstrate typical student misconceptions and provide ideas for helping address them.
Assessment Tools for teachers and students to formally and informally evaluate learning and guide instruction.
Equity and Access Support for ELL, unbiased materials, support for gifted and talented students, support for students with disabilities, differentiated instruction, diversity of role models, parent involvement, intervention strategies, quality website, community involvement ideas.
Figure 2. Proposed category weights for Part 2 of the Mathematics Instructional Materials Review. Note that Content Alignment is not shown in this chart. Content Alignment is a threshold category, meaning that curricula must meet a to-be-determined percentage of agreement before the material can be considered for possible inclusion in the three recommended core/comprehensive texts.
Proposed Measurement Criteria
For Part 1 (Content/Standards Alignment), there will be a 3 point scale (0-2, corresponding with No, Partial, Yes) for each performance expectation. The criteria are the Washington Revised Mathematics Standards (4/08).
Part 2 will use a consistent measurement scale for each item; a 4-point Likert scale (strongly disagree, disagree, agree, strongly agree). For each of the Part 2 categories listed above, stakeholders identified proposed measurement criteria.
Part 1: Content/Standards Alignment
Part 1 will ensure that Washington state math standards designated for the specific course and/or grade level/band are addressed. It will ensure that the mathematics content within the program is rigorous and accurate, with few errors of fact or interpretation. A sample rating form is shown below.

Figure 3. Sample rating form for Content/Standards Alignment Review.
Note to reviewers. I wonder if we should have the Content/Standards Alignment be both a threshold category AND have a weighting like the Part 2 review categories. For example, what if Content/Standards Alignment were weighted at 70% and the other categories distributed across the remaining 30% in rough proportion to the weights proposed in Figure 2?
Part 2: Other Factors
Program Organization and Design
1. The scope and sequence of materials matches state standards
2. The content has a coherent and well-developed sequence
3. Program includes a balance of skill-building, conceptual understanding, and application
4. Builds from and extends concepts previously developed
5. Intentional review & application of previously taught skills and concepts
6. Some but not all tasks are of an open nature with multiple solutions, others with a correct and verifiable answer
7. The materials help promote classroom discourse
8. The program is organized into units, modules or other structure so that students have sufficient time to develop in-depth major mathematical ideas
9. The instructional materials provide for the use of technology with reflects 21st century ideals for a future-ready student and school
10. Support materials provided, such as electronic learning resources or manipulatives, are an integral part of the instructional program and are clearly aligned with the content
11. Objectives are written from the student, not the teacher perspective
12. Instructional materials include mathematically accurate and complete indexes and tables of contents to locate specific topics or lessons
13. The materials have pictures that match the text in close proximity, with few unrelated images
14. Materials are concise and balance contextual learning with brevity
15. Mathematics concepts are developed for conceptual understanding, computational fluency and problem solving ability, including how to address non-routine problems
16. The program contains an balance of skill building, conceptual understanding and application
17. The materials include formulas and teaches standard algorithms

Note to reviewers: I think there are too many criteria in this category and recommend eliminating those that are either difficult to measure, are redundant with other criteria, or have less value in relation to the remaining criteria. Recommend a target maximum of 10 criteria in this category. 1 & 2 seem redundant. 3 & 4 seem redundant, 3 and 16 seem redundant. 15 and 17 might be accounted for in the Content Alignment/Standards Review. 11 might be best placed in Student Learning. I am concerned about ambiguity and inter-rater reliability on several of these elements.

Student Learning
1. Tasks lead to the development of core content and process understanding
2. Tasks build upon prior knowledge
3. Tasks lead to problem solving for abstract, real-world and non-routine problems
4. Tasks require students to think about their own thinking
5. The program provides opportunities to develop students’ computational fluency using brain power rather than technology
6. Tasks occasionally use technology to deal with messier numbers or help the students see the math with graphical displays
7. The program promotes understanding and fluency in number sense and operations
8. The program leads students to mastery of rigorous multiple-step story problems
9. The materials build students’ understanding of standard mathematics terminology/vocabulary

Instructional Planning and Professional Support
1. The instructional materials provide suggestions to teachers so that in tasks and lessons teachers can help students to:
a. Access prior learning as a foundation for further math learning
b. Learn to conjecture, reason, generalize and solve problems
c. Connect mathematics ideas and applications to other math topics, other disciplines and real world context
d. Develop a responsibility for learning and self confidence
2. Instructional materials provide support to teachers in supplementary mathematical content knowledge
3. Background information is included so that the concept is explicit in the teacher guide

4. Instructional materials help teachers anticipate common student misconceptions
5. The instructional materials identify typical student misconceptions
6. The materials can be used by a wide range of teachers with different teaching styles
7. The materials support a balanced methodology including direct instruction, example-based instruction and discovery
8. Math concepts are addressed in a context-rich setting (giving examples in context, for instance)
9. Teacher’s guides are clear and concise with easy to understand instructions

Note to reviewers: Multi-part items are difficult to deal with in scale design, and are subject to variable interpretation. Consider simplifying #1. Numbers 2 & 3 seem redundant. Numbers 4 & 5 seem redundant. Numbers 6 & 7 seem redundant.
Assessment
1. The program provides regular assessments to guide student learning
2. There are opportunities for student self-assessment of learning
3. Assessments reflect content and process goals and objectives
4. The program includes assessments with multiple purposes (formative, summative and diagnostic)
5. Assessments include multiple choice, short answer and extended response.
6. Rubrics or scoring guidelines accurately reflect learning objectives
7. Rubrics or scoring guidelines identify possible student responses both correct & incorrect
8. Accurate answer keys are provided
9. Equity and Access
10. The program provides methods and materials for differentiating instruction (students with disabilities, gifted/talented, ELL, disadvantaged)
11. Materials support intervention strategies
12. Materials, including assessments are unbiased and relevant to diverse cultures
13. Materials are available in a variety of languages
14. The program includes easily accessible materials which help families to become active participants in their students’ education (e.g. “How You Can Help at Home” letters with explanations, key ideas & vocabulary for each unit, free or inexpensive activities which can be done at home, ideas for community involvement)
15. The program includes guidance and examples to allow students with little home support to be self-sufficient

Next Steps
These steps remain:
1) Solicit feedback from reviewers on Draft #2
2) Clearly define the process for how OSPI will select three programs to recommend. Will it be based solely on the defined criteria, or will there be some level of review/decision making by a panel?
3) Review each measurement criteria with the following framework in mind:
a) Is the criteria measurable using professional judgment and/or available evidence? (measurability)
b) Is the criteria clear enough to avoid multiple interpretations? (specificity, variability)
c) How will this help inform our recommendation process? (value)
d) Is the criteria in the most logical category? (organization)
e) Does each criteria cover just one concept? (atomicity)
4) Remove redundant criteria
5) Test the draft review tool

Bibliography
(To be completed in 3rd draft. It will contain bibliographic references to primary and secondary sources in the Research section.)