ADDITIONAL RECOMMENDATIONS OF THE CAEP COMMISSION

The CAEP Commission was also charged to determine what information would be reported to the public, how often programs are reviewed and monitored, and what the levels of accreditation would be.

Commission members were guided in their work by analyses of recent trends and promising practices in accreditation.[i]  In particular, the members put the most weight on student learning outcomes, referring to both candidate outcomes and P-12 student outcomes.  In addition, however, Commissioners included consideration of program characteristics that would be expected to ensure and enhance quality, and that would support fair treatment of candidates.

Indicators (1) through (4) are in-service measures of quality that are broadly consistent with recommendations from the National Research Council[ii] regarding the incorporation of value-added measures, satisfaction and employment milestone measures from employers, and preparation satisfaction from program completers.  Indicators (5) through (8) are intended to ensure the fair treatment of candidates and completers, so that candidates accepted to an educator preparation program would have specific information about chances for completion, licensure, finding a job in field for which they prepare, and student loan default rates.

As seen by the Commission, these data and their annual review serve a variety of purposes.  They are incentives for providers to routinely gather, analyze and report critical data about their programs as one means for public accountability and transparency.  Such data encourage more in-depth evaluation, self-interrogation, and reporting on the full breadth of standards and components.  Employers and prospective applicants for admission need this kind of information in user-friendly, transparent, forms.

For CAEP, itself, there are many uses:

• The data will become the foundation of a national information base that increases in value over time.
• The data can send an alert to CAEP that trigger points have been exceeded so that closer inspection of a provider’s preparation program should be scheduled.  (See the explicit provision in the recommendation, above, for indicators of change that would prompt investigation to initiate (1) adverse action that could include revocation of accreditation status or (2) recognition of eligibility for a higher level of accreditation.)
• They will be a source of information for CAEP’s annual report, will complement descriptive measures for all accredited providers, facilitate monitoring of trends over time, allow analysis of preparation patterns for different subgroups of institutions (e.g., state, regional, urban, rural), and be a resource for identifying benchmark performances.

The database will enable CAEP to report on the progress of continuous improvement not just for an individual provider but for educator preparation across all accredited providers.

The Commission proposes four levels of accreditation decisions.  The first three would be “denial,” “probationary” and “full accreditation.”  The fourth or highest level would be the Commission’s vision for an exemplary or “gold” accreditation.  Such a designation would break a new path in accreditation, giving visibility to attainment of a superior level of performance.

The Commission recommends that CAEP establish “threshold” classifications that define evidence from “beginning” to “leading” for each component.  The threshold would be set on the basis of CAEP’s experience in identifying and updating evidentiary measures that represent best current practice in provider performance.  Threshold classifications would be defined by rubrics that describe both characteristics of the evidence and markers of performance.  Each component of each standard would contribute to the composite evaluation for the standard.

The generic classification definitions are illustrated in the following example:

• beginning; a plan is in place for gathering data or identification of metrics and initial data collection has begun
• developing; actual data collection has been completed for at least a year  and studies to examine and verify the data are underway
• operating; studies to examine and verify the data are completed, there is some reliability evidence, and data are available for more than one year.  Data demonstrate performance markers meeting a threshold requirement, and data have been used for at least one cycle of evaluation, analysis, and subsequent improvement decisions.
• leading; data are available for several years, with completed validity and reliability information about the use and interpretation of the data.  The actual values of the data are higher than for the “operating” threshold, and data are routinely used to evaluate and improve preparation.

A CAEP decision to award “full accreditation” would signal that the provider’s efforts and results substantially comply with the rigorous levels recommended by the Commission.  Accreditation could be achieved if there are some areas where component evidence fails to reach the set threshold, with two exceptions.  Meeting the “operating” threshold criteria would be required for:

• all components of standard 4 on program impact, and
• components 5.4 and 5.5 on continuous improvement.

Achieving an “exemplary” CAEP accreditation decision would signal that the provider’s evidence meets the “leading” classification for a specified number of standards, including standard 4 on program impact and standard 5 continuous improvement components.

Commissioners are aware that program impact data are not universally available.  Asking providers to develop data collection systems individually raises challenges of costs, efficiency, and comparability of data.  In the short term, CAEP must work with states and providers to develop the necessary information metrics and systems to gather data.  CAEP collaboration with States and providers, and federal support through initiatives in statistics, research, and resources are necessary.

The qualities of evidence might be improved through actions of the provider, with the maturing of its quality assurance system and use of data for continuous improvement.  However, Commissioners anticipate that, over time, the information available for accreditation decisions will grow much stronger, permitting a gradual shift in CAEP’s evidentiary expectations.   The Commissioners especially draw attention to the statement in President Cibulka’s covering letter for this report:

“As the knowledge base improves, CAEP standards and the evidence we use to measure performance against those standards can be revised to reflect what truly matters in producing effective teachers who improve P-12 student learning.”

The anticipated revisions over time will enable CAEP to rely more on program outcomes and performance results, and less on inputs and processes to make its judgments.

The Commission proposes that CAEP undertake decisive steps to design and test this approach  for exemplary accreditation over a specific timeline.  The Commission’s vision for exemplary accreditation status may be implemented in a variety of ways, but it must be merited by performance beyond the rigorous expectations for full accreditation that the Commission is recommending, with the aspiring institutions displaying evidence that they have achieved a good number of “leading” evidence threshold ratings.  A two level review process in which the second level would employ a special panel of peers to evaluate the higher performance expectations might be considered as a means of awarding exemplary status.

The CAEP design and test initiative for awarding exemplary status should engage appropriate technical and teacher education experts.  It should refine and calibrate rubrics to guide designation of exemplary or “gold” level accreditation, and conduct validity and reliability studies of the judgments inherent in those decisions.

While the system for reaching “exemplary” level accreditation decisions is under development, the Commission recommends that the CAEP Accreditation Council consider an interim process for recognizing truly outstanding preparation programs.