Session I: The National Evaluation
Bob St. Pierre/ Fumiyo Tao

Fumiyo Tao presented the evaluation segment at both the Washington, D.C. and San Antonio, Texas Academies. Bob St. Pierre presented the evaluation segment at the San Francisco, California Academy

The presentation focused on five particular areas:

1. Even Start National Evaluation Review.
2. Evidence From the Past Evaluations.
3. Updates on the Current National Evaluation.
4. ESPIRS Revisions. Even Start Performance Information Reporting System
5. The Experimental Design Study (EDS).

1. Even Start National Evaluation Review

Bob reviewed the three four-year evaluation cycles that have two parts. One is the ESPIRS, which deals with descriptions of the program. This is useful information for Congress and well as the local projects and has been called the universe study. All projects fill out these forms and provide forms to the national evaluation

The second part of the evaluation is comprised of a sub-set of original projects. The sub-set study attempts to focus on questions such as: Do the children learn? Do parents learn? Do parents get jobs? The purpose to measure the outcomes of those sub-set projects.

2. Evidence From the Past Evaluations 

Some interesting outcomes from these project evaluations are:

• Projects serve and continue to serve very needy families.
• Projects have significantly increased the amount of instruction offered 
  in all three of the core service areas.
• On average, more hours of instruction offered is related to more hours 
  of participation by children and adults.
• Adults and children with high levels of participation have larger 
  learning gains.
• Even Start children and parents make gains; but data are not yet 
  available to attribute the gains to Even Start.

Bob noted that Pat McKee has been supporting these ideas over a number of years and has continued to push for such efforts. He also emphasized the concern he has for seeking additional information to identify those Even Start factors, which contribute to the gains.

3. Updates on the Current National Evaluation

The local programs should have received project Profile Reports during mid to late June 1999. This is information gained from the summer of 1998. Reports should be helpful and if not ABT should be notified about individual concerns. These are meant to be a helpful tool to increase understanding of the programs. These reports allow comparisons of other projects with similar characteristics. Suggestions were made to include some project dollar figures to allow additional comparisons.

A revised set of ESPIRS reports will be available in the fall. These reports are 
based on the comments and complaints from projects which feel there are areas that could be done better. In addition, the Department of Education and ABT Advisory Panel is looking at areas that could be improved as well as new issues to be studied. These changes should be ready in October 1999. Requests were made for the new forms to be available early however ABT felt there was less likelihood of confusion if the forms were provided after the current ones were completed. The new forms will hopefully be available for the National Even Start Conference. 

4. ESPIRS Revisions (Even Start Performance Information Reporting System)
.
A number of examples of new questions added to the ESPIRS were presented such as:

• How do the adult education services in your project work with welfare 
  reform and how are they coordinated with it?
• Does staff from both adult education and parenting education share 
  information on individual participants?
• Does staff from both of those areas share student information formally 
  or informally?
• Do they participate in joint training?

Additional forms designed to account for the number of service hours for both 
adult education and early childhood education on a monthly basis were introduced. If program is rather consistent this should not be a great problem. The benefit for such information is allowance for better understanding of what happens over the down months or during the summer.

Collaboration was another issue of concern on the new forms. It was felt that the information being collected did not accurately reflect the real meaning of collaboration and therefore a more inclusive list of possible collaborative activities was included. This list included such things as cash contributions, hours of operation, shared facilities and staff sharing. 

5. The Experimental Design Study (ESD)

The experimental study is a best attempt to learn what difference Even Start 
makes in the lives of the families. It is an attempt to gather data and show on paper what programs feel, see, hear and better understand what is going on in projects. In conjunction with the Department of Education , ABT came up with plan to pick 20 programs from across the country to determine the effectiveness of Even Start around the country. 

A variety of programs offering high intensity services area being recruited. The 
inclusion of programs both with and without ESL (English Second Language) services were felt to be an important part of this study. Programs with high service intensity have been a concern over a number of years. The issue of "more is better" is to be part of the study. The project wants to select programs with high service providers. High levels and moderate levels of service are being defined and targeted. The goal is to have a total of 600 families with 400 families receiving Even Start services and the remaining group of 200 families as a control group. The control group issue has been an important ingredient missing from other evaluations.

Bob went on to discuss the benefits of being a part of such a study which included sharing in the outcomes of good research, family stipends, and the notoriety of being in such a project. He continued his presentation with an in-depth description of the Experimental Design Study, which includes the various measures to be used, the time frame and schedule of activities.

In conclusion ABT Associates encouraged the participants to communicate with them regarding the study and any of the ESPIRS forms which were being sent out to projects. ABT's goal is to collect the best and most useful information concerning Even Start.

ABT Associates provided each participant with a set of handouts, which were 
followed, for the presentation as well as information as to:

• How to Obtain Summaries of ESPIRS Data
• How to work with the ESPIRS Data Summary Module
• Examples of ESPIRS reports.
• How to Access ESPIRS Data Files
• Contact numbers for Feedback of National Evaluation Data to Local Even Start Projects – http://www.ed.gov. or 1-877-433-7827