Determining and Corroborating Eligibility Decisions

The AEPS Test is a curriculum-based assessment (CBA) measure designed to yield a comprehensive and detailed picture of children’s behavioral repertoires. The original purposes of the AEPS Test were to 1) determine a child’s present level of functioning across six developmental areas, 2) provide content for formulating goals and objectives for individualized family service plans (IFSPs)/individualized education programs (IEPs), 3) guide subsequent individualized intervention activities, and 4) monitor child progress toward selected intervention goals over time. These important purposes dictate that the AEPS Test items be selected based on their educational and treatment relevancy—a hallmark of this CBA.

In the ensuing years since the development of the AEPS, growing concern about the methods used to determine children’s eligibility for services has arisen. From the inception of early intervention/early childhood special education (EI/ECSE) services, the predominant way to ascertain children’s eligibility has been to administer standardized, norm-referenced tests. Contrary to the items selected for the AEPS Test (i.e., items with high educational and therapeutic value and relevancy), items on standardized, norm-referenced tests are chosen based on their ability to discriminate between children of different chronological ages and are assigned age-norms in order to allow comparisons between children’s performances with their chronological age peers. Such comparisons permit determining the degree or percent of delay children may be from their chronological age group; however, they often have little relevance for developing IFSP/IEP goals or intervention content.

It is not clear whether most state agencies adopted criteria for receiving EI/ECSE services that rely on such comparisons because these are the type of data generated by standardized, norm-referenced tests or because such comparisons seemed most appropriate for determining eligibility. Whatever the reason, most state agencies require that children’s performance be so many standard deviations from the norm or a specified percent delay in order to be eligible for services (Bricker, Yovanoff, Capt, & Allen, 2003).

The use of standardized, norm-referenced tests to determine eligibility has come under increasing criticism because, as many experts argue, their use is contrary to recommended practice for a number of important reasons (Neisworth & Bagnato, 2004). Perhaps most important of these reasons is that outcomes from these traditional assessments offer little or no assistance in the development of intervention activities and therefore do little to improve services to children and families. Consequently, valuable time and resources are expended in the administration and interpretation of assessments that contribute very little to the welfare of children and families.

DEVELOPMENT OF CUTOFF SCORES FOR THE AEPS TEST

Establishing AEPS Test cutoff scores for eligibility decisions required the following steps:

1. collecting AEPS Test performance and age data from a group of typically developing children and a group of children already determined eligible and who were receiving EI/ECSE services; 2) completing Item Response Theory (IRT) analyses of the groups’ AEPS Test results for Level I: Birth to Three Years and Level II: Three to Six Years; and 3) creating tables that show the established cutoff scores for the six developmental areas of the AEPS Test by age interval for the AEPS Test Birth to Three Years and Three to Six Years.

Study Sample and Data Collection Procedures

Date of birth and AEPS Test performance data (i.e., completed AEPS Child Observation Data Recording Forms [CODRFs]) were collected on two groups of children (N=1,381). The first group was composed of typically developing children who ranged in age from 3 to 66 months (N=719). Typically developing was defined as the child having no history or no current evidence of a developmental delay or disability. AEPS Test performance data were also collected on a second group of children who were currently receiving services under the Individuals with Disabilities Education Act (IDEA) Amendments of 1997, PL 105-17, for a diagnosed disability (N=662). For the group of children with disabilities, the chronological ages also ranged from 3 to 66 months. Data were collected using the second edition of the AEPS.

The study sample was obtained by contacting community-based programs that offered educational or child care services to a range of young children. Data were collected from programs located in eight different states. The CODRFs were completed for participating children by the staffs of EI/ECSE and child care programs. Training background and experience levels of staff differed across and within programs; however, the majority of staff completing the AEPS Test protocols had received some training on how to use the AEPS.

Completed CODRFs were coded by identification numbers assigned to each child. In addition, each test protocol noted the child’s birth date and status (i.e., typical or eligible).

DATA ANALYSES TO ESTABLISH AEPS TEST CUTOFF SCORES

IRT modeling procedures were used to determine AEPS Test cutoff scores for the typical sample. IRT identifies psychometric properties of the AEPS Test (or any test) that are essential to understanding measurement error at specific levels of performance. A hallmark of IRT is the assessment of conditional measurement error (i.e., conditional on the ability level; Kolen, Zeng, & Hanson, 1996; Lord, 1980). All psychological measurements have errors, and similar to all tests, the AEPS functions better at some performance levels than others. Conditional measurement error is attributable to the specific objectives that compose the test. Note the number of goals achieved is translated into age because it is assumed (and tested) that goals achieved are linearly correlated with age.

The IRT analyses were conducted separately for the AEPS Test Level I and Level II and are described in detail in Bricker, Yovanoff, Capt, and Allen (2003) and in Bricker, Clifford, Yovanoff, Waddell, Allen, Pretti-Frontczak, and Hoselton (in press). The IRT-derived AEPS cutoff scores for each developmental area for Level I and Level II are contained in the following tables:

Table 1: AEPS Test cutoff scores by developmental area and age intervals

Note: AEPS Test eligibility cutoff scores are not provided for the 0-3 and 67-72 month intervals because sufficient data were not available to derive valid cutoff scores.

Table 2: Correlations between AEPS Area Goal Scores and chronological age

Once the cutoff scores were determined for each age interval, it was possible to determine how accurately the cutoff scores identified the children in the sample as eligible. Children’s eligibility status was determined by whether or not they were receiving IDEA services (i.e., those receiving services were classified as eligible while those children not receiving services were classified as typical). Once children’s status was determined, a comparison was made between their AEPS Area Goal Scores and the cutoff scores. Children with AEPS Area Goal Scores above the cutoff score were classified as typical while children with AEPS Area Goal Scores at or below the cutoff score for their age interval in two or more developmental areas were classified as eligible.

Table 3: Calculated sensitivity and specificity for Level I and Level II by age intervals

PROCEDURE FOR USING AEPS TEST CUTOFF SCORES TO DETERMINE ELIGIBILITY

Using the AEPS Test scores to determine eligibility or to corroborate other findings is a straightforward and simple process. The process involves computing an Area Goal Score for each of the six developmental areas of the AEPS Test. The resulting six scores then can be compared with the cutoff scores for those areas for a particular chronological age interval. To use the cutoff scores to determine eligibility or corroborate findings from other sources, a three-step process should be followed.

Step 1: Determine the child’s chronological age in months.

The child’s chronological age should be determined by subtracting the date of birth from the date of the test used to calculate the Area Goal Scores and rounding up or down appropriately. When subtracting days, take into consideration the number of days in the particular month (i.e., 28, 30 or 31). Two examples are presented to clarify the procedure.

Example 1

In the first example, the child’s CA is 21 months. Because 15 or fewer days have passed into his twenty-second month (in this case, 10 days), the child’s CA is rounded downward.

Example 2

In the second example, the child’s CA is 62 months. On February 6th, the test date, 23 days have passed since the 15th of the previous month. Therefore, even though he has not yet reached 62 months, because more than 15 days have passed into his sixty-second month, his age is rounded upward to 62 months.

If the child’s age has been adjusted for prematurity, a similar adjustment should be made for the AEPS Test.

Step 2: Calculate the child’s Area Goal Score for each developmental area.

Once the child’s chronological age has been determined, the appropriate level of the AEPS Test should be completed for that child. When administering the AEPS Test for eligibility determination purposes, all goals should be scored. The goals that received scores of 2s and 1s should be summed for each of the six developmental areas to calculate the six Area Goal Scores. Scores on objectives should be ignored when calculating the Area Goal Score; they are useful for programmatic decisions but are not used to determine eligibility.

Step 3: Compare the child’s Area Goal Score by area with the cutoff scores for each area.

Once the Area Goal Scores are calculated, the examiner is ready to use the cutoff scores contained in Table 1. The examiner must first locate the age interval that corresponds to the child’s chronological age in order to locate the appropriate cutoff scores for comparison with the Area Goal Scores. If the child’s Area Goal Score is at or below the cutoff score for that area, then the child may be eligible for services. If the child’s Area Goal Score is greater than the cutoff score, then the child may not be eligible for services. It is important to note that the AEPS Test developers highly recommend that AEPS cutoff scores only be used if a complete AEPS Test has been administered (i.e., at least all goals from each developmental area have been scored) because a child who has delays in one area may have unapparent delays in another area.

EXAMPLES OF HOW TO USE AEPS TEST CUTOFF SCORES

Bianca

Upon referral, the diagnostic team completed the AEPS Test Level I with 24-month-old Bianca. All goals in the six developmental areas were scored and then the 2s and 1s tallied for each area. Below is a comparison of Bianca’s Area Goal Scores with the cutoff scores for the 22–24 month age interval by developmental area.

These comparisons show that Bianca’s Area Goal Score is below the cutoff score for each of the six developmental areas assessed on the AEPS Test. It is highly likely that she is functioning at a level significantly below her chronological-age peers and is therefore eligible to receive EI/ECSE services.

Daniel

At 61 months of age, Daniel was referred for evaluation because of an apparent speech delay. State law required administration of two measures, at least one of which is standardized, in areas of concern. Because communication appeared to be the primary area of concern, the assessment team decided to use a standardized speech and language measure and the AEPS Test as the second measure to determine eligibility. Daniel’s performance on the speech and language measure indicated his language skills to be significantly below his age norms. His Area Goal Scores on the AEPS Test are shown below.

These comparisons indicate that Daniel’s performance in the Fine Motor, Gross Motor, Adaptive, and Social Areas does not fall at or below the cutoff scores, suggesting his behavior in these areas is similar to that of his chronological-age peers. However, his performance in the Cognitive and Social-Communication Areas is at or below the respective area cutoff scores for his age interval. These findings corroborate the findings of the speech and language measure and indicate that Daniel may be eligible for EI/ECSE services.

SUMMARY

The AEPS is a CBA measure that offers a set of empirically derived cutoff scores that can be used to help determine if children are eligible to receive EI/ECSE services. As with all tests, the AEPS Test should be used in conjunction with other available sources of information about children’s past and current development. Parents and other caregivers are a rich source of developmental data that can be tapped through interview or by using the AEPS Family Report. In addition, information from other professionals and agencies should be considered. It is the combining of information from multiple sources that will lead to the greatest accuracy in determining children’s eligibility for services.