Evidence-Based Practice (Technical Analyses)

Examiners need to know that the interpretations they make are supported by data-based evidence. In this chapter, we review the types of questions examiners should ask about the nature of any test, which will inform their conclusions about test performance. Therefore, this section covers validity and reliability information as well as information concerning analysis of test bias. We continue to use the concept of constrained and unconstrained skills (cf. Paris, 2005) to interpret results. To review, constrained subtests tap skills that develop relatively early in development (e.g., knowing one’s name, naming common items). Unconstrained subtests tap skills that tend to show continued refinement over a wide range of ages. The PTBI contains both types of subtests, and the statistical results are presented separately for each type. A question and answer framework is used to describe the evidence base related to various clinical questions about the nature of the PTBI.

EVIDENCE FOR CONSTRUCT-IDENTIFICATION VALIDITY

Question 1: How do I interpret the criterion categories?

The categories of performance for each age are based on the performance of the group of children without brain injuries selected to represent the general population. The percentages of these children whose scores fell into each of the four criterion categories is provided in Table 6.1. As this table shows, most scores fell into the High performance category. The differences in the percentages of children classified into the different performance categories reflect naturally occurring differences in the range of performance that children in the general population show in these different skill areas.

Technical Notes

Scores from 103 children (57 girls, 46 boys) were classified against the criterion categories for this source of evidence. They ranged in age from 6 to 16 years, with an average age of 9 years, 11 months. No child had been diagnosed with a brain injury (TBI or ABI).

As Table 6.2 shows, the typically developing children received higher ability scores on all subtests compared with the ABI group. Children with TBI scored significantly lower than their peers without brain injuries on all seven subtests that assess unconstrained skills and two of the constrained skills subtests.

Technical Notes

Ninety-nine children’s scores were compared for this source of evidence. Thirty-three children were included in each of the three groups (TBI, ABI, typical). These children were matched by age (mean age of 10 years, 7 months; range 6–16 years) and gender (14 males, 19 females per group).

Question 3: Are scores on the PTBI related to the degree of severity of the brain injury for children with TBI?

To address this issue, we correlated the scores on the unconstrained skills subtests of PTBI with scores on the Glasgow Coma Scale and the Rancho Los Amigos Scale. These two scales are frequently used to index severity of brain injury. As Table 6.3 shows, the scores on most subtests of the PTBI are correlated with severity.

Technical Notes

Thirty-eight individuals (14 females, 24 males) with TBI received both the Glasgow Coma Scale and the Rancho Los Amigos Scale at or near the time of testing. The mean age of these children was 12 years, 9 months (range 7–16 years).

Question 4: How do I know that the scores I obtain actually reflect a child’s ability on the skills tapped by the subtest?

State-of-the-art methods of estimating a child’s ability require computation that is often complex and/or time-consuming. Therefore, it was important to have a scoring method that accurately estimated a child’s ability but could be computed with relative ease. As Table 6.4 illustrates, there was a very high correlation between the calculated ability score.

Question 6: Do scores on subtests designed to assess unconstrained (developmental) skills correlate with age?

...Overall, performance on the constrained skills subtests were near ceiling for most children. This made conducting correlations inappropriate for these subtests because of the highly restricted variability in scores.

EVIDENCE FOR RELIABILITY

Reliability data indicate how likely the test is to yield stable information about the child.

CONCLUSION

This chapter guided examiners through the questions to ask to determine test performance. The evidence base for the PTBI was presented. Chapter 7 provides a case study demonstrating a test and retest of the PTBI.