08smith2TechApp.059_080
Technical Appendix
This technical appendix reports data that were collected from 1997 to 2002 as part of the development of the ELLCO Toolkit, Research Edition, as well as additional data that were collected from 2002 to 2007 using the Research Edition. We made numerous changes, based on this data and feedback, to make the ELLCO Pre-K easier to use and score. The most significant changes include the integration of the Literacy Environment Checklist and the Literacy Activities Rating Scale into the observation architecture, and the inclusion of detailed descriptive indicators for each of the five scale points. Specific psychometric analyses on the current ELLCO Pre-K will be reported as the tool is used.
Since its initial publication, researchers have used the ELLCO Toolkit, Research Edition, in more than 250 classrooms across six different projects focused on the language and literacy development of children from lower income families and communities. The data reported here come from 30 classrooms from the NEQRC study and a total of 117 classrooms for the LEEP study.
Major Research Projects Involved:
- The New England Quality Research Center: The Next Generation
- Examining the Efficacy of Two Models of Preschool Professional Development in Language and Literacy
- Child Care Quality: Does Partnership Make a Difference—an Extension of the Partnership Impact Project
- Evaluation of the Newport Early Reading First Collaborative
- Evaluation of the Springfield Early Reading First Initiative
- Connecticut Is Reading First
As with the data used for the Cronbach’s alpha analyses originally reported, some classrooms were visited on multiple occasions, where each visit was counted as a separate observation.
Psychometric Properties of the Literacy Environment Checklist
The psychometric properties for the Literacy Environment Checklist are based on data from Year 4 of the NEQRC project and Years 1–3 of the LEEP project. Together, the projects resulted in a total sample size of 255, although the actual subsample sizes vary depending on the analyses conducted. The items from the Literacy Environment Checklist have been incorporated into the main body of the ELLCO Pre-K.
Summary Statistics for the Literacy Environment Checklist (n=255)
| Composite variable | Mean | Standard deviation | Minimum | Maximum |
|---|---|---|---|---|
| Books subscale | 11.13 | 3.90 | 2.00 | 20.00 |
| Writing subscale | 10.44 | 4.22 | 1.00 | 20.00 |
| Literacy Environment Checklist Total score | 21.57 | 7.37 | 5.00 | 40.00 |
Reliability Analysis (n=255)
| Composite variable | Alpha |
|---|---|
| Books subtotal | .73 |
| Writing subtotal | .75 |
| Literacy Environment Checklist Total score | .84 |
Measuring Stability and Change
Using the data collected from the LEEP classrooms, we reported preliminary findings on the ability of the Literacy Environment Checklist to measure both stability and change over time.
Stability and Change in Literacy Environment Checklist Scores
| Composite variable | Fall | Comparison group (n=38) | LEEP intervention (n=40) | Spring | Comparison group (n=38) | LEEP intervention (n=40) |
|---|---|---|---|---|---|---|
| Books subtotal | 9.25 | 10.53 | 10.45(t=3.27,p<.01) | 14.84(t=7.18,p<.001) | ||
| Writing subtotal | 8.77 | 11.21 | 9.12(t=1.14,p=n.s.) | 14.26(t=5.72,p<.001) | ||
| Literacy Environment Checklist Total score | 18.12 | 20.86 | 19.52(t=2.87,p<.01) | 29.03(t=7.82,p<.001) |
Psychometric Properties of the Classroom Observation
The Classroom Observation has been used for research for both the NEQRC and LEEP projects, with data gathered from 308 classrooms.
Summary Statistics for the Classroom Observation (n=308)
| Composite variable | Mean | Standard deviation | Minimum | Maximum |
|---|---|---|---|---|
| General Classroom Environment subtotal | 3.44 | 0.79 | 1.20 | 5.00 |
| Language, Literacy, and Curriculum subtotal | 3.02 | 0.75 | 1.13 | 5.00 |
| Classroom Observation Total score | 3.15 | 0.71 | 1.29 | 5.00 |
Reliability Analysis (n=308)
| Composite variable | Alpha |
|---|---|
| General Classroom Environment subtotal | .83 |
| Language, Literacy, and Curriculum subtotal | .86 |
| Classroom Observation Total score | .90 |
Predicting Child Outcomes
The Classroom Observation has been used to predict children’s literacy development. The data suggest that the quality of preschool classrooms attended by children from low-income families plays a significant role in supporting their vocabulary growth and early literacy development.
Psychometric Properties of the Literacy Activities Rating Scale
The Literacy Activities Rating Scale has been used for research as part of the NEQRC and LEEP, and the data presented here are from centers and classrooms in lower income communities.
Summary Statistics for the Literacy Activities Rating Scale (n=262)
| Composite variable | Mean | Standard deviation | Minimum | Maximum |
|---|---|---|---|---|
| Full-Group Book Reading subtotal | 2.86 | 1.95 | 0 | 6.00 |
| Writing subtotal | 2.10 | 1.39 | 0 | 5.00 |
| Literacy Activities Rating Scale Total score | 5.80 | 2.63 | 0 | 13.00 |
Reliability Analysis (n=262)
| Composite variable | Alpha |
|---|---|
| Full-Group Book Reading subtotal | .92 |
| Writing subtotal | .73 |
| Literacy Activities Rating Scale Total score | .66 |
Correlations Among the ELLCO Toolkit Measures
Tables below report correlations among the three measures that comprise the ELLCO Toolkit, Research Edition:
- The Books subtotal, the Writing subtotal, the Literacy Environment Checklist Total
- The Language, Literacy, and Curriculum subtotal, the General Classroom Environment subtotal, and the Classroom Observation Total
- The Full-Group Book Reading subtotal, the Writing subtotal, and the Literacy Activities Rating Scale Total.
Overall, the psychometric properties of the ELLCO Toolkit, Research Edition indicate strong evidence of reliability and validity, supporting its use in assessing literacy environments in preschool settings.