Global Guidelines Assessment
Investigating the Psychometric Properties of the
ACEI Global Guidelines Assessment in Nine Countries
In the most recent test[i] of the viability of the GGA as a global tool for assessing and improving early care and education program quality, a sample of 346 early childhood programs and 678 individuals participated in a reliability and validity study to investigate the psychometric properties of the GGA from 2011 to 2014 (Hardin, Bergen, Busio, & Boone, 2016). The study took place in nine countries (People’s Republic of China (2 sites), Guatemala, India, Italy, Mexico (2 sites), Peru (2 sites), Thailand, Taiwan, and the United States) and across 12 research sites. A stratified sampling procedure was used based on geographic area (e.g., world region, country), type of setting (private/public), and service type (rural/urban). A summary of these results are listed below:
- The internal consistency of the GGA was examined to determine how well the instrument holds together as a single measure and how well the items are measuring defined constructs (e.g., environment, physical space, curriculum, parent/community involvement, and special needs). The alpha coefficients for the total sample (N=678) indicate very high positive internal consistency for each subscale (0.90 to 0.92) and the total GGA (0.97).
- Program interrater reliability was examined by comparing the ratings between the two different participants (director and teacher) from each program for 550 participants from 275 programs. Pearson’s r correlations indicate a moderate to high positive degree of consistency (0.51 to 0.74) for the five subscales and the total GGA (0.72) when completed by two different raters for the same program.
- External interrater reliability was conducted with a small subset of programs (n = 44) from two research sites. For this analysis, a GGA was completed by an external rater and the results compared to the mean of the two program raters using the Pearson’s r. While the correlations for Site 1 in Peru (“Peru1”) were stronger than those from the U.S. site and the total interrater comparisons, subscales 2 and 3 showed low correlations across the sample and further investigation is needed.
- To examine criterion (concurrent) validity, the GGA and the ECERS-R (Early Childhood Environment Rating Scale-Revised) were administered in a subset of 44 programs. The results indicate moderate positive correlations (using the Pearson’s r) between the GGA and ECERS-R with correlations ranging between 0.43 to 0.70 when comparing subscales of both instruments and 0.55 to 0.70 for the total GGA.
- The GGA study required raters to provide examples that gave evidence for their ratings to examine the validity of the ratings. The overall mean for 11 research sites was 3.47 (SD = 0.93) out of a possible 5.00, which indicated that there was a moderate level of evidence provided to support ratings.
[i] Hardin, B. J., Bergen, D., Busio, D. S., & Boone, W. (2016). Investigating the psychometric properties of the ACEI global guidelines assessment, third edition (GGA) in nine countries. Early Childhood Education Journal. doi:10.1007/s10643-016-0795-9