Observational Measurement of Behavior

About the Authors

Paul J. Yoder, Ph.D.

Professor of Special Education, Department of Special Education, Box 220, Peabody College, Vanderbilt University, Nashville, Tennessee 37203.

For more than 30 years, Dr. Yoder has used observational measurement to study communication and language development in children with disabilities and how parental interaction influences their immediate and sustained use of nonverbal and verbal communication acts. Throughout his career, Dr. Yoder has contributed to the empirical basis for decisions affecting the scientific utility of observational variables. He teaches graduate courses on observational measurement and research design at Vanderbilt University.

Blair P. Lloyd, Ph.D., BCBA-D

Assistant Professor of Special Education, Department of Special Education, Box 228, Peabody College, Vanderbilt University, Nashville, Tennessee 37203.

Dr. Lloyd’s research focuses on individualized assessment and intervention for students with persistent challenging behavior. She is an active user of observational measurement and sequential analysis methods in her own research and has published multiple methodological papers on sequential analysis. She teaches graduate courses in experimental analysis of behavior and single-case research design.

Frank J. Symons, Ph.D.

Professor, Department of Educational Psychology, College of Education and Human Development, 56 East River Road, Education Sciences Building, University of Minnesota, Minneapolis, Minnesota 55455.

Dr. Symons is a Distinguished McKnight University Professor in Special Education and Educational Psychology at the University of Minnesota. His research agenda positions him in the crossroads of interdisciplinary inquiry in behavioral disorders and neurodevelopmental disabilities with several specific foci, including self-injury, pain, and Rett syndrome. Many of his approaches rely on direct observational methods.

SECTION I: FOUNDATIONAL TOPICS

CHAPTER 1: Introduction to Systematic Observation and Measurement Contexts

The purpose of this chapter is to review a number of underlying issues involved in observational measurement of behavior. These issues, although not always explicitly articulated in a given research report, are critical to understanding the logic behind the different research approaches to quantifying behavior using systematic direct observation and the strategies used for doing so. In this chapter, we define the book’s central topic: systematic observation using count coding. We then promote hypothesis-driven research as a general approach to maximize a study’s scientific rigor and interpretability. Next, we discuss an important distinction between observed behavior as context-dependent and observed behavior as a sign of a generalized person characteristic.

SYSTEMATIC OBSERVATION USING COUNT CODING

The systematic observation approach to measurement requires that before beginning data collection the following elements have been decided: the procedure (i.e., type of session) to observe, the definitions of key behaviors, and the type of number used to quantify the phenomenon of interest (Suen & Ary, 1989).

An example of systematic observation is an observer recording the presence, quality, or amount of communication from a 15-minute parent–child interaction session. Other examples include observing engagement during a classroom activity or rating or counting key behaviors in a structured diagnostic evaluation, such as the Mullen Scales of Early Learning (Mullen, 1995). Systematic observation is contrasted with the type of observation used in qualitative research.

Systematic observation: A method of quantifying variables in which a coding manual, context of measurement, sampling methods, and metric are decided prior to collecting data.

Alternatives to Systematic Observation

Alternatives to systematic observation include self-report, that is, asking the participants what they generally do, and third-party report, also known as other or proxy report, that is, asking people who have experience with the participant to make conclusions about the extent to which, or quality with which, the participant generally engages in particular behaviors.

Self-report: Measurement approach involving asking the participant what they do, feel, or think.

Third-party report: Measurement approach involving asking people who have experience with the participant to quantify some aspect of participant’s general behavior.

Generalized Person Characteristics

Generalized behavioral characteristics are descriptors of what people usually do. As such, they are typically measured in the natural environment and are expected to be stable across valid measurement contexts.

Generalized behavioral tendency: Descriptor of what people usually do.

Skills

Skills are constructs that we call abilities or developmental achievements. Skills represent a highly generalized ability that can be and is used in a wide variety of contexts, regardless of level of prompting from the environment.

Skill: What a person does in a situation in which the effect of the context is made irrelevant by using a structured measurement context.

Reliability

Reliability is the degree to which a measure is consistent with another measure of the same thing. The most relevant types of reliability to observational measurement include interobserver agreement and stability of scores.

Validity

Validity is the degree to which a measure represents what we believe it represents. Validity evidence can include content validity, sensitivity to change, and construct validity.

Conclusion

In this chapter, we defined what we mean by systematic direct observation using count coding. We also discussed the distinction between measuring a context-dependent behavior and measuring two types of person characteristics. The distinction is very important for proper framing and interpretation of a study and for many measurement decisions that affect the measures’ psychometric properties.