skip to main content

NCHAM: National Center for Hearing Assessment and Management, Utah State University

Last Modified: 09/04/2018

VII. Evaluating Tele-Intervention Outcomes

a teleintervention session in progress

On This Page

Outcomes of TI for Investigation

As with any type of intervention, evaluating TI is critical to ensure that it is in fact resulting in the desired outcomes:

  • Increased parent knowledge, skills/competence in fostering development, and satisfaction with services and providers, increased competence with technology
  • Increased Child developmental/communication outcomes
  • Improved parent-child interaction

Additionally, it is important to measure the impact of TI on the following additional outcomes that are associated with delivery of TI:

  • Increased frequency and intensity of intervention
  • Provider skills/competence in coaching
  • Incorporation of natural routines
  • Decreased costs, considering travel time, opportunity cost to providers as well as families

There are also factors that can influence outcomes that are important to capture:

  • How much acoustic and visual clarity is needed in the technology used?
  • Can the provider offer needed emotional support to families when needed via TI?
  • Does TI work for various interventions?
  • Are some parents/children a better fit?
  • Are some “interveners” a better fit? An important place to start in evaluating the cost-effectiveness of TI is to develop a logic model.
  • Do some internet connection options (i.e., wifi, wired connection, hotspots) work better in some areas?

Creating a Logic Model to Guide Evaluation

A logic model serves to guide the identification of measurement tools to answer questions such as those posed above. A logic model is a way to graphically depict the resources, actions, and expected results of an intervention. Here are some resources to guide develop of a logic model:

Download an example of a logic model developed by NCHAM [PDF]

Selecting Measurement Tools and Administration

A logic model serves to guide the identification of tools to measure outcomes. The table below corresponds with the NCHAM logic model, delineating the constructs to be measured, the tools to be used, and the frequency of data collection. Creating an outline such as the one provided below is important in creating your evaluation plan. Creating such a table helps ensure that your evaluation plan is comprehensive but also practical in regard to your capacity to employ all the measurement activities.

Measurement Tools and Administration (NCHAM, 2011)

Model of Change Component Instruments Description & Administration Frequency of Administration
Service delivery: Dosage Provider contact logs: Number & duration of sessions conducted; other providers involved; Interruptions in service delivery (technology problems, cancellations, other home visit interruptions) Provider self-report contact form completed weekly by each family in the study Each scheduled session
Service delivery: Technology satisfaction Family Technology Survey & Provider Technology Survey Self-report survey completed by both families & providers in tele-intervention group Pretest; every 6 months
Service delivery: Audiology
  1. Interventionist Hearing & Technology Management Log
  2. Audiological Services Record
  1. Provider report
  2. Provider report
  1. Each scheduled session
  2. Each scheduled session
Service delivery: Costs
  1. Provider cost form, program cost form, time diary
  2. Family Services and Cost Survey
  1. Researcher-developed measures on cost and time data completed by program and providers
  2. Family telephone survey regarding services/supports received & associated costs to the family
  1. Every month
  2. Every 6 months
Family-centered Care, Parent Knowledge & Competence
  1. Home Visit Rating Scales—A; subscales re: parent-child interaction & engagement
  2. Parental Involvement & Self-Efficacy (SPISE)
  3. Language sample: Whitehurst, et al. (1988) coding scheme applied to videotaped sessions
  1. Standardized rating scales coded by research staff
  2. Parent self report survey
  3. Standardized instructions for parent- child activity; parent Mean Length of Utterance (MLU) coded by research staff
  1. Pretest; 1 visit per month
  2. Pretest; every 6 months
  3. Pretest; 1 visit per month
Natural Environments & Provider Coaching
  1. Home Visit Rating Scales, subscales re: home visitor
  2. Coaching Practices Rating Scale
  3. Natural Environments Rating Scale
1-3. Standardized rating scales coded by research staff from videotaped sessions
  1. Every month
  2. Every month
  3. Every month
Child Communication and Development Outcomes
  1. MacArthur-Bates Communication Development Inventory/Inventarios
  2. Battelle Developmental Inventory -2
  3. Preschool Language Scale 4 (PLS-5)
  4. Language sample
  5. Auditory Skills Checklist
  1. Norm-referenced (ages 8-37 mos.) Parent self-report measure; Spanish version available; administered by providers
  2. Norm-referenced (birth – 7 yrs 11 mos.), English/ Spanish versions, administered by research staff
  3. Standardized (birth- 6 yrs 11 mos.), receptive & expressive communication; English & Spanish versions, administered via research staff
  4. MLU’s measured via coding of standardized activity by research staff
  5. Provider–scored checklist
  1. Pretest; every 6 months
  2. Pretest; every 6 months
  3. Pretest; every 6 months
  4. Pretest; monthly
  5. Pretest; monthly

Recent TI Evaluations

Several studies to measure actual differences between services provided by tele-intervention and face-to-face home visits have been conducted with families who have children who are DHH, and there are relevant studies of TI conducted with other EI populations. Here is a literature review of recent publications that support the use of TI as a cost-effective, family-centered approach to delivering EI services:

Featured Articles/ Presentations

Read about the latest research about tele-intervention for families of infants and toddlers with disabilities: