Centre for Developmental & Applied Research in EducationView our Privacy & Cookie Policy

CeDARE Reports Logo
  • Follow us on Twitter
  • View our full site at the University of Wovlerhampton

I CAN's Early Talk Programme:


Independent evaluation of the impact of Early Talk on addressing speech, communication & language needs in Sure Start Children's Centre settings

Dr Judy Whitmarsh, Dr Michael Jopling, Prof Mark Hadfield

Feedback


These videos were produced by Soundhouse Media.

Click here to download a folder containg all of these videos (.zip file 200MB)

Share

Theoretical Framework

Multiple method evaluation is considered to be the most effective way of providing a more complete picture of the evaluated subject (Grammatikopulous et al, 2008); using multiple methods also offers a means of triangulating the data. To structure the multiple methods employed, the research design and data analysis drew on the work of Guskey (2000) and Kirkpatrick (1994) to enable us to conceptualise and develop the following framework for the research process, looking at impact, fidelity of implementation, and sustainability.

Impact

We adapted Guskey's (2000) and Kirkpatrick's (1994) models of impact evaluation to underpin our evaluation of ET. Impact was assessed at five levels:

  1. How did adult participants engage with ET? (Did staff like the programme? Did it fit with their objectives?)
  2. What did adult participants learn from the programme? (What were the changes in knowledge and understanding)?
  3. How did adult participants' behaviour change following their engagement with ET?
  4. What was the impact on the children's centre (organisation and resources)?
  5. What were the outcomes for children, staff and parents?

In addition, impact was viewed from the following perspectives:

  • Framing the pedagogy
  • Interaction
  • Parents, culture and the community
  • SLC specific needs (or issues around early identification).

Within this, the study investigated two different dimensions of speech, language and communication:

  1. The interactive dimension: The communicative and pedagogical behaviour of the practitioner towards the child(ren).
  2. The contextual dimension: language and learning opportunities of the environment.

Addressing the impact of ET in this way allowed us to address all the research objectives and undertake detailed cross-case analysis (see Appendices 2 and 3).

To evaluate the sustainability of ET, we drew on the work of Cynthia Coburn (Coburn 2003; Coburn and Russell 2008). During data analysis, we therefore examined crucial features such as:

  • how managers mediated messages about the change and the curriculum and the extent to which they reflected the I Can principles;
  • examples of shift in ownership of ET from external (I CAN) to internal (managers and practitioners) in the children's centre. This would be demonstrated by adapting resources to changing situations or the environment and/or contextualising the ET principles and standards;
  • the development of experienced pools of support and advice on SLC on which staff could draw for knowledge and skills;
  • the development of a core of professional practitioners whose knowledge and practice was underpinned by pedagogical ET principles;
  • mentoring and coaching arrangements; and
  • challenges to programme sustainability from competing priorities.

Rather than merely analysing the data for evidence of a change to a language-rich centre following implementation of ET, we sought also to identify the depth of the implementation. Adapting Coburn and Russell's (2008: 230) levels of depth of change, we analysed the data for evidence of:

Low levels of implementation
Changes to the surface structure (for example, display labelling) or room organisation (such as room re-arrangement) or the addition of new materials or resources. Talk related to how to use the materials, standards, assessments, general discussion of how an activity went.

Medium levels of implementation
Detailed planning for an activity, including purpose. Talk related to how an activity had gone (including why). Specific and detailed discussion of how children learn. Discussion of strategies in relation to observations. Shared problem-solving.

High levels of implementation
Talk related to pedagogical principles underlying how children learn, pedagogical principles underlying strategies, the nature of children's language learning or principles and concepts of SLC and SLCN.

High levels of change can perhaps best be seen as evidence of an enacted curriculum, that is by paying attention to practitioners' beliefs, norms and pedagogical principles. This relates to the ways practitioners drew on "pedagogical principles and norms of interaction in areas of the classroom beyond those subjects, times of day, or particular activities targeted by reform" (Coburn 2003: 7).

Fidelity of implementation

We examined the extent to which children's centres' implementation of ET matched the programme's aims and guidelines and represented a consistent approach in and across children's centres. We also looked at children's centres' perceptions of ET's fitness for purpose with respect to their SLC needs.

Sustainability

The use of a three-staged approach to the children's centres (see 3. Methodology) allowed us, to some extent, to investigate the sustainability of the ET programme. We also sought to explore this in greater depth than a comparison between stages of ET implementation. Again, drawing on the work of Coburn and Russell (2008) we identified and adapted strategies to evaluate the sustainability of ET such as:

  1. Did change persist over time?
  2. Were the underlying principles and practices of good SLC practice, as detailed in the ET standards, embedded in the children's centres in practice and policies?
  3. Were there accessible, knowledgeable leaders in the LA and the centres?
  4. Could the principles and practices associated with ET be seen beyond the curriculum in post-accreditation centres?
  5. Was there a shift in ownership from external (I Can) to internal (managers and practitioners)? This relates to the extent to which ET resources and practices were integrated and adapted in children's centres.
  6. Were there knowledgeable leaders on-site to induct newcomers and oversee CPD?
  7. Did ET lead to the development of key leaders in the LA and in practice who could interrogate and adapt new initiatives and provide expert knowledge and skills?