2. Methodology

2.1 Approach

A mixed method approach was applied to this review to obtain information from a variety of new and existing sources about operation of the CIS Scheme in the first two years since commencement of the whole-of-Victorian-government initiative. The review, commissioned by the Department of Education and Training (DET) as lead agency for the CIS Scheme, was conducted by ACIL Allen Consulting in partnership with Wallis Market and Social Research (Wallis).

The approach to the review was underpinned by development of a program logic for the CIS Scheme providing an overview of the key inputs, outputs and expected outcomes of the CIS Scheme (see Appendix B). Alignment of the program logic and the key areas of inquiry for the review underpinned the research questions in the evaluation framework that has guided this review. The framework identified measures and potential sources of information to inform a response to each of the research questions.

Data collection was undertaken predominantly at two points in time with the first commencing in July 2019 (providing input to the Baseline Report) and the second in June 2020. Information gathering focused on establishing attitudes and practices of prescribed workforces to information sharing, the support provided by peak bodies to their respective sectors for effective implementation of information sharing reforms and the extent of change to organisational record keeping processes and systems to facilitate implementation of the CIS Scheme.

Ethics and other research approvals were obtained for engagement with prescribed workforces and planned consultation with young people accessing services provided by information sharing entities under the CIS Scheme. Accessing the views of young people supported by information sharing entities is discussed under section 2.3 relating to limitations of the review.

2.2 Review inputs

Data collection methods included surveys, interviews, virtual workshops, document review and case studies illustrating aspects of implementation of the CIS Scheme. Tools developed for the review included questionnaires, discussion guides and background material to facilitate workshop discussion.

A summary of data collection activities is provided in the following table. A profile of respondents to the workforces surveys is provided at Appendix C. Copies of data collection tools are provided at Appendix D.

Source: ACIL Allen Consulting 2020

To the extent appropriate, quotations used in this report and drawn from qualitative feedback provided as part of the surveys of prescribed workforces may include identification of the respondent’s workforce category to provide additional context.

2.3 Limitations

The review methodology has been adapted to accommodate scheduling changes related to roll out of the CIS Scheme and the impact of the coronavirus pandemic on Phase One prescribed organisations and services.

Establishing a baseline

A key component of establishing a baseline related to information sharing was the survey of prescribed workforces. Recruitment of workforces to this research was linked to their online registration for initial training related to the information sharing reforms. Workforces were surveyed for baseline attitudes and practices approximately 10 months after the CIS Scheme had been proclaimed, in part due to a delay in commencement of training. To accommodate this timing, survey respondents were asked to consider their attitudes and practices to information sharing 12 months previously, prior to introduction of the CIS Scheme.

Workforce survey sample

There are two possible areas of limitations caused by bias in the sample of workforces participating in the 2019 and 2020 surveys, namely non-response bias among the survey sample and sample bias. Non-response bias, where those responding to the survey invitation fundamentally differed from the overall population, is likely to have been mitigated as a serious source of bias by the relatively high response rates achieved and the use of the multi-modal approach (option for participants to complete the survey by telephone or online). In relation to sample bias, as the sample of workforces opted in to the research, they could be considered more engaged with the CIS Scheme. While the likelihood of this phenomenon occurring has not been measured, it is expected that it would be limited in nature.

Progressive feedback from prescribed organisations and services

Early feedback on implementation of the CIS Scheme was intended to be captured late in 2019 through roundtables involving prescribed organisations and services, including representation from those entities expected to have been prescribed under Phase Two. Other priority consultation on the information sharing reforms at that time and deferral of prescription of Phase Two entities until 2021 due to the coronavirus pandemic made this approach impracticable. Consequently, it was agreed that the planned workshops in 2020 would be augmented to provide increased opportunity for Phase One prescribed organisations and services to access this review. In the event, the impact of the coronavirus pandemic influenced stakeholder ability to participate and required the conduct of virtual workshops. Three workshops were offered and two were conducted in early August 2020.

Including the views of young people

Attempts as part of this review to provide an opportunity to hear from young people about their experiences of sharing their information were unsuccessful. This included a direct approach to eligible ISEs in the 2019 data collection period and an approach through the Centre for Excellence on Child and Family Welfare’s sector e-newsletter inviting services to obtain further information should they be willing to assist in the recruitment of young people for the 2020 data collection. The lack of response from ISEs can in part be attributed to the demands on sector resources, which were intensified in the latter part of the review during the coronavirus pandemic, and a reluctance to engage their clients in research activity.

While it has not been possible to include the voice of the young person in this research, some of the emerging benefits of the CIS Scheme are illustrated.

However, a focus on the experiences and perspectives of children and young people is even more appropriate for the subsequent five-year review of the CIS Scheme when it has become better embedded in workforce practice and its potential to support earlier intervention and prevention is realised. Future consideration could be given to support for collaboration with a selection of prescribed organisations and services and the independent reviewer for annual consultation with young people through focus groups or interview about the way in which they experience information sharing and any ways in which this experience might be improved. There may also be potential to identify ISEs who are implementing or who would agree to implement good practice record keeping (as described in the Ministerial Guidelines) when disclosing information by recording additional details about what the views of the child and/or relevant family member were about information sharing. With appropriate research approvals, this information could be reviewed for any common themes.