2021-22 Quarter 2 Fire Services Outcome Framework Progress Report

This Quarter 2 (Q2) Fire Services Outcomes Framework Progress Report outlines the Fire Services Implementation Monitor (FSIM)’s observations of Country Fire Authority (CFA) and Fire Rescue Victoria (FRV)’s performance against their respective outcomes framework indicators.

CFA and FRV published their respective Year Two Outcomes Frameworks in October 2021. FRV continues to refine its Year Two Outcomes Framework indicators and, given the Year Two indicators are not yet finalised, has reported against both its Year One and a limited number (5) of Year Two Outcomes Framework indicators in Q2.

FSIM also notes that CFA continue to use baselines to assess performance in the Year Two Framework. Baselines are not always an effective tool to measure improved performance, particularly when they do not show improvements over time. In order to provide clearer assurance to community, CFA should consider how the baselines can be used in conjunction with targets to provide a clearer measure of performance.

The reformed fire services have now been in place for a year and a half. Over that time, FSIM has encouraged agencies to refine and mature their understanding of the linkages between outcomes results and program delivery, noting outcome performance is most useful to the community where impacts can be explained. FSIM’s longer term objective for these outcomes frameworks is for agencies to demonstrate how their activities deliver a modern fire service that protects Victorians, such as demonstrating impacts to the community where a target or baseline is met/not met.

Training and Community Engagement

Both organisations report that COVID-19 related restrictions have resulted in a reduced number of training activities being undertaken in Q2. As restrictions and other COVID-19-related requirements ease, CFA and FRV should take steps to prioritise training and skills maintenance activities, ensuring that there are sufficient firefighters with the right skills to undertake operational and specialist response activities now and into the future.

FRV has been unable to meet its community education and engagement targets since the reform commenced in July 2020. FRV reported no formal community education programs in FY 2020-21 and a year-to-date total of 29 Firelighting Consequence program engagements in Q2 FY 2021-22. This result is significantly below FRV’s comparable Q2 FY 2020-21 target of 80 community engagements. FRV reported that COVID-19 restrictions constrained their capacity to maintain minimum crew levels for operational response and consequently, delivery of non-urgent programs by firefighters was paused for the safety of the community and firefighters.

CFA introduced a new indicator in this quarter relating to the number of training courses completed by CFA members. CFA did not meet the nominated baseline primarily due to course cancellations resulting from COVID-19 restrictions and the prioritisation of work to ensure compliance with the National Units of Competency as prescribed by the Victorian Registration and Qualifications Authority (VRQA).

Community education and engagement activities are a vital tool to equip Victorians with knowledge and skills to prevent and prepare for fire and other emergencies. Further delays to community education program delivery may impact the overall prevention and preparedness capabilities of the Victorian community.

Service Level Agreements

FRV currently reports on the implementation progress of operational service level agreements (SLAs) and the corporate memorandum of understanding (MoU). While FRV is the only agency reporting on this indicator, and the FSIM notes that this indicator is not included in FRV’s Year Two Outcomes Framework, these agreements and MoUs are foundational tools that clarify roles and responsibilities for both CFA and FRV in service delivery provision. Both agencies have responsibilities to progress, finalise, implement and embed the service delivery arrangements outlined in these draft agreements. A number of SLAs remain outstanding and further delays to finalising the SLAs will result in continued service delivery inefficiencies arising from interim administrative arrangements.

Concluding Remarks

FSIM acknowledges CFA and FRV for their cooperation and active contribution to FSIM’s monitoring and reporting activities. This is particularly appreciated given the ongoing administrative and operational pressures CFA and FRV continue to face. As agencies continue to report against their Year 2 outcomes framework, FSIM will be able to set out a clearer baseline of fire service performance against outcomes, following implementation of the reform.

Definitions

Reference to fire services agencies

References to the fire services in this document relate specifically to CFA and FRV. Where more than one fire services agency is referenced in this document, they are listed in alphabetical order.

Comparison between agencies’ performance against published Outcomes Frameworks

As the two outcomes frameworks are based on the agencies’ own strategic and operational priorities, comparisons between the agencies should not be made.

Purpose of the report

This independent quarterly report provides observations on the progress of CFA and FRV against their respective outcomes-based fire services performance indicators.

Background

The Victorian Government has embarked on a 10-year reform agenda to enhance fire services in Victoria, releasing its Year Two to Five Fire Services Reform Implementation Plan (Year Two to Five Plan) in 2021. The Year Two to Five Plan outlines a shared course of action for fire services agencies over four years to implement the vision set out in the 2017 Fire Services Statement (Fire Services Statement).

Fire Services Implementation Monitor (FSIM)

FSIM was appointed in 2020 in accordance with the Fire Rescue Victoria Act 1958 (the Act). FSIM’s functions under the Act are to assess the effectiveness of agencies in delivering against Implementation Plan actions and provide independent assurance to government and the community on the progress made towards modern fire services providing for a safer Victoria. FSIM is required to prepare and publish quarterly reports on CFA and FRV Outcomes Frameworks measures under s141 of the Act.

The role of Outcomes Frameworks in monitoring and evaluation

Outcomes frameworks are a monitoring and evaluation tool linking a vision to a set of outcomes, indicators (including targets), and measures. They help demonstrate the value and achievements of agencies and can be used to identify change. They can also determine whether agencies are moving towards agreed outcomes while identifying areas for improvement and help focus monitoring and evaluation on potential high-risk issues.

FSIM’s approach

This report provides commentary where possible, on progress made towards the reform outcomes and on operational performance against the emergency sector’s vision for safer, more resilient communities. FSIM has reviewed CFA’s and FRV’s quarterly results and provides a summary and analysis of the data from Q2 and, where possible, trend data across the first years of reformed operations, taking into consideration Implementation Plan priorities and the operational performance reported by both agencies.

Limitations on data

As reported in Q1, CFA and FRV have finalised their Year Two to Five Outcomes Frameworks. FRV is yet to fully finalise and transition to reporting on its new indicators, whilst CFA will continue to review and refine its Year Two indicators over time. This report continues to provide commentary against indicators from FRV’s Year One Outcomes Frameworks, with the addition of five indicators from FRV’s Year Two to Five Outcomes Frameworks.

Publication of reports

FSIM publishes quarterly reports on this website. Publishing these quarterly reports promotes transparency and provides meaningful benchmarking that can measure the impact of the government's reform agenda on the fire services agencies, other emergency services stakeholders, and the broader community over time.

Country Fire Authority Q2 results overview

CFA has transitioned to its Year Two Outcomes Framework and provided data for all measures in the Year Two Framework. CFA recognise that as outcomes reporting matures, they will continue to review and refine indicators over time.

Appendix A provides a complete report of CFA Outcomes Framework data reported in Q2.

Fire fatalities and injuries

CFA’s vision for the future is for zero fire fatalities and fire related injuries in the country area of Victoria, however CFA did not meet this aspirational target in Q2.

FSIM notes that meeting the target of zero fatalities and injuries is complex and that CFA is undertaking a range of activities to reduce fire fatalities and fire-related injuries. For example, CFA is collaborating with Monash University Accident Research Centre (MUARC) to develop a suite of tools to inform prevention, preparedness and response activities including:

• a residential fire safety mapping tool to assist program planners in targeting prevention and preparedness programs to specific local and neighbourhood risk factors
• a Victorian fire risk register which combines census data to develop residential fire risk ratings across all CFA districts
• linking CFA incident data with Ambulance Victoria injury data for the same incidents, allowing CFA to consistently identify the frequency, nature and severity of injuries from fires, leading to improved understanding of the overall burden of fire related injuries.

FSIM notes that CFA is also collaborating with FRV to develop an annual Fire Fatality Insights workshop. Furthermore, CFA incident data has been used to inform Australasian Fire and Emergency Service Authorities Council (AFAC) doctrine ‘Residential Fire Fatality and Injury Prevention Strategy: Towards Zero Fatalities’.

FSIM supports CFA’s commitment to reduce the number of fire related fatalities and injuries to zero through multi organisation collaborations and the use of CFA incident data in research and development of new prevention, preparedness and response programs.

Training

CFA assesses competency of CFA members against both National Unit of Competency and enterprise training. CFA did not meet its baseline relating to the average number of training courses expected to be offered during Q2 due to:

1. ongoing work to ensure compliance for all training undertaken against National Units of Competency as prescribed by the Victorian Regulator for Vocational Education and Training, the VRQA and
2. limited training opportunity resulting from multiple public health-related lockdowns and disruptions.

CFA reports that strategies are in place to ensure compliance and increase in training courses. This includes the development of state-wide training work plans and schedules that take into account Victoria’s current health-related directions. FSIM will continue to monitor this indicator and will work with CFA to obtain further insights on the extent to which CFA staff and volunteer competencies align with CFA training strategies and expectations.

Volunteers under the age of 40

CFA did not meet the baseline for volunteers under the age of 40 in Q2 and FSIM notes the number of volunteers under the age of 40 has remained consistent since the beginning of the reform. An ageing population, rural decline and the required efforts to maintain current volunteer levels are ongoing challenges recognised by CFA.

CFA has introduced a Young Adults Advisory Committee to explore issues regarding leadership, recruitment and retention of young people. CFA is also nearing release of the Volunteer Recruitment Hub Registration Module which will provide it with demographic insights of new and transferring members which will aid in the development of targeted recruitment strategies.

FSIM acknowledges the challenges CFA faces maintaining an active cohort of young volunteers particularly in low density rural areas. FSIM looks forward to learning more about how CFA will address this issue in the future.

Female leadership

CFA has revised the definition of leadership from select brigade operational ranks (Captain, 1st through to 4th Lieutenant) to a more inclusive definition of leadership. The revised definition includes volunteer leadership roles categorised as Brigade Management Team (BMT) and Group Management Teams (GMT). As a result of this change, CFA met the female leadership baseline this quarter and also reached the Year Two to Five Fire Services Implementation Plan action target of 15 per cent of brigade leadership roles being held by women.

It is clear that leadership is broader than operational rank. CFA’s updated leadership definition incorporates a larger spread of leadership roles occupied by female volunteers including some non-operational roles and operational leadership roles that were previously not recognised (such as Brigade Training Coordinator, Brigade Community Safety Coordinator and Group Officer roles). FSIM will continue to monitor the number of female volunteers in brigade leadership roles as CFA continues to implement its diversity and inclusion strategies throughout the year.

Fire Rescue Victoria Q2 results overview

FRV’s transition to the Year Two Outcomes Framework continues to progress. FRV continues to report on Year One Outcomes Framework measures and has provided five Year Two Outcomes Framework indicators and proxy measures in Q2. FRV has not developed 2021-22 targets for Year One Outcomes Framework measures as it did not plan to report on these measures beyond June 2021, and is using the 2020-21 targets while finalising indicators for FY 2021-22, as noted in the data tables at Appendix B.

Appendix B provides a complete report of FRV Outcomes Framework data reported in Q2.

Community engagement

FRV did not meet the FY 2020-21 Q2 target for community engagement and the FSIM notes that FRV’s community engagement activities have been significantly impacted by COVID-19 related restrictions since the beginning of the reform.

FRV continues to deliver the Firelighting Consequence Awareness Program (Firecap), however FRV continues to be constrained in delivering a multitude of programs due to COVID-19 related restrictions and the number of staff returning positive COVID-19 tests and/or requiring to isolate after being identified as close contacts. FRV reports that this has impacted FRV’s ability to maintain minimum crew numbers for operational response, and as a result, delivery of non-urgent programs by firefighters was paused considering the impacts of COVID-19, for the safety of the community and firefighters.

FSIM notes that FRV has yet to meet the FY 2020-21 target and continues to explore and adapt its community engagement program to COVID-19 restrictions. FSIM supports FRV exploring alternative channels to deliver critical community engagement programs noting potential impacts on the prevention and preparedness capabilities of the Victorian community.

With the easing of restrictions, FSIM anticipates an increase in FRV community engagement activities in Q3, noting FRV have advised that there may be some reluctance for schools and senior groups to engage in such programs while the risk of contracting COVID-19 remains high

Skills maintenance

Specialist skills maintenance and core skills maintenance did not meet the Q2 FY 2020-21 Target.

Specialist skills maintenance

FRV reports that COVID-19 continues to impact results and that maintaining minimum staff crewing levels are being prioritised over specialist skills maintenance, such as marine and heavy rescue skills. FRV reports that COVID-19 restrictions and isolation requirements will continue to impact specialist skills maintenance results.

FSIM notes the importance of maintaining minimum staff crewing levels as a priority, whilst also balancing the need for adequate numbers of trained specialists to maintain operational capability.

Core skills maintenance

FRV reports on Division A (former Metropolitan Fire Brigade) data for its core maintenance skills indicator and noted that training was delayed due to COVID-19 related restrictions.

As noted in the Q1 FY 2021/22 Progress Report, this result does not include Division B (former CFA) data and therefore does not provide a complete picture of overall skills maintenance across FRV. FRV reports that the capture of Division B skills maintenance data will commence when all stations and platoons have been trained in the use of the skills maintenance database and are regularly inputting the skills maintenance drills they are currently undertaking. FRV has commenced a comprehensive program to deliver training to 1,400 Division B staff across 38 stations. At the end of Q2, 20% of staff (31.5% of stations) had received training.

FSIM notes that the inclusion of Division B staff in results for this indicator would significantly decrease the Q2 result. As the objective of this indicator is to ensure regular firefighting skills maintenance is undertaken at station level, it is important for FRV to include all relevant staff in its data capture to understand the requirements of core skill maintenance across the organisation.

Conversations in the Mess

FRV did not meet the FY 2020-21 Q2 target for stations and work sites visited under the ‘Conversations in the Mess’ program, noting FRV have previously reported that this program has been put on hold due to COVID-19 restrictions. FSIM notes FRV reviewed this program in Q2 and confirmed that the program will continue to be delivered in a face-to-face format. While the face-to-face format of the program was scheduled to commence in February 2022, the Fire Rescue Commissioner and broader Executive Leadership Team (ELT) have undertaken a comprehensive program of in-person station visits across Victoria and have attended FRV events such as Remembrance Day ceremonies and meetings with new recruits.

FSIM notes FRV leadership is continuing to undertake engagement activities with staff outside the formal ‘Conversations in the Mess’ program. Such activities support the reform priority to maintain an engaged workforce that feels safe and respected.

Hoarding and residential risk referrals

FRV met its residential and hoarding risk referrals FY 2020-21 Q2 targets.

FSIM notes that the FRV ‘At Risk Group’ was instrumental in helping FRV meet its hoarding risk referral target due to its continued follow up work on a backlog of notifications. The FRV ‘At Risk Groups’ Unit was unable to deliver additional education programs with external agencies due to limited resourcing. However, FRV is currently working with CFA to develop home fire safety e-learning modules for older people and people with disabilities, carers and support workers that provide information about home fire safety, how to reduce risk and how to prepare for fire events in the home. The module is scheduled for release in Q1 FY 2022-23.

FSIM acknowledges the efforts undertaken by the FRV ‘At Risk Group’ Unit to improve FRV’s hoarding risk referral processes, in addition to dedicated collaboration with the CFA to disseminate important safety information to ‘at risk’ groups.

This is FSIM’s fifth quarterly report on CFA’s and FRV’s Outcomes Framework results, having covered the timeframe from 1 July 2020 to 31 December 2021. Over this 18-month period, CFA and FRV have developed their initial (Year One) and subsequent (Year Two) Outcomes Frameworks to support active monitoring of progress towards reform outcomes and what success looks like for each agency. The full summary of CFA’s and FRV’s results against their respective Outcomes Frameworks is provided in Appendix A and Appendix B.

Achieving reform outcomes over the long term depends on good quality and consistent data and information. As indicators continue to be refined and data collection and analysis continues to mature, FSIM will be able to set out a clearer baseline to assess performance.

FSIM acknowledges CFA and FRV for their cooperation and active contribution to its monitoring and reporting activities, particularly given the significant administrative and operational pressures the two agencies continue to experience throughout this reporting period.

FSIM will continue to monitor and report on the Outcomes Framework reports in accordance with its legislated objectives to provide independent assurance to government and the community on the progress towards modern fire services that provide a safer Victoria.

The CFA Outcomes Framework uses an annual baseline total and year to date (YTD) baseline total in lieu of a target measure. This report includes the YTD baseline which is a rolling five-year average of the cumulative, quarter three performance against indicators.

Data against Year One performance indicators is available from page 11 in the report document.

2021-22 Fire Services Outcome Framework Progress Report Quarter 2

PDF 3.83 MB

(opens in a new window)

The FRV Outcomes Framework uses an annual target to assess performance against indicators. Some targets, however, are based on a YTD, cumulative result and have been identified as such in this report. FSIM notes that for this quarter, FRV has not confirmed targets as it is reporting on interim indicators in anticipation of reporting against Year Two Outcomes Framework indicators. FSIM has therefore aimed to make a comparison with FY 2020-21 Q2 targets where possible.

Data against Year One performance indicators is available from page 31 in the report document.

2021-22 Fire Services Outcome Framework Progress Report Quarter 2

PDF 3.83 MB

(opens in a new window)