As we enter this second year of outcomes reporting, I am pleased to note that both CFA and FRV have published their respective Year Two Outcomes Frameworks as an action arising from the Year One Fire Services Reform Implementation Plan (Year One Plan).

Although the frameworks are complete there is still more work to do by both CFA and FRV to finalise their indicators. Both CFA and FRV have reported against their Year One Outcomes Framework for this report, as they transition to year two indicators. I look forward to CFA and FRV finalising their indicators and expect to be able to report against them, and the related outcomes, in Q2. I also expect that CFA and FRV will finalise new indicators that provide greater insight into progress to reform outcomes.

In Q1, FRV experienced a significant decline against its specialist capability staff skills indicator, predominantly due to COVID-19 related restrictions that reduced training opportunities. To address this, FRV is trialling online training for some Emergency Medical Response (EMR) training activities. Although virtual training is one approach to help alleviate the training backlog, there is a risk that FRV may not meet established training targets. I will continue to monitor this situation in future quarters.

COVID-19 related restrictions also continue to impact CFA and FRV community engagement activities. The traditional methods of face-to-face delivery were not always possible for either agency but virtual adaptions were undertaken by the CFA such as for Property Advice Visits and Community Fireguard programs. FRV delivered the Firelighting Consequence Awareness Program (Fire-Cap) and commenced planning on a virtual Fire Education Program. FRV is also establishing plans to recommence community engagement and education activities upon Victoria meeting its 90% double vaccination target. CFA and FRV’s continued adaptation to COVID-19 restrictions should help equip the community with sufficient knowledge to prepare for emergency situations.

CFA and FRV are working to improve their workplace hazard reporting processes, not yet realised in the results. CFA did not meet its hazard reporting baseline but is developing a new hazard reporting checklist to improve reporting processes. FRV has resolved most of the IT issues experienced in previous quarters and has seen improved results.

CFA did not meet its persistent target of zero injuries or death to members of the public this quarter. However, CFA has initiated a pilot study with Monash University Accident Research Centre to link CFA incident reporting system data with Ambulance Victoria injury data. This improved data should assist CFA into the future to design appropriate interventions to decrease fire-related injuries.

Finally, I note that the Year Two to Five Fire Services Reform Implementation Plan (Year Two to Five Plan) was published in November. I look forward to considering how this plan helps guide the fire services progress in delivering reform outcomes and establishing modern fire services over the coming years.

Hon. Niall Blair
Fire Services Implementation Monitor

Definitions

Reference to fire services agencies

References to the fire services in this document relate specifically to CFA and FRV. Where more than one fire services agency is referenced in this document, they are listed in alphabetical order.

Comparison between agencies’ outcomes frameworks

As the two outcomes frameworks are currently based on the agencies’ own strategic and operational priorities, comparisons between the agencies should not be made.

Purpose of the report

This independent quarterly report provides observations on the progress of CFA and FRV against their respective outcomes-based fire services performance indicators.

Background

The Victorian Government has embarked on a significant 10-year reform agenda to enhance fire services reform in Victoria. In 2020, the Victorian Government released its Year One Fire Services Reform Implementation Plan (Year One Plan). The Year One Plan outlines a shared course of action for fire services agencies over the first year of reform to start to implement the vision set out in the 2017 Fire Services Statement (Fire Services Statement).

Fire Services Implementation Monitor (FSIM)

The Implementation Plan included an action to establish the Fire Services implementation Monitor (FSIM) under amendments to the Fire Rescue Victoria Act 1958 (the Act).

FSIM’s function is to assess the effectiveness of agencies in delivering against Implementation Plan actions and provide independent assurance to government and the community on the progress made towards modern fire services providing for a safer Victoria. FSIM is required to prepare and publish quarterly reports on CFA and FRV Outcomes Frameworks measures under s141 of the Act.

The role of Outcomes Frameworks in monitoring and evaluation

Outcomes frameworks are a monitoring and evaluation tool linking a vision to a set of outcomes, indicators (including targets), and measures. They help demonstrate the value and achievements of the fire services and can be used to identify change. They can also determine whether the services are moving towards intended outcomes and an agreed vision while also identifying areas for improvement and help focus monitoring and evaluation activities on potential high-risk issues.

Outcomes frameworks can be used to track an organisation’s progress or function at a sector level to measure the progress of multiple organisations towards a shared vision and a common set of outcomes.

FSIM’s approach

This report provides commentary where possible, on progress made towards the reform outcomes and on operational performance against the emergency sector’s vision for safer, more resilient communities.
FSIM has reviewed CFA’s and FRV’s quarterly results and provides a summary and analysis of the data from Q1 and, where possible, trend data across the first year of reformed operations, taking into consideration Implementation Plan priorities and the operational performance of both agencies.

Limitations on data

CFA and FRV have finalised their Year Two to Five Outcomes Frameworks, however both agencies are yet to fully finalise and transition to their new indicators. This report therefore continues to provide commentary against indicators from CFA and FRV’s Year One Outcomes Frameworks. FSIM intends to report against Year Two to Five Framework indicators in Q2 2021-22.

Publication of reports

FSIM publishes quarterly reports on this website. Publishing these quarterly reports promotes transparency and provides meaningful benchmarking that can measure the impact of the government's reform agenda on the fire services agencies, other emergency services stakeholders, and the broader community over time.

Fire services reform program

The Victorian Government released its Year One Fire Services Reform Implementation Plan in October 2020 that set out 54 actions to support the implementation of the new fire services model. The Fire Services Implementation Monitor Annual Report 2020-21 (FSIM Annual Report) provides an assessment of the first eight months of implementation progress of Victoria’s fire services reforms against activities included in this plan.

The Year One Plan concluded in October 2021 and the Year Two to Five Fire Services Reform Implementation Plan (Year Two to Five Plan) came into effect in November 2021. The Year Two to Five Plan transitions the focus of the reform from initiation towards strategic and future focused actions while continuing to operationalise and embed reform priorities. FSIM has a legislated role to monitor progress and effectiveness of agencies in delivering against actions outlined in this Year Two to Five Plan and to measure progress towards reform outcomes.

Year Two to Five Outcomes Frameworks

CFA and FRV published their respective Year Two Outcomes Frameworks¹ in October 2021 as an action arising from the Year One Plan. As noted in the Year Two to Five Plan, strong mechanisms to monitor and measure performance against the outcomes of the reform are critical. Effective monitoring holds agencies to account to ensure their service delivery priorities align with agreed outcomes. The agencies’ outcomes frameworks will continue to mature and are a critical tool to measure agency progress towards reform outcomes and operational performance.

Data sets to support indicators under the Year Two Outcomes Frameworks have not yet been finalised by the agencies. As an interim approach, CFA and FRV have reported data against Year One Outcomes Framework indicators for this FY 2021-22 Q1 quarterly report while they finalise the data that supports the year two indicators.

FSIM anticipates that both CFA and FRV will report against their respective Year Two Outcomes Framework indicators in Q2. Moving to reporting against the Year Two Outcomes Frameworks is a critical step to enable effective monitoring of fire service performance over the longer term.

Where particular outcomes or trends from the Year One Outcomes Frameworks have raised concerns for FSIM and are not included in the Year Two Outcomes Frameworks, FSIM may request agencies to provide additional information or data relating to year one indicators in future reports.

---

¹ Access CFA’s Year Two Outcomes Framework. Access FRV’s Year Two Outcomes Framework.

Country Fire Authority Q1 results overview

Community engagement

Although CFA did not meet the YTD community engagement baseline, it delivered formal programs via online platforms, leveraging and utilising the resources and capabilities CFA has invested in over the past year. For much of the quarter, CFA was not able to deliver any community engagement activities in person (its primary mechanism to deliver these programs) due to COVID-19 related restrictions. As restrictions are eased, FSIM expects to see an increase in CFA community engagement activities.

Hazard identification

CFA did not meet its YTD hazard identification baseline and experienced a further decrease in CFA volunteer and staff hazard reports from Q4 FY 2020-21. CFA reports that the significant reduction in on site non-response activity due to COVID-19 related restrictions has resulted in a reduction in hazard identification. CFA has committed to developing a new hazard reporting checklist that aims to assist with targeting key hazards for identification and control. Further promotion of reporting and identification of hazards among CFA staff and volunteers is critical to meeting the key reform priority of keeping staff and volunteers safe.

Fatalities and injuries

CFA did not meet its persistent target of zero fire fatalities and fire related injuries. There were four fire fatalities in Q1, which is an improvement on the three-year average (five fatalities). CFA also reported 15 fire-related injuries in Q1 which is an improvement of the three-year average (21 fire-related injuries).

CFA has identified that current data sets provide limited insight into linkages between action and outcomes relating to injuries. To address this issue, CFA partnered with the Monash University Accident Research Centre to improve data around community fire-related injuries. CFA has conducted a pilot program to link CFA incident reporting system data with Ambulance Victoria injury data. CFA reports that this pilot has attracted interest nationally. CFA intends to extend beyond the scope of the pilot program to be able to link data between the two systems to produce monthly reports of injury count and severity. CFA anticipates that this improved data will assist it into the future to design appropriate interventions with the aim to decrease fire-related injuries. FSIM welcomes this approach to better understand the linkages between action and outcome for this indicator.

Female leadership targets

CFA is making progress towards the government mandated target of 15% (909) females in brigade leadership roles² by mid-2022³. Under the definition of brigade leadership roles used for this target in the Year One Plan, this target is unlikely to be met by mid-2022. However the number of females in brigade leadership positions is on an upward trend from Q2 FY 2020-21.

CFA continues to provide opportunities for females to undergo active operational roles continually meeting the YTD baseline each quarter. Also, CFA has more than 50% females in senior paid staff positions including the CEO and several heads of directorate.

Female brigade leadership as defined in this target does not accurately incorporate the multitude of leadership positions at brigade level. CFA reports that the female brigade leadership target in Q2 will include Brigade Management and Group Management roles, reflecting a broader definition of brigade leadership than is reported in Q1. In line with findings in the FSIM Annual Report⁴ and previous outcomes reporting⁵, FSIM supports amendments to the definition of brigade leadership that incorporates operational and non-operational brigade leadership roles.

Fire Rescue Victoria Q1 results overview

FSIM notes that FRV is finalising data sets to report against the Year Two Outcomes Framework indicators. As an interim reporting approach, FRV provided data in Q1 FY 2021-22 against the year one indicators. Given the outcomes and indicators will change under the year two framework, FRV has not identified updated targets for Q1 indicators. FSIM has therefore provided a comparison between FY 2020-21 and Q1 FY 2021-22 results.

Skills maintenance

Q1 results are significantly below FY 2020-21 results for specialist capability staff (Technical Operations skills maintenance completed). FRV reported that COVID-19 related restrictions remain a key factor impacting the delivery of planned skills maintenance activities, due to cancellation of courses and lack of access to non-FRV facilities.

FRV traditionally conducts skills maintenance activities in a face-to-face forum. However, the organisation is now developing and trialling flexible approaches to the delivery of skills maintenance programs including using online platforms and working to harmonise skill sets across the fire agencies.

With limited opportunities to conduct skills maintenance, there is a risk that FRV may not meet established training targets. FSIM anticipates that with eased COVID-19 restrictions and implementation of flexible delivery approaches, FRV will be able to demonstrate an increase in specialist skills maintenance activities in future quarters.

FSIM also notes that the percentage of ‘core skills maintenance drills completed’ indicator does not include Division B staff. The objective of this indicator is to ensure regular firefighting skills maintenance is undertaken at station level.
Although the Q1 FY 2021-22 result is 6.5% below Q4 FY 2020-21, without the inclusion of Division B staff in the Q1 results, it does not provide a complete picture of core skill maintenance across the organisation. The inclusion of Division B staff in results for this indicator in future quarters would be required for FSIM to appropriately monitor the FRV staff core skills maintenance indicator.

Fire education and risk reduction programs

FRV reported 21 fire education and risk reduction programs delivered in Q1, which is an increase from zero in FY 2020-21. As of Q1, FRV will report on the Firelighting Consequence Awareness Program (Fire-CAP) in FY 2021-22 quarterly outcomes reports, which is delivered as a critical risk mitigation program. FRV advised that COVID-19 related restrictions prevented all non-urgent programs from being delivered.
FRV’s Community Education Unit is preparing to recommence all fire education and risk reduction programs once Victoria reaches the 90% vaccination status for people over the age of twelve and school-based education programs are planned to recommence in 2022. FSIM supports FRV’s preparation to resume community education programs following the easing of COVID-19 related restrictions.

Conversations in the Mess⁶

The Conversations in the Mess program was postponed due to COVID-19 related restrictions with no sessions delivered in Q1, a continuing trend from FY 2020-21. FRV plans to re-launch the program in February 2022 once relevant restrictions are eased. FRV reports that alternative leadership engagement activities were undertaken via online platforms but not reported for this indicator.

FSIM is supportive of FRV leadership continuing to undertake engagement activities with staff and notes that a reduction in staff engagement activities risks impacting the reform priority of maintaining an engaged workforce that feels safe and respected.

FRV Safe

FRV resolved system generated email issues reported last quarter that delayed investigation initiation response times for Division B employees. However, there are still IT issues that continue to reduce efficiency and FRV reported that further improvements to FRVSafe indicator results are possible.

Road crash rescue (RCR)⁷

Q1 results for road crash rescue (RCR) response are a decrease from FY 2020-21 results. During the quarter, nine RCR responses exceeded the benchmark time. However, FRV reported that three (33%) of these responses were not more than 30 seconds over the benchmark and a further two (22%) incidents appear to have been impacted by severe weather events and/or traffic congestion.

In Q4 (FY 2020-21) FRV noted that a potential contributing factor to not meeting the target was that former CFA areas, now within the FRV Fire District, were set at the 20-minute standard but are now reported on via FRV’s internal 13.5-minute standard. Although no RCR responses that exceeded the benchmark time were within the CAoV for this quarter, FSIM will continue to monitor the appropriateness of this target. FRV also reports that there are other contributing factors that increase response times including distance of appropriately equipped vehicles to incidents, traffic congestion, limited access to incident scene and competing on-scene response priorities.

FRV continues to expand RCR support capability, focusing on attainment of equipment and skills to access patients in road crashes across more FRV locations. The review of the State Road Crash Rescue Arrangements Victoria 2017, anticipated to be implemented in Q3 2021-22, may identify some of the causes for unmet response times for road crash rescue incidents.

While FSIM understands that the RCR indicator may not be included in FRV’s Year Two Framework, FSIM may continue to monitor this indicator to ensure the target is appropriate and reflects community risk.

Percentage of Emergency Medical Response (EMR) times within benchmark⁸

Q1 results for emergency medical response (EMR) includes an increase (2 per cent) from Q4 FY 2020-21, which is a positive result for the community. The window for survival for patients who stop breathing is recognised as 10 minutes which sets a different service delivery standard timeframe compared to structure fires (7.7 minutes). This means that FRV has a greater chance to arrive on scene within the timeframe and improves the chance of a successful outcome for this particular type of emergency.

FSIM will continue to work with FRV to understand how FRV’s actions contribute to improvements in quarterly results.

---

² Leadership defined as Captain and 1st through to 4th Lieutenant roles.

³ This target completion date is described in the Year One Fire Services Implementation Plan. Any amended targets and completion dates included in the Year Two to Five Fire Services Implementation Plan released in November 2021 will be used in future FSIM Quarterly Outcomes Framework Reports.

⁴ Fire Services Implementation Monitor Annual Report 2020-21, page 13

⁵ Fire Services Outcomes Framework Progress Report: Quarter 4 FY 2020-21, page 7

⁶ Conversations in the Mess refers to a formal program of visits to stations and work sites by FRV leadership aimed at improving engagement within FRV at a leadership level. The results are presented as the number of stations/work site visits as a percentage of the number of scheduled visits for the quarter.

⁷ This indicator calculates the number of emergency response times to road rescue callouts meeting the benchmark of 13.5 minutes as a percentage of the total calls for each quarter.

⁸ This indicator calculates the number of emergency response times to emergency medical response (EMR) callouts meeting the benchmark of 9.2 minutes as a percentage of the total calls for each quarter.

COVID-19 related restrictions significantly impacted FRV’s ability to conduct special skills maintenance training. FRV is trialling online training of some tasks however, with limited skills maintenance opportunities, there is a risk that FRV staff may not meet established training targets. CFA does not currently report on training capability although intends to do so as part of its Year Two Outcomes Framework. FSIM will monitor training in future quarters to identify potential risk, particularly as access to resources, training and support to keep firefighters safe are objectives of the fire services reform.

COVID-19 related restrictions have continued to constrain CFA and FRV’s ability to undertake community engagement activities and education programs. Both organisations are working to create more flexible modes of delivery through online activities, reducing the reliance on traditional face-to-face methods. FSIM will continue to monitor the impact of COVID-19 related restrictions on community engagement and education programs in future reporting.

FSIM welcomes CFA’s and FRV’s Year Two Outcomes Frameworks with the expectation that the outcomes and indicators included in the frameworks will better reflect the Fire Services Statement outcomes. FSIM expects that interim indicators reported in Q1 will be replaced in future reports by revised indicators, aligned to the Year Two Outcomes Frameworks, that are currently being finalised by CFA and FRV. FSIM acknowledges the work undertaken by CFA and FRV to revise their outcomes frameworks and notes that a shift to reporting on indicators aligned to the Year Two Outcomes Frameworks will enable more effective reporting by agencies and monitoring by FSIM from Q2 onwards.

The CFA Year One Outcomes Framework uses an annual baseline total and year to date (YTD) baseline total in lieu of a target measure. This report includes the YTD baseline which is a rolling five-year⁹ average of the cumulative, quarter one (Q1) performance against Year One Outcomes Framework indicators.

Data against Year One performance indicators is available from page 11 in the report document.

2021-22 Fire Services Outcome Framework Progress Report Quarter 1

PDF 1.03 MB

(opens in a new window)

---

⁹ The baseline is generally established as a five-year average however, some indicators are based on a one, two or three-year average and are identified as such in the report.

The FRV Outcomes Framework uses an annual target to assess performance against indicators. Some targets, however, are based on a year to date (YTD), cumulative result and have been identified as such in this report.

FSIM notes that for this quarter, FRV has not confirmed targets as it is reporting on interim indicators in anticipation of reporting against its Year Two Outcomes Framework indicators. FSIM has therefore aimed to make a comparison with FY 2020-21 results where possible.

Data against Year One performance indicators is available from page 23 in the report document.

2021-22 Fire Services Outcome Framework Progress Report Quarter 1

PDF 1.03 MB

(opens in a new window)