This Fire Services Outcomes Framework – Progress Report provides the Country Fire Authority (CFA) and Fire Rescue Victoria (FRV) Quarter 3 (Q3) FY 2022-23 fire services performance indicator results, in line with FSIM’s legislative requirements under section 141 of the Fire Rescue Victoria Act 1958 (FRV Act).
This report presents quarterly results provided by CFA and FRV against their respective performance measures and agency commentary on quarterly performance. Where possible, FSIM has included historical data against measures provided by agencies from 1 July 2020 as a reference for the reader, noting that historical data is significantly limited due to changes in agencies’ outcomes frameworks in Q21 and other incidents outlined below.
FRV continues to experience problems accessing data following the cyber-attack incident in December 2022. This issue continues to significantly limit FRV’s reporting in Q3. Although FSIM understands that FRV continues to work to resolve data access and reporting issues, the lack of data and ability to report results reduces transparency in reporting progress toward outcomes.
Reporting on outcomes
Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. In developing and refining their outcomes frameworks, CFA and FRV each determined their strategic organisational priorities and the associated indicators and measures of performance to be included in those frameworks.
Tracking the progress of agencies towards their outcomes is a long-term proposition and trends or patterns may only be observable over an extended period. FSIM continues to update its outcomes reporting approach and may highlight particular indicators in the summary by exception where a result:
- shows a continuing off-track trend over repeated quarters
- demonstrates a notable achievement
- varies more than 5% from the agencies baseline or average over time
- is a significant change from the previous quarter (positive or negative).
FSIM notes that some performance measures are influenced by factors outside the control of CFA or FRV and in some instances, agencies may not be able to provide substantiated reasons, supported by evidence for changes to results.
In the context of the current data limitations, once per year, FSIM will work with agencies to provide an annual overview of agency outcome progress. With the foundational elements of each agencies’ outcomes frameworks now in place, agencies can begin to develop a deeper understanding of trend data and meaningful insights towards outcomes performance.
Maturing outcomes frameworks
The CFA and FRV Outcomes Frameworks signal the strategic priorities of each organisation, and each provides a set of indicators and measures to demonstrate progress towards their respective strategic goals. The ongoing review and refinement of both Outcomes Frameworks is important to assist agencies to assess progress towards delivering against their objectives and support informed, evidence-based decision making. Updates to the agencies’ outcome frameworks for Q3 are outlined below.
CFA’s Year Three Outcomes Framework
As reported in Q2, CFA has transitioned to the CFA Year Three Outcomes Framework. CFA has reported on a total of 29 quarterly indicators, and baselines for five indicators (1.2.1, 2.3.2, 2.3.3, 3.3.3, 4.1.1) will be introduced in future quarterly outcomes framework reports when historical data becomes available.
FRV’s Outcomes Framework
FRV finalised its Outcomes Framework in FY 2021-22. Ongoing impacts of the December 2022 cyber-attack on FRV’s systems (outlined below) continue to limit FRV’s ability to report this quarter, with data provided against seven of its 27 indicators. Of these seven indicators, FRV reported on twelve out of 67 measures. Measures are aimed to inform progress against a given indicator.
FRV cyber-attack
The cyber-attack on FRV in December 2022 continues to severely impact FRV’s information and computer technology systems and FRV’s Q3 FY 2022-23 reporting. As a result, FRV can only report on twelve measures in Q3, which is an increase of seven measures from Q22. FRV reports that it is undertaking work to investigate the cyber-attack and restore the IT environment. FRV also reports it is developing and implementing interim solutions to maintain system functionality and data collection functions, where possible. However, the timeframe for retrieval of data stored in those systems, and access to systems and applications is still unknown.
FRV has been able to retrieve some Q2 data and in Q3 has reported retrospectively on the following measures:
- % Workcover claims who ceased work with no return to work
- Average number of days for first return to work within quarter
- Lost Time Injury: severity rate per 1000 FTE
- Number of injuries and near misses per 1000 FTE
- Percentage of stations/work sites visited against scheduled visits
- Percentage of registered risks reviewed this quarter as per mandatory requirement in the government risk management framework to review all risks within 12 months.
FRV reports that it is implementing business continuity measures, workarounds and contingencies to ensure that service delivery continues. FRV reports that it is also exploring developing and implementing interim reporting solutions using data provided by ESTA to restore essential incident reporting functionality within FRV. This will include data for the period of the outage being made available to be reported on retrospectively. FRV advises that crews continue to turn out through the use of mobile phones, pagers and radios and FRV has maintained a range of corporate support functions during the cyber-incident. FRV is publishing regular updates for the community and stakeholders on its website at www.frv.vic.gov.au.
FRV’s continued data and reporting challenges impacts FSIM’s ability to provide more comprehensive outcomes performance commentary. In addition, the lack of data and reporting results in less transparency to the community on FRV’s progress against its identified outcome indicators.
Outcomes framework Q3 results commentary overview
FSIM has highlighted indicators that show an off-track trend and indicators that show a significant change from the previous quarter. As reported above, FRV’s data limitations constrain FSIM’s ability to assess FRV quarterly progress.
Indicators showing a continued off-track trend
Service Level Agreements (SLAs)
FRV’s ‘Service Level Agreement’ indicator monitors the number of SLAs (known by both agencies as Service Level Deeds of Agreements) that are agreed to in-principle by CFA and FRV. Consistently noted in previous reports, FSIM considers that completion of clear, agreed, and implemented SLAs between agencies underpins the broader success of the Fire Services Reform, particularly supporting "an effective and productive working relationship across the fire services and other emergency service agencies, building on the principle of interoperability”.3 These SLAs provide a foundation for role clarity and functions for volunteers and staff in both agencies and are required to harmonise procedures.
FRV’s Q3 result (88%) did not meet its FY 2020-21 defined target of 100%. The result has not changed since Q2 FY 2021-22 and represents a continued off-track trend. FRV does not report on the number of executed documents (i.e., signed by the Fire Rescue Commissioner and Chief Officer, CFA). Rather, this indicator refers to MOUs, SLAs and supporting appendices having been agreed in principle and approved through formal consultation. FSIM notes that the SLA and three SLA schedules are executed, and nine schedules remain outstanding.
Indicators with a variance of more than 5% from the agencies baseline or average over time
Fire suppression
In Q3, CFA did not meet its baseline for three operational indicators relating to Outcome 2.2 (fires are suppressed quickly and effectively). Specifically, CFA reported results outside of its 5% variance for indicator 2.2.3 (average time spent suppressing structure fires) and indicator 2.2.4 (average time to control bushfires). Indicators 2.2.3 and 2.2.4 report the average time from the time the first appliance arrives on scene of a fire until the stop message is received, which confirms that the scene is now safe and under control. In addition, FSIM notes that CFA’s Q3 result for indicator 2.2.2 (percentage of grass and scrub fires contained to 5 hectares) did not meet the baseline; however, the result was within CFA’s 5% variance tolerance.
These results indicate a drop in performance for these operational indicators when compared to the previous quarter. CFA advises it has not identified any specific factors that have contributed to Q3 performance for these indicators and notes that a range of factors can influence results. FSIM notes that there is no identified trend and that CFA met or was close to meeting the baseline for these indicators in Q2. FSIM will continue to monitor all three indicators in future reports to identify whether a trend is emerging.
Footnotes
[1] 2022-23 Quarter 2 Fire Services Outcomes Framework Progress Report
[2] 3.2.1a All operational firefighters turnover
- operational women firefighters
- operational men firefighters
3.2.1b Number of women firefighters in leadership roles
3.2.1c Proportion of women and men in leadership roles
- women in leadership roles (proportion of all women Operational Staff)
- men in leadership roles (proportion of all men Operational Staff)
3.4.5 Percentage of registered risks reviewed this quarter as per mandatory requirement in the government risk management framework to review all risks within 12 months
[3] Year Two to Five Fire Services Reform Implementation Plan, pg. 25
Updated