2022-23 Quarter 3 Fire Services Outcomes Framework Progress Report

This Fire Services Outcomes Framework – Progress Report provides the Country Fire Authority (CFA) and Fire Rescue Victoria (FRV) Quarter 3 (Q3) FY 2022-23 fire services performance indicator results, in line with FSIM’s legislative requirements under section 141 of the Fire Rescue Victoria Act 1958 (FRV Act).

This report presents quarterly results provided by CFA and FRV against their respective performance measures and agency commentary on quarterly performance. Where possible, FSIM has included historical data against measures provided by agencies from 1 July 2020 as a reference for the reader, noting that historical data is significantly limited due to changes in agencies’ outcomes frameworks in Q2¹ and other incidents outlined below.

FRV continues to experience problems accessing data following the cyber-attack incident in December 2022. This issue continues to significantly limit FRV’s reporting in Q3. Although FSIM understands that FRV continues to work to resolve data access and reporting issues, the lack of data and ability to report results reduces transparency in reporting progress toward outcomes.

Reporting on outcomes

Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. In developing and refining their outcomes frameworks, CFA and FRV each determined their strategic organisational priorities and the associated indicators and measures of performance to be included in those frameworks.

Tracking the progress of agencies towards their outcomes is a long-term proposition and trends or patterns may only be observable over an extended period. FSIM continues to update its outcomes reporting approach and may highlight particular indicators in the summary by exception where a result:

• shows a continuing off-track trend over repeated quarters
• demonstrates a notable achievement
• varies more than 5% from the agencies baseline or average over time
• is a significant change from the previous quarter (positive or negative).

FSIM notes that some performance measures are influenced by factors outside the control of CFA or FRV and in some instances, agencies may not be able to provide substantiated reasons, supported by evidence for changes to results.

In the context of the current data limitations, once per year, FSIM will work with agencies to provide an annual overview of agency outcome progress. With the foundational elements of each agencies’ outcomes frameworks now in place, agencies can begin to develop a deeper understanding of trend data and meaningful insights towards outcomes performance.

Maturing outcomes frameworks

The CFA and FRV Outcomes Frameworks signal the strategic priorities of each organisation, and each provides a set of indicators and measures to demonstrate progress towards their respective strategic goals. The ongoing review and refinement of both Outcomes Frameworks is important to assist agencies to assess progress towards delivering against their objectives and support informed, evidence-based decision making. Updates to the agencies’ outcome frameworks for Q3 are outlined below.

CFA’s Year Three Outcomes Framework

As reported in Q2, CFA has transitioned to the CFA Year Three Outcomes Framework. CFA has reported on a total of 29 quarterly indicators, and baselines for five indicators (1.2.1, 2.3.2, 2.3.3, 3.3.3, 4.1.1) will be introduced in future quarterly outcomes framework reports when historical data becomes available.

FRV’s Outcomes Framework

FRV finalised its Outcomes Framework in FY 2021-22. Ongoing impacts of the December 2022 cyber-attack on FRV’s systems (outlined below) continue to limit FRV’s ability to report this quarter, with data provided against seven of its 27 indicators. Of these seven indicators, FRV reported on twelve out of 67 measures. Measures are aimed to inform progress against a given indicator.

FRV cyber-attack

The cyber-attack on FRV in December 2022 continues to severely impact FRV’s information and computer technology systems and FRV’s Q3 FY 2022-23 reporting. As a result, FRV can only report on twelve measures in Q3, which is an increase of seven measures from Q2². FRV reports that it is undertaking work to investigate the cyber-attack and restore the IT environment. FRV also reports it is developing and implementing interim solutions to maintain system functionality and data collection functions, where possible. However, the timeframe for retrieval of data stored in those systems, and access to systems and applications is still unknown.

FRV has been able to retrieve some Q2 data and in Q3 has reported retrospectively on the following measures:

• % Workcover claims who ceased work with no return to work
• Average number of days for first return to work within quarter
• Lost Time Injury: severity rate per 1000 FTE
• Number of injuries and near misses per 1000 FTE
• Percentage of stations/work sites visited against scheduled visits
• Percentage of registered risks reviewed this quarter as per mandatory requirement in the government risk management framework to review all risks within 12 months.

FRV reports that it is implementing business continuity measures, workarounds and contingencies to ensure that service delivery continues. FRV reports that it is also exploring developing and implementing interim reporting solutions using data provided by ESTA to restore essential incident reporting functionality within FRV. This will include data for the period of the outage being made available to be reported on retrospectively. FRV advises that crews continue to turn out through the use of mobile phones, pagers and radios and FRV has maintained a range of corporate support functions during the cyber-incident. FRV is publishing regular updates for the community and stakeholders on its website at www.frv.vic.gov.au.

FRV’s continued data and reporting challenges impacts FSIM’s ability to provide more comprehensive outcomes performance commentary. In addition, the lack of data and reporting results in less transparency to the community on FRV’s progress against its identified outcome indicators.

Outcomes framework Q3 results commentary overview

FSIM has highlighted indicators that show an off-track trend and indicators that show a significant change from the previous quarter. As reported above, FRV’s data limitations constrain FSIM’s ability to assess FRV quarterly progress.

Indicators showing a continued off-track trend

Service Level Agreements (SLAs)

FRV’s ‘Service Level Agreement’ indicator monitors the number of SLAs (known by both agencies as Service Level Deeds of Agreements) that are agreed to in-principle by CFA and FRV. Consistently noted in previous reports, FSIM considers that completion of clear, agreed, and implemented SLAs between agencies underpins the broader success of the Fire Services Reform, particularly supporting "an effective and productive working relationship across the fire services and other emergency service agencies, building on the principle of interoperability”.³ These SLAs provide a foundation for role clarity and functions for volunteers and staff in both agencies and are required to harmonise procedures.

FRV’s Q3 result (88%) did not meet its FY 2020-21 defined target of 100%. The result has not changed since Q2 FY 2021-22 and represents a continued off-track trend. FRV does not report on the number of executed documents (i.e., signed by the Fire Rescue Commissioner and Chief Officer, CFA). Rather, this indicator refers to MOUs, SLAs and supporting appendices having been agreed in principle and approved through formal consultation. FSIM notes that the SLA and three SLA schedules are executed, and nine schedules remain outstanding.

Indicators with a variance of more than 5% from the agencies baseline or average over time

Fire suppression

In Q3, CFA did not meet its baseline for three operational indicators relating to Outcome 2.2 (fires are suppressed quickly and effectively). Specifically, CFA reported results outside of its 5% variance for indicator 2.2.3 (average time spent suppressing structure fires) and indicator 2.2.4 (average time to control bushfires). Indicators 2.2.3 and 2.2.4 report the average time from the time the first appliance arrives on scene of a fire until the stop message is received, which confirms that the scene is now safe and under control. In addition, FSIM notes that CFA’s Q3 result for indicator 2.2.2 (percentage of grass and scrub fires contained to 5 hectares) did not meet the baseline; however, the result was within CFA’s 5% variance tolerance.

These results indicate a drop in performance for these operational indicators when compared to the previous quarter. CFA advises it has not identified any specific factors that have contributed to Q3 performance for these indicators and notes that a range of factors can influence results. FSIM notes that there is no identified trend and that CFA met or was close to meeting the baseline for these indicators in Q2. FSIM will continue to monitor all three indicators in future reports to identify whether a trend is emerging.

 

Footnotes

[1] 2022-23 Quarter 2 Fire Services Outcomes Framework Progress Report

[2] 3.2.1a All operational firefighters turnover

• operational women firefighters
• operational men firefighters

3.2.1b Number of women firefighters in leadership roles

3.2.1c Proportion of women and men in leadership roles

• women in leadership roles (proportion of all women Operational Staff)
• men in leadership roles (proportion of all men Operational Staff)

3.4.5 Percentage of registered risks reviewed this quarter as per mandatory requirement in the government risk management framework to review all risks within 12 months

[3] Year Two to Five Fire Services Reform Implementation Plan, pg. 25

Definitions

Reference to fire services agencies

References to the fire services in this document relate specifically to CFA and FRV. Where more than one fire services agency is referenced in this document, they are listed in alphabetical order.

Comparison between agencies’ performance against published Outcomes Frameworks

As the two outcomes frameworks are based on the agencies’ own strategic and operational priorities and data, comparisons between the agencies should not be made.

Legislative requirements to report on outcomes

The performance measures that CFA and FRV report on are set by the agencies in their outcomes frameworks. Section 140(1) of the FRV Act requires CFA and FRV to each prepare an outcomes framework that sets out their respective outcomes-based fire service performance measures. Section 140(4) of the FRV Act further requires CFA and FRV to prepare and submit to FSIM a quarterly update that sets out their performance against their performance indicators.

Reporting and data limitations

CFA has reported on its Year Three Outcomes Framework in Q3. FSIM will continue to use historical data published in previous quarterly progress reports to track indicator trends over time where indicator business rules and baseline calculations in the Year Three Outcomes Framework remain consistent with earlier CFA outcomes framework iterations.

As noted above, FRV reports that the cyber-security breach of its systems has significantly reduced its capability to report on outcomes data. FRV has provided data and commentary against twelve measures which is an increase of seven measures from Q2.

Report structure

This report comprises FSIM summary (section one) and the CFA and FRV performance measure results received for this quarter (sections four to six). Where possible, FSIM presents performance results and historical trends. FSIM also includes CFA and FRV explanations of measures (business rules) and where appropriate, provides confirmation of whether the agency met or did not meet targets or baselines. FSIM also includes CFA and FRV commentary on indicator progress.

Fire Services Implementation Monitor (FSIM)

FSIM’s functions under the FRV Act are to assess the effectiveness of agencies in delivering against Implementation Plan actions and provide independent assurance to government and the community on the progress made towards modern fire services providing for a safer Victoria. FSIM is required to prepare and publish quarterly reports on CFA and FRV Outcomes Frameworks measures under s141 of the FRV Act.

Publication of reports

FSIM publishes quarterly reports at Fire Services Implementation Monitor publications | Victorian Government.

CFA’s Year Three Outcomes Framework uses a quarterly baseline which calculates the average result for Q3 over a specified time period (1, 2, 3 or 5 years). CFA reports the use of baseline data in lieu of specified targets for many measures. CFA’s target therefore is to ‘increase’ or ‘decrease’ (as appropriate) and improve performance with consideration to baseline data.

CFA’s Year Three Outcomes Framework was first reported in Q2 FY 2022-23. The CFA Year Three Outcomes Framework has revised some indicator baselines and business rules, and this quarter FSIM have reported historical data trends and baselines only where business rules from the CFA Year Three Outcomes Framework remain consistent with the CFA Year Two Outcomes Framework.

CFA categorises Year Three Outcomes Framework indicators into four domains:

1. strong relationships to keep communities safe
2. a responsive and innovative volunteer service
3. high performing volunteer and paid workforce
4. a continuous improvement culture.

Data is available from page 8 in the report document.

FRV report that the majority of indicator data is unavailable this quarter. However, FRV has reported on seven Year Two Outcomes Framework indicators in Q3.

FRV categorises Year Two Outcomes Framework measures into three domains: Prevention and Preparedness; Response and Recovery; and Organisational Excellence. In Q3, FRV provided commentary against indicators under domain three: Organisational Excellence.

FRV does not report a baseline or target for its indicators and measures and does not qualify results as “on track” or “meeting baseline”.

Due to the cyber-attack on FRV IT systems, FRV’s ability to report this quarter is limited to seven of its 27 indicators.

Data is available from page 23 in the report document.