Country Fire Authority Q3 results overview
CFA has transitioned to its Year Two Outcomes Framework and provided data for all measures in the year two framework. CFA recognises that there are improvements to be made to link the measured data to outcomes. The planned continuous improvement approach to mature CFA’s outcomes framework includes improving data quality and integrity, refining baselines and setting appropriate targets where relevant to demonstrate progress towards intended outcomes. Currently, CFA uses a range of approaches to set baselines, some of which include pre-reform data, and some which have had pre-reform data excluded. CFA’s broader outcomes review and planning will consider how baselines can more accurately reflect post-reform operations and other improvement opportunities.
Appendix A provides a complete report of CFA Outcomes Framework data reported in Q3.
Introduction
CFA’s Year Two Outcomes Framework comprises four goals that form the basis of CFA’s outcome reporting and related measures and indicators. These goals are:
- We put the community at the centre of everything we do.
- We deliver programs and services that make a positive difference.
- We provide a great place to volunteer and work.
- We are a progressive emergency service.
This section provides insights into CFA’s progress towards these goals.
Overview of progress towards outcomes
Putting the community at the centre of everything CFA does
CFA’s outcomes framework quarterly measure for its community engagement goal captures the number of community members engaging with CFA. This measure includes CFA engagement activity via community meetings and workshops, online sessions and e-learning modules, visits to homes and properties, and at community events or places.
CFA reported 40,179 engagements year to date (YTD) as at Q3 FY 2021-22. CFA commenced collating data for this indicator in Q4 FY 2020-21 and has advised that a baseline for this indicator will be considered in the development of CFA’s Year Three Outcomes Framework. This Q3 result is an increase of 22,600 community members engaging with the CFA from Q2 2021-22 FY. CFA reports that factors contributing to this increase include:
- easing of COVID-19 restrictions
- an increase in programs and products developed for delivery (including but not limited to, the development of the Preparing Vulnerable People program, review and update of Farm Fire Sector products, CFA member training in community engagement delivery)
- reporting captures community engagement activities and services that were not previously captured post service delivery (for example door knocking and online self-paced bushfire safety training).
CFA has introduced a web-based monitoring, evaluation and reporting system (ART) to capture data collected through community engagement activities. CFA uses ART to better understand how it engages with communities and the impact of engagement activities on the community.
In Q4 FY 2021-22, CFA will report on annual metrics to assess the effectiveness of community engagement considering community trust in CFA and community knowledge of fire risk. FSIM looks forward to reviewing the effectiveness of CFA community engagement in Q4 with the release of community trust and community knowledge of fire risk indicators.
Delivering programs and services that make a positive difference
CFA’s programs and service goal captures operational performance data including the number and containment of fires, fatalities and injuries, stock loss, and preventative intervention including smoke alarm installation.
CFA has identified 13 indicators to demonstrate progress towards this outcome. Generally, CFA is tracking close to baselines set for these indicators, however important indicators in prevention, suppression and harm reduction have not met baselines for this goal. Key measures where CFA has not met baselines or targets relate to decreases in the number of grass and scrub fires, decreases in fire fatalities and injuries and reduced stock loss due to fires.
CFA reported a year to date total of 2,353 grass and scrub fires which is above the YTD baseline of 1,802. CFA notes that this increase is in line with previously recorded and comparable seasonal variations. While a decrease against the baseline for grass and scrub fires was not met, CFA’s containment of grass and scrub fires to 5 hectares measure met its baseline with a result of 92.7% of fires contained. Containment of these fires to five hectares reduces the risk of damage to large areas providing good outcomes for community.
CFA’s aspirational target is for zero fire fatalities and injuries in the Country Area of Victoria. As reported in Q2 2021-22, CFA is developing a suite of tools to inform the development of fire related fatality and injury prevention, preparedness and response programs and activities. Given these tools are not yet finalised or embedded in CFA’s operational processes, FSIM is unable to determine what impact these programs may have on reducing the number of fire-related fatalities and injuries.
CFA reported a YTD total of 4,122 stock lost, which is above the YTD baseline of 1,508. In Q3, CFA retrospectively revised its Q2 stock loss data to a total of 4,115 stock lost with an additional seven stock lost in Q3. FSIM has noted in prior reports the inconsistencies with stock losses reported. FSIM encourages CFA to review business rules and reporting categories to inform its Year Three Outcomes Framework to report against this indicator more effectively.
In Q2, CFA commenced reporting on data that records the number of smoke alarms installed and replaced in homes and the provision of fire prevention devices to vulnerable community members to determine if the program makes a difference in reducing harm from fires in vulnerable communities. While CFA is working in partnership with other agencies like the Red Cross to increase this type of activity, CFA’s data collection and analysis must continue to mature to draw clearer connections between the causes and results.
FSIM notes that CFA performed well this quarter regarding average time spent suppressing structure fires with an average time of 53 minutes and 51 seconds which is more than 5 minutes below the baseline. While this is a good outcome for the community and moves CFA towards its indictor of fires being suppressed quickly and effectively, clear linkages to programs that are contributing to this result are not yet available.
Providing a great place to volunteer and work
CFA’s “great place to volunteer and work” goal captures OH&S data, diversity measures, training progress and delivery against corporate requirements.
CFA is generally tracking well against baselines for this goal. However, hazard reporting continues to track significantly below the baseline, with ongoing impacts of COVID-19 related restrictions impacting onsite presence. CFA intends to implement a new hazard reporting checklist in Q4 FY 2021-22 that will assist with targeting key hazards for identification and control.
FSIM notes that CFA’s five-year baseline data for several measures including hazard reporting, emergency response injuries, and WorkCover claims includes data pertaining to former CFA staff who were transferred to FRV as part of the fire services reforms. As such, baselines for this data are not wholly reflective of the organisation post reform. In Q2, CFA noted that a new WorkCover claim measure would be introduced to normalise results, however this has not yet been completed for inclusion in the Q3 report.
A progressive emergency service
CFA’s progressive emergency service goal captures data on timeliness of reporting and implementing audit recommendations, workplan delivery and number of after action reviews.
CFA continues to have a high number of overdue internal audit recommendations, noting 85% of the internal audit recommendations continue to be overdue. CFA has advised that it has taken steps to address overdue internal audit recommendations including appointing a new internal audit supplier, strengthening arrangements for audit planning and delivery, and implementing processes for management action plans and for reporting to the Audit and Finance sub-committee of the CFA Board.
FSIM is supportive of CFA’s newly introduced measure of after action reviews to support continuous improvement.
Fire Rescue Victoria Q3 Results Overview
FRV’s transition to the Year Two Outcomes Framework continues to progress. FRV continues to report on Year One Outcomes Framework measures and has provided seven year two framework indicators in Q3. As noted in previous reports, FRV has not developed FY 2021-22 targets for year one framework measures as it did not plan to report on these measures beyond June 2021. FRV continues to use the FY 2020-21 targets as noted in the data tables at Appendix B.
FRV has provided FSIM with a Measurement and Monitoring Plan that outlines a pathway towards the development and publication of outcomes measures against the year two framework. FRV’s goal is to have at least one outcome measure for each outcome indicator to provide evidence of the success or progress towards the indicator’s desired outcomes. Developing a complete range of measures for the framework is likely to be an ongoing iterative project over several years, as various measures are proposed and trialled.
To accommodate FRV’s measurement approach and ensure the public have a level of transparency on FRV’s fire service performance outcomes, FSIM will continue to publish year one framework measures until such time as FRV has more comprehensive set of measures. The risk with this approach is that FRV has not set amended targets for the year one framework measures, and it is unclear if FY 2020-21 targets are still appropriate.
Appendix B provides a complete report of FRV outcomes framework data reported in Q3.
Introduction
FRV continues to build its data set for reporting against its Year Two Outcomes Framework. As an interim measure, FRV continues to provide data against its Year One Outcomes Framework that reports against the three domains of community, service and people.
FRV’s year two framework comprises three domains, and related goals, that form the basis of FRV’s outcome reporting and related measures and indicators. These goals are:
- Victorians understand risk in their local environment and know how to prevent and prepare for emergency incidents.
- Victorians can rely on a world-class fire and rescue emergency response (FRV is not currently reporting on data for this goal).
- Victorians value FRV as a leading, progressive and accountable fire and rescue service.
This section provides insights into FRV’s progress against its year one framework domains and towards its year two framework goals.
Overview of progress against year one domains
Community
FRV’s community domain captures data on community education programs, risk referrals, and containment of fires.
While FRV has made significant progress on providing community education and risk reduction programs from previous quarters, it remains below the target set in FY 2020-21. FRV anticipates a higher result for Q4 noting that COVID-19 related restrictions were a key challenge to the delivery of programs, particularly FireEd in schools. FSIM is supportive of a strong focus from FRV to address the backlog of education programs to be delivered to the community.
FRV continues to report below the target for containment of structure fires. FSIM notes that containment information is incomplete for a small number of fires within this dataset.
FRV has reported an above average result for residential and hoarding risk referrals this quarter, building on the work from Q2 to manage the backlog of notifications. Hoarding and residential risk referrals enable targeted assistance and education aimed to influence and reduce the impact of fire within the community. FSIM supports the use of hoarding and residential risk referrals to inform the Safer Homes Year 2 Outcomes Framework Indicator in future outcomes reporting.
Service
FRV’s service domain captures data on fleet availability, response times, service level agreement status and specialist capability skills maintenance.
Service Level Agreements are a key outstanding issue arising from the reform’s implementation. The reported figure of 88% relates to agreements in principle and not executed agreements. Executing these agreements is a necessary step to ensure clarity of role and functions for volunteers and staff in both agencies and is required to inform updates and harmonisation of procedures for staff. Delays in finalising these agreements has led to downstream issues impacting the community. Issues include delays in land use planning approvals, complex work arounds for staff completing the work, and interim work arrangements which differ from the initially agreed “in principle” approach and involve quality assurance processes that impact staff and volunteer morale. The impacts of delayed SLAs include inefficiencies, higher costs, and delayed service provision to the community. There are significant challenges in finalising this program of work and FSIM considers finalising SLAs to be a priority issue for both agencies as they underpin agencies’ agreement on complementary fire service delivery.
FRV’s positive fleet availability result means that firefighters can respond to emergencies with fleet that is maintained and available. Response times in general are tracking close to the FY 2020-21 target with 88% of structure fires responded within 7.7 minutes in FRV’s response area, noting that this target relates to FRV’s overall footprint, and not individual maintenance areas. FSIM notes that most (240 of 300 incidents) that missed the 7.7 benchmark responded within an additional two minutes however 20 incidents missed the benchmark by more than 5 minutes.
In situations where both the response time benchmark and the containment benchmark are not met, there may be a greater chance of significant loss to the property. For the nine months to 31 March 2022, the number of incidents where both response time and containment were not met is 40 incidents, which represents (1.6%) of total emergency structure fires. FSIM notes that FRV has not incorporated response time benchmark and containment benchmark reporting in its year two framework however, FRV will continue to provide this data through its quarterly response time reporting requirements and ongoing public reporting against BP3 measures.
FRV is still reporting below target results for road crash rescue. FRV notes that key contributing factors to not meeting benchmark times for this quarter include traffic congestion, particularly for incidents that occurred on freeways and distances of over 12 km to incident.
FRV has progressed significantly from the previous quarter regarding specialist capability skills maintenance, increasing the percentage of skills maintenance completed from 29.5% in Q2 to 76% in Q3. Significant increases to emergency medical training, marine rescue, heavy rescue and trench rescue training programs have all significantly increased this quarter and FSIM commends FRV for the work undertaken to achieve this result.
People
FRV’s people domain captures data on core skills maintenance, FRVSafe, staff engagement, workforce turnover, staff numbers, engagement with local government authorities (LGAs) and representation at state emergency management meetings.
FRV is slightly below target for core skills maintenance with 87.8% of Division A core maintenance drills scheduled and completed. FRV intends to commence reporting on Division B data from 1 July 2022.
FRV has made significant progress in meeting its “stations and work sites visited against schedule” measure, increasing from 0% to 86.4% in Q3 as a result of the lifting of COVID-19 restrictions. In Q2, FRV reviewed its “Conversations in the Mess” program to align it with other employee engagement initiatives to facilitate state-wide reach. These results are a significant development as FSIM continues to receive feedback from staff across the state regarding the ongoing need for timely and relevant communication on key changes and general reform progress.
FRV reports how quickly OH&S incidents are reviewed and closed out in its internal OH&S incident reporting program, FRVSafe. Previously, FSIM reported that FRV had IT issues with incident report escalation to responsible staff in FRV, which contributed to FRV being below target for both indicators. FRV reports that work is still ongoing to remedy these issues, however FRV is tracking close to target in implementing remedial actions.
FRV’s workforce turnover rate is tracking below target at around 2.6%. A low turnover rate means that FRV has fewer available roles to fill, reducing the opportunities to change the diversity profile of the organisation. FRV has undertaken workforce planning to identify the workforce profile and is implementing strategies to address areas where there is an aging workforce profile. FRV has established “indicative forecast exits” to assist with workforce planning and support the development of a more diverse and inclusive workforce.
FRV provides advice to Local Government in the preparation of Municipal Emergency Management Plans (MEMP). FRV has reported a result that is below target, however legislative changes have resulted in FRV not being represented on every MEMP Committee. Although not ideal, FRV reports that MEMPs are still being prepared when it is not represented at an MEMP Committee. FRV recognises that providing its expertise and knowledge regarding FRV linked responsibilities/activities and visibility is critical to the development of these plans and is working on a strategy to ensure an appropriate level of representation is achieved at these meetings.
Overview of progress towards year two outcomes
As noted above, FRV provided data against seven of the 30 Year Two Outcomes Framework indicators in Q3, with a number of measures informing each indicator.
Domain 1: Prevention and Preparedness. Victorians understand risk in their local environment and know how to prevent and prepare for emergency incidents
Outcome: Safer homes
FRV’s safer homes outcome captures data on preventable structure fires attended by FRV and the number of homes with working smoke alarms.
FRV notes there was no noticeable deviation between the past three quarters and baseline data and that the higher number of preventable residential structure fires in Q1 is consistent with baseline data. Over time, FSIM expects FRV to be able to explain how its prevention programs and fire safety advocacy impacts trends in preventable residential structure fires.
Outcome: A well-regulated built environment
FRV’s well-regulated built environment outcome captures data on preventable fires in high-risk buildings, impacts from fire due to early detection and decrease in unwanted false alarms.
FRV reports a consistent increase in premises with six or more false alarms on different days over a 12-month rolling period, since the establishment of FRV. FRV notes the increase could be partially due to COVID-19 related restrictions and will continue to monitor this measure. FSIM expects that FRV will be able to focus attention on those premises types with the most frequent false alarms so that underlying systemic issues are addressed and corrected at these premises. FSIM looks forward to data that shows how FRV intervention decreases the number of false alarms.
Outcome: Enabled, empowered and resilient communities
FRV’s enabled, empowered and resilient communities outcome currently uses the number of non-structure fire incidents attended by FRV in FRV Fire Districts according to incident type. FRV is yet to finalise and report on subsequent measures that will inform this outcome in future reports.
FRV notes that Q1 data shows a lower number of incidents for all types of incidents which it attributes to non-structure fires being more prevalent in the summer months (such as in Q3) and COVID-19 related restrictions still in place during Q1, reducing the number of vehicles on the road. FSIM expects FRV to be able to explain how FRV programs and actions will lead to an enhanced capacity to improve community resilience and emergency prevention as the data matures and year two outcomes’ measures are confirmed.
Domain 3: Organisational Excellence. Victorians value FRV as a leading, progressive and accountable fire and rescue service
Outcome: A diverse and inclusive organisation
FRV’s diverse and inclusive organisation outcome compares workforce turnover by men and women and measures the number and proportion of women in leadership roles.
FRV notes that commentary against this measure is limited due to the small number of measures and that only one quarter of data has been collected to date. FRV anticipates that the proportion of women in leadership roles will increase in line with, or at a faster rate than, previous years.
Updated