About the state disability plan outcomes framework

The original outcomes framework was developed to help the Victorian Government understand if progress was being made for Victorians with disability under Absolutely everyone: state disability plan 2017–2020 (the old plan).

It was designed to measure, track and report on outcomes for people with disability. It aligned to the 4 pillars in the old plan.

The 4 pillars reflect areas in life that people with disability told us are important:

• Pillar 1 – Inclusive communities. Actions under this pillar aim to improve access, inclusion and participation in the community.
• Pillar 2 – Health, housing and wellbeing. Actions under this pillar aim to improve health and wellbeing outcomes.
• Pillar 3 – Fairness and safety. Actions under this pillar aim to uphold and protect the rights of people with disability. They also ensure high-quality and strong safeguards are in place.
• Pillar 4 – Contributing lives (‘Opportunity and pride’ in the current plan). Actions under this pillar aim to ensure that people with disability have the benefit of good education and training and contribute to the economy as consumers, employees and employers.

The research team at the time developed outcomes, indicators and measures.

• Outcomes are the things people with disability wanted to achieve.
• Indicators assess the positive change we want for an outcome. For example, ‘People with disability report improved physical health’ was an indicator for the health outcome.
• Measures described how changes in the Indicators will be assessed using data.

In March 2022, Inclusive Victoria: state disability plan 2022–2026 (the current plan) was launched. We needed to consider whether we could use the outcomes framework for the old state disability plan to assess progress against the outcomes under the current plan.

We also needed to make sure the outcomes framework is accessible and understandable.

The outcomes framework is comprehensive, with 14 outcomes, 37 indicators, and many more measures that are used for monitoring.

With so many indicators and measures, it was challenging to produce a useable and understandable report on progress against the aims of the current plan.

There was a risk that reporting measures for all indicators could lead to a data deluge. This could make the outcomes framework difficult to apply and understand in a meaningful way.

We found that the old plan and the current plan share many of the same pillars and that the outcomes are almost identical between the 2 plans.

We also found that the indicators in the old plan were the same as the indicators in the current plan, aside from some realignment of indicators to different outcomes and pillars.

Overall, we found strong overlap in the indicators between the old and current plans.

This meant we could use the outcomes framework as the starting point for monitoring progress for the current plan.

However, we wanted to simplify the presentation of the outcomes framework. We met with VDAC to discuss the project. We formed a VDAC outcomes framework subcommittee.

We met with the VDAC subcommittee 3 times during July and early August 2024 to work through the 37 existing indicators line by line. We aimed to select one headline indicator for each of the 14 outcomes in the current plan.

This was not always possible, as sometimes more than one indicator was an important headline summary for an outcome.

We recommend keeping indicators we did not choose as headline indicators in the framework.

In some cases, the decision not to make an indicator a headline indicator was due to a lack of data for this reporting period. For others, we need to explore whether we could use administrative data, such as the National Disability Data Asset, in the future.

We note where VDAC highlighted an indicator was important, but where the indicator was not made a headline indicator this was because there was no readily available and suitable data.

We worked to incorporate VDAC’s expertise with our own expert input and knowledge of available data sources. This included updating headline indicators and accompanying measures. This involved investigating the availability and relevance of potential data sources and identifying specific variables within data sources.

We also assessed if variables were appropriate to measure changes in the relevant indicators and were consistent with VDAC’s feedback.

In addition, we investigated how variables and scales have been used in the wider literature to ensure consistency and robustness.

This process of converting the headline indicators into quantifiable measures involved experts in data science. It has produced transparent, reproducible results that can be replicated for reporting against the current plan in the future.

The aims of the refresh were to ensure that the outcomes framework is:

• streamlined – moving from 37 indicators to 19 headline indicators
• simple to understand – we removed and amended language that was potentially confusing, based on feedback from VDAC and the research team
• consistent with VDAC’s advice – VDAC’s assessment about what is most important for people with disability now and into the future was incorporated into Headline Indicator selection
• measurable – we focused on selecting headline indicators where suitable data is likely to be readily available.

Indicator: People with disability report increased social connections
Headline indicator: social connection. There was only one indicator. VDAC advised making this the headline indicator.
Measures: Access to social support

Indicator: People with disability report increased activity in the community
Headline indicator: activity in the community. VDAC advised making this the headline indicator.
Measures: Participation in community events, and Satisfaction with community participation

Indicator: An increase in positive community attitudes towards people with disability
Headline indicator: Not chosen due to lack of data.

Indicator: People with disability report increased connections to culture and community
Headline indicator: Not chosen because too similar to the preferred headline indicator.

Indicator: Local neighbourhoods are more inclusive
Headline indicator: Not chosen as headline indicator due to lack of data.

Indicator: Public transport is more accessible
Headline indicator: accessible transportation. VDAC advised that accessible transport is an important indicator.
Measures: Use of public transport

Indicator: More spaces and places with universal design
Headline indicator: spaces and places with universal design. VDAC advised that accessible transport is only effective if destinations are accessible too. This indicator captures accessibility of places.
Measures: Access to buildings and facilities

Indicator: More accessible or adapted footpaths and crossings
Headline indicator: Not chosen as headline indicator due to lack of data.

Indicator: Increased access to transport
Headline indicator: Not chosen as headline indicator due to most data sources used previously are not available or do not cover the current reporting period.

Indicator: People with disability report greater mobility
Headline indicator: mobility. There was only one indicator, and VDAC advised that we should elevate it to headline status.
Measures: Affordable transportation

Indicator: More people with disability have access to affordable housing
Headline indicator: affordable housing. VDAC advised that cost of living was important
Measures: Affordable housing, and Capacity to pay bills.

Indicator: More people with disability have access to suitable housing
Headline indicator: suitable housing. VDAC advised that a headline indicator for housing needs to consider suitability
Measures: Housing satisfaction

Indicator: More people with disability have access to stable and secure housing
Headline indicator: VDAC advised this indicator is closely related to the headline indicator chosen.

Indicator: People with disability report improved physical health
Headline indicator: physical health. VDAC deemed each indicator in the health domain is important.
Measures: Good general health

Indicator: People with disability report an increase in healthy living
Headline indicator: healthy living
Measures: Smoking, and Affordable healthcare

Indicator: People with disability report improved mental health
Headline indicator: mental health
Measures: Psychological distress

Indicator: People with disability report increased overall life satisfaction
Headline indicator: life satisfaction. VDAC advised that life satisfaction should be a headline indicator
Measures: Satisfied with life

Indicator: People with disability report increased resilience
Headline indicator: VDAC stated that this indicator puts the onus back on people with disability to cope with structural inequities they experience.

Indicator: People with disability experience less unfair treatment
Headline indicator: unfair treatment. VDAC wants to capture discrimination that may occur for many reasons, across diverse demographics.
Please note that because data availability is limited, constructing measures for this indicator that cover the monitoring period is not possible.
Measures: Experiences of discrimination

Indicator: People with disability experience less disability-related discrimination
Headline indicator: There is limited data for this indicator.

Indicator: People with disability experience less interpersonal violence
Headline indicator: interpersonal violence. VDAC advised that future development of data needs to consider that unsafe circumstances, environments and events are often not reported. Understanding what progress looks like needs to consider the complexity of reporting on progress in the context of underreporting.
Measures: Physical violence, and Sexual violence.

Indicator: People with disability report increased community safety
Headline indicator: Data not available

Indicator: People with disability report less bullying
Headline indicator: Data not available

Indicator: People with disability experience less abuse and neglect
Headline indicator: Data not available

Indicator: Increased educational engagement of students with disability
Headline indicator: educational engagement. VDAC advised that engagement was an important indicator and should be elevated to headline status.
Measures: Continuing education

Indicator: Increased education achievement of students with disability
Headline indicator: VDAC advised this is not a good indicator. People with disability face systemic barriers to completing education.
Overall, this is an outcome area that needs data development.

Indicator: Increased wellbeing of students with disability
Headline indicator: Data not available.
VDAC advised that wellbeing of students is important and should be prioritised in future data development.

Indicator: Increased engagement of Victorian children with disability in state-funded kindergarten
Headline indicator: Data not available.

Indicator: Increased wellbeing of Victorian children with disability at school entry
Headline indicator: Data not available.

Indicator: Increased employment
Headline indicator: employment. VDAC advised that each of the employment indicators was important. However, it was critical to understand where the gaps in the data are. For example, if data from closed workplaces cannot be included then it is important to acknowledge this limitation in our reporting.
Measures: Employment to population ratio; Labour force participation, and Unemployment

Indicator: Increased job quality
Headline indicator: job quality. VDAC advised that an overall job quality indicator should be elevated to headline indicator status.
Measures: Employment satisfaction

Indicator: Increased positive attitudes towards people with disability in the workplace
Headline indicator: Data not available.

Indicator: Increased job flexibility and job design adjustment
Headline indicator: Data not available. However, there is some overlap in the measures for inclusive employment we used for outcome 14 (see below).

Indicator: Improved financial stability and economic independence
Headline indicator: financial stability and economic independence. Cost of living was raised as a priority throughout consultation with VDAC.
Measures: Household income, Emergency funds, and Food insecurity

Indicator: Increased involvement in civic activities
Headline indicator: civic activities. No data was available for the other indicator for this objective, so this indicator was elevated to headline status.
Measures: Civic activities

Indicator: Increased leadership opportunities
Headline indicator: Data was not available

Indicator: People with disability report increased opportunities to pursue and achieve aspirations
Headline indicator: opportunities to pursue and achieve aspirations. This was the only indicator for this outcome. There is some data available, so it was elevated to headline status.
Measures: Inclusive employment potential, and Satisfaction with employment opportunities.

While we tried to keep data availability and headline indicator selection separate, there is a lack of available data for the ‘Education’ outcome. This meant we could not include measures on childhood experiences of the education system.

This is also an outcome area where VDAC had concerns about how measures could reinforce some of the disadvantage and stigma children with disability face in the education system.

We recommend working with VDAC to co-design some new indicators. This should include a thorough data audit is undertaken to help construct appropriate measures.

This further work will be undertaken in consultation with Victorian Government departments, including the development of any new measures and indicators.

Furthermore, our main source of data, the Household, Income and Labour Dynamics in Australian (HILDA) survey only covers people aged 15 years and over. This meant that many measures do not reflect the experiences of children.

There are a number of other outcomes frameworks in Australia that could be adapted for the state disability plan.

For example, the Australian Disability Strategy (ADS) has an outcomes framework. There is potential to see if ADS Indicators could be utilised.

VDAC members discussed how differences in people’s social circumstances and contexts can influence their experience of the indicators.

For example, people with disability living in rural areas are likely to experience greater barriers to accessing public transportation than people with disability living in metropolitan areas.

VDAC also emphasised that people with disability are not a homogenous group. The VDAC subcommittee had concerns with indicators and measures that did not consider intersecting identities, such as gender, age, location and socioeconomic status.

We have discussed this further and included this as a recommendation in the outcomes report.

The outcomes framework was developed in 2017.

VDAC discussed how the world has changed substantially since then (for example the impacts of the COVID-19 pandemic, and the rising cost of living).

Committee members were concerned that the indicators and measures did not adequately capture these changes.

We have attempted to capture this feedback in the development of cost of living measures such as ‘food insecurity’.

It also includes measures that may capture the impact of COVID-19 on flexible working such as ‘employment satisfaction’.

However, more work needs to be done in this domain.

Some groups of people with disability are routinely excluded from available survey data.

This includes people living in group homes and those experiencing homelessness.

These data sources also lack information on important lived experience. This includes people with disability working in environments that are not part of open employment (for example, Australian Disability Enterprises).

There is more detail in the outcomes report on the measures where this is an issue.