Prepared by:
Victorian Aboriginal Child Care Agency, University of Melbourne and Department of Families, Fairness and Housing.

Authorised and published by the Victorian Government, 1 Treasury Place, Melbourne.

© State of Victoria, Australia, Department of Families, Fairness and Housing, June 2021.

In this document, ‘Aboriginal’ refers to both Aboriginal and Torres Strait Islander people.‘Indigenous’ or ‘Koori/Koorie’ is retained when part of the title of a report, program or quotation.

ISBN 978-1-76096-425-2 (pdf/online/MS word) Available at www.vic.gov.au/aboriginal-family-violence-reform

Cover image designed by Trina Dalton-Oogjes.

Acknowledgement of Aboriginal people and communities in Victoria

Aboriginal people are acknowledged as Australia’s First Nations Peoples and the custodians of the land and waterways upon which we depend. We acknowledge Victoria’s Aboriginal communities and culture and pay respect to their Elders past and present.

Aboriginal culture is founded on a strong social, cultural and spiritual order that has sustained more than 60,000 years of existence. Victorian Aboriginal communities and peoples are culturally diverse, with rich and varied heritages and histories. Aboriginal cultural heritage and the ongoing contribution of Aboriginal people and communities to culture and society is acknowledged as a source of strength and pride that enriches the whole Victorian community. The long-standing leadership of Aboriginal communities and Elders in preventing and responding to family violence, and improving outcomes for Aboriginal people, children and families, is recognised. At the same time, the devastating impacts and accumulation of trauma across generations as a result of colonisation, genocide and the violent dispossession of land and children is also acknowledged.

The invaluable contributions of all those who have paved the way and fought for the rights of Aboriginal people, including the right to self-determination and the right to live free from violence, are acknowledged.

To ensure that Aboriginal people, children, families and communities thrive, the Victorian Government is committed to an enduring community-led response to end family violence against Aboriginal people, underpinned by self-determination. Self-determination requires government to value and respect Aboriginal knowledge, systems and expertise and to transfer authority, decision-making control and resources to Aboriginal people. This requires a significant cultural shift and a new way of working together. The Government acknowledges that this is the key to better outcomes for Aboriginal people and stronger, safer families and communities. Aboriginal self-determination is the foundation of Dhelk Dja - Safe Our Way: Strong Culture, Strong Peoples, Strong Families (‘Dhelk Dja’).

Other acknowledgements

Acknowledgement and thanks go to the Aboriginal service representatives and individuals who participated in consultations to identify Aboriginal-defined outcomes and indicators for the Dhelk Dja strategy.

Special thanks go to the Dhelk Dja Priority 5 Sub-Working Group members of the DhelkDja Partnership Forum for their considerable time and expertise in review of the DhelkDja Theory of Change and Monitoring, Evaluation and Accountability Plan.

Thanks go to the Dhelk Dja Partnership Forum for their leadership in contributing to the overall Aboriginal Health Wellbeing and Safety Evaluation Framework project.

Thanks are due also to the government representatives within Family Safety Victoria and the Aboriginal Strategy and Oversight Branch of DFFH for their commitment and hard work through the life of the project.

Thanks finally to all members of the Evaluation Framework Project team within VACCA and the University of Melbourne.

1. Introduction

Purpose of this document

This Monitoring, Evaluation and Accountability Plan (MEAP) has been prepared for the Dhelk Dja Partnership Forum to accompany Dhelk Dja: Safe Our Way – Strong Cultures, Strong Peoples, Strong Families, the Aboriginal 10-year family violence agreement for 2018-2028.

This document lays out a plan for how the Dhelk Dja Partnership Forum can monitor and evaluate its strategy throughout its life, including conducting an evaluation study.

The MEAP, including the Dhelk Dja theory of change and data plan, was endorsed and adopted by the Dhelk Dja Partnership Forum on 22 July 2020.

Dhelk Dja: Safe Our Way –Strong Culture, Strong Peoples, Strong Families

Dhelk Dja: Safe Our Way – Strong Culture, Strong Peoples, Strong Families (the Dhelk Dja Agreement) is the Aboriginal 10-year family violence agreement for 2018-2028 and is a community-led Aboriginal agreement to address family violence. It was written by the Dhelk Dja Partnership Forum to build on the last 10-year plan, Strong Culture, Strong Peoples, Strong Families: Towards a safer future for Indigenous families and communities (2008-2018).

The Dhelk Dja Agreement, is the key Aboriginal-led Victorian agreement that commits the signatories – Aboriginal communities, Aboriginal services and government – to work together and be accountable for ensuring that Aboriginal people, families and communities are stronger, safer, thriving and living free from family violence. It articulates the long-term partnership and directions required at a state wide, regional and local level to ensure that Aboriginal people, families and communities are violence free, and built upon the foundation of Aboriginal self-determination.

Aboriginal members of the Dhelk Dja Partnership Forum state that “Safe Our Way means the Victorian Aboriginal community will drive and deliver Dhelk Dja ‘our way’ with Aboriginal self-determination, our cultures, our approaches, our responsibility, our communities, our families.”

The MEAP recognises that Aboriginal people experience and are victims of family violence at disproportionate rates compared with non-Aboriginal people. This can be attributed to the devastating effects of colonisation which continue to be felt by the Aboriginal community. Unfortunately, the disproportionate rates of family violence are not reflected in a culturally appropriate, safe and effective service system for Aboriginal people. Aboriginal people continually struggle to access the services and resources that they need to escape family violence, or to escape the spiral of violence that people who use violence are ensnared in.

The fact that the service system does not always reflect the needs of the Aboriginal community in some cases may be attributed to systemic bias. The MEAP is committed to ensuring an equitable access to services and resources based upon the real needs of the Aboriginal community throughout Victoria. These services must be delivered in a culturally-strengthening, appropriate manner.

Purpose

Dhelk Dja is built upon the foundation of Aboriginal self-determination and articulates the long-term partnership and directions required at a statewide, regional and local level to ensure that Aboriginal people, families and communities are violence free from violence.

It recognises the urgent need to reduce the disproportionate impact of family violence on Aboriginal people, particularly women and children; ensuring there is a strengths-based and gender-informed approach for people who experience and use violence.

Dhelk Dja also recognises that family violence is not part of Aboriginal culture and that family violence against Aboriginal people is perpetrated by both non-Aboriginal and Aboriginal people. Dhelk Dja recognises the importance of whole of family and whole of community approaches that are inclusive across the diversity of Aboriginal people and communities. It values the strength, knowledge and rich diversity of Aboriginal people, families and communities. It encompasses Aboriginal people with a disability or mental health issues; LGBTIQA+ people; Elders and older people; children and young people; people in or exiting out-of-home care, prison or other institutions; people living in rural and regional areas; and families comprising Aboriginal and non-Aboriginal family members.

Definition of family violence

The Victorian Indigenous Family Violence Task Force (2003) defined family violence as:

The definition also recognises that the cycle of family violence brings people into contact with many different parts of the service system, and efforts to reduce violence and improve outcomes for Aboriginal people and children must work across family violence services; police, the justice system and the courts; housing and homelessness services; children and family services; child protection and out-of-home care; and health, mental health, and substance abuse.

Dhelk Dja recognises the need to respond to all forms of family violence experienced by Aboriginal people, children, families and communities.

Aboriginal Health, Wellbeing and Safety Evaluation Framework Project

Despite decades of health-related research and a myriad of health strategies and initiatives across Australia, evidence shows there has been limited progress in improving the health, wellbeing and safety outcomes of Aboriginal peoples.

The Department of Families, Fairness and Housing (the department) is committed to building a deeper, richer understanding of what is needed to improve outcomes for Aboriginal Victorians, who continue to negotiate the devastating effects of colonisation and past government policies and practices.

To help deliver on this commitment, the department commissioned a project to develop the Aboriginal health, wellbeing and safety evaluation framework (the framework project). The project objectives are to:

• Develop Aboriginal-defined indicators and measures of success in health, wellbeing and safety, including a focus on cultural determinants.
• Enable Aboriginal communities and services, through the governance mechanisms of the Aboriginal Strategic Governance Forum, and the Dhelk Dja Partnership Forum, to effectively monitor progress against the four key strategies seeking to address the health, wellbeing and safety needs of Aboriginal Victorians.

The four key strategies that are the focus of the Framework project are:

• Korin Korin Balit-Djak: Aboriginal health wellbeing and safety strategic plan 2017-2027
• WungurilwilGapgapduir: Aboriginal children and families agreement 2018
• DhelkDja: Safe Our Way - Strong Culture, Strong Peoples, Strong Families 2018-2028
• BalitMurrup: Aboriginal social and emotional wellbeing framework 2017-2027.

A key commitment of the department is to advance Aboriginal self-determination in evaluation and monitoring of Aboriginal health, wellbeing and safety. This includes adopting Aboriginal defined measures of progress and success that align with holistic understandings of health and wellbeing, where the social, emotional and cultural well-being of the whole community impacts on the physical and emotional wellbeing of an individual.

This in turn will ensure that resultant evaluations will generate evidence about what works in Aboriginal health, wellbeing and safety from an Aboriginal perspective, with findings supporting improved future policy and program development (Kelaher et al., 2018; Rootman et al., (eds) 2001). The Framework acknowledges Aboriginal expertise and cultural knowledge as a legitimate source of truth that is integral for providing evidence of Victorian Aboriginal health, wellbeing and safety.

Deliverables of the project included:

• A literature review on cultural determinants of health and wellbeing.
• A specific monitoring, evaluation and accountability (MEA) plans for each of the strategies.

The Victorian Aboriginal Child Care Agency (VACCA) and its partners at the University of Melbourne were awarded the contract in late 2018 to develop the framework and associated deliverables over the course of 2019.

Literature Review on cultural determinants of health, wellbeing and safety

Aboriginal health strategies and initiatives have largely taken the widely researched and broadly accepted social determinants approach to health and wellbeing (Brown 2013). However, social determinant measures and indicators, as they are understood by the World Health Organisation, were developed almost entirely in a European cultural context for use within dominant Western populations (Brown et al., 2006). We also know that social determinants present health as a goal of the individual rather than the community (Campbell et al., 2007:166), and this fails to acknowledge an Aboriginal perspective of health and wellbeing, where health is:

not just the physical well-being of an individual but also the social, emotional and cultural well-being of the whole Community in which each individual is able to achieve their full potential as a human being thereby bringing about the total well-being of their Community(NAHSWP, 1989).

The department has recognised that consideration of the cultural determinants is necessary as part of a more comprehensive approach that addresses structural and systemic factors, has a focus on prevention and is trauma informed in order to support Aboriginal people to have ‘healthy and independent relationships between families, communities, land, sea and spirit’ (NATSIHC, 2004). This has been termed a ‘cultural determinants approach’ to wellbeing, and unlike a social determinants approach, it promotes a strength-based perspective by focusing on ‘life­giving values from which individuals, families and communities can draw strength, resilience and empowerment (Markwick et al., 2014:2).

The University of Melbourne and VACCA teams prepared a review of literature to explore what is already known about cultural determinants of health for Aboriginal Australians. This review was a resource used to inform the thematic analysis of material out of the community consultations and to identify potential additional indicators and measures for the overall framework and MEA Plans.

2. What is a monitoring, evaluation and accountability (MEA) plan?

A monitoring, evaluation and accountability plan is an integral part of the cycle for managing the performance of projects, programs, organisations and other initiatives that seek to make change.

The Dhelk Dja Agreement is a statewide partnership initiative that seeks to prevent and respond to family violence impacting Aboriginal people in Victoria. In order to improve its work, the Dhelk Dja Partnership Forum needs to track whether the Dhelk Dja Agreement is achieving its intended outcomes (as specified in a theory or pathway of change), and if not, what actions and investments need to be adjusted and/or re-prioritised. Figure 1 below shows the relationship between the strategy and key monitoring and evaluation steps.

3. The theory of change underpinning the Dhelk Dja Agreement

What is a theory of change?

A theory of change (also known as a program logic or pathway of change), is a conceptual map of how the Dhelk Dja Agreement is thought to logically achieve short, medium and long-term outcomes, through its key actions and activities, that together will achieve its ultimate vision and intended impact. It includes assumptions about how the activities and processes will achieve the intended progressive outcomes.

The value of a theory of change is that it draws out the key outcomes, in the short, medium and long-term, that the Dhelk Dja Agreement is seeking to achieve. This in turn makes it easier to identify appropriate indicators for the Dhelk Dja Partnership Forum to measure whether those outcomes have been achieved. It is important to note that a theory of change is a dynamic tool: it should be reviewed periodically to ensure that it remains relevant over time. This is especially the case for the Dhelk Dja theory of change given the complex nature of the changes being sought and the context within which this is taking place. The Dhelk Dja 3 Year Action Plans allow the Dhelk Dja Partnership Forum to be responsive to these changes, and adapt and improve the theory of change over time.

This in turn means that the MEA Plan for the Dhelk Dja Agreement needs to be able to handle this complex setting, and promote learning, adaptation and improvement in the Agreement as it is implemented.

Dhelk Dja theory of change - narrative version

The Dhelk Dja Agreement offers a comprehensive and systemic approach to preventing and responding to family violence. It takes a whole of system view to advance Aboriginal self-determination and create service system reform. The theory of change itself is graphically represented in Figures 2, 3 and 4 on pages 15-17. Before this, what follows is a narrative-based version of the theory of change.

The vision for the Dhelk Dja Agreement is for:

In order to contribute to this vision, the Dhelk Dja Agreement will deliver a set of actions as outlined in the initial Dhelk Dja 3 Year Action Plan 2019­-2022. These actions align with the five Dhelk Dja Strategic Priorities:

• Aboriginal culture and leadership
• Aboriginal-led prevention
• self-determining family violence support and services
• system transformation
• Aboriginal-led and informed innovation, data and research.

If these actions are all delivered in line with the guiding principles, they will enable five interrelated outcome pillars to be achieved. As shown in Figure 2, achieving outcomes that embed Aboriginal culture and leadership and Aboriginal-led and informed innovation, data and research will enable achievement of outcomes for family violence organisations and for the family violence service system. In turn, achieving outcomes across family violence organisations and the wider service system will enable the positive outcomes for Aboriginal individuals, families and communities to be reached.

Ultimately, achieving outcomes across all five pillars will result in four important impacts:

• Aboriginal self-determination, voice and cultural ways are embedded across the family violence system
• Aboriginal people have access to high quality services based on Aboriginal-led best practice and evidence
• Aboriginal people, especially women and children, are safe and thriving
• Aboriginal people have healthy and respectful relationships across gender, backgrounds and generations.

Outlined in Appendix 3 are the five prioritised outcomes that the Dhelk Dja Partnership Forum expect to see in the short, medium and long-term as a result of delivering their 3-5 year actions.

DhelkDja theory of change - graphic version

The graphic version of the Dhelk Dja theory of change can be found below in Figures 2, 3 and 4.

Figure 2 provides a high-level (highly summarised) version of how the outcome pillars relate to each other, as well as to the 3-year actions and the impact/vision.

Figure 3 provides a detailed version.

Figure 4 documents the ‘levers' the theory of change is seeking to use, along with key assumptions, gaps in the existing theory of change, and the key stakeholders the theory of change is targeting. These key assumptions, gaps and stakeholders will need to be monitored to ensure that progress is being made towards the intended outcomes and the overall success of the strategy.

Clarification of terms

Where outcomes in the theory of change refer to the family violence service system this is intended to encompass all relevant services that prevent and respond to family violence related issues, including:

• specialist family violence services
• sexual assault services
• services for people who use violence
• housing, homelessness and crisis accommodation services
• legal services
• child protection
• alcohol and other drug services and mental health services – where relevant and to the extent possible.

Where outcomes refer to Aboriginal people or individuals, this is inclusive of Aboriginal women, men, children and young people, Elders and older people, LGBTIQA+ people, people in or exiting out-of-home care, prison or other institutions, people living in rural and regional areas, and families comprising Aboriginal and non-Aboriginal family members. This is consistent with the Dhelk Dja purpose and definition of family violence as described in the ‘Introduction’.

The Dhelk Dja 10-Year Agreement takes a gender informed approach to family violence prevention and response.

For a full glossary of terms please refer to the Dhelk Dja Agreement.

4. What is the purpose, scope and frame of this MEA Plan?

Purpose

The purpose of the Dhelk Dja MEA Plan is to

• equip the Dhelk Dja Partnership Forum to understand the logic underpinning the relationship between 3-year actions, intended outcomes and the overarching Dhelk Dja vision
• provide a mechanism for government and other parties responsible for delivering 3-year actions to be accountable to the Dhelk Dja Partnership Forum
• enable the Dhelk Dja Partnership Forum to monitor progress and make strategic decisions about investment and future actions.

Scope

Ongoing monitoring

The MEAP provides a guide for the Dhelk Dja Partnership Forum to track and evaluate progress against the Dhelk Dja Agreement. Monitoring is proposed to take place routinely, with annual reporting against actions and indicators. A sub-set of indicators will be suitable for reporting annually, while the remainder will be suitable for reporting on at the end of each 3-year Action Plan. This annual reporting requires a coordinated effort to collate data for a range of indicators from a range of sources, and prepare a report which supports the Partnership Forum to track annual change.

Evaluation

At the completion of the first and second 3-Year Action Plans, an independent evaluation is recommended. At the conclusion of the final 3-year Action Plan and lifespan of the Dhelk Dja Agreement (at Year 10) a final independent evaluation is recommended.

Recommended evaluation approach

A mixed-methods approach is recommended, in recognition of the fact that no single method of data collection is completely free of limitations and bias, and a combined approach can better manage this.

Resourcing available to undertake both the annual reporting against monitoring data, and the periodic evaluations will need to be further identified with the Dhelk Dja Partnership Forum. This MEAP provides a guide and recommended approach, however it may need to be adjusted once resourcing becomes clearer.

The evaluation audience

The primary audience for the monitoring and evaluation activity recommended in this plan is the Dhelk Dja Partnership Forum. This is the group who require the monitoring and evaluation information, are involved with the MEAP process, and will make decisions about the Agreement and its 3-year Action Plans.

Performance expectations

Currently this plan provides a set of indicators and corresponding measures but does not advise on performance standards. While the theory of change and corresponding indicators give the Dhelk Dja Partnership Forum an understanding of what change is expected to occur and how, it does not specify how much of this change would be needed to consider the strategy successful.

It can be useful to decide on performance standards for key outcomes so the Partnership Forum and the appointed evaluators can make judgements about how well the strategy is going. Performance standards can take the form of targets or rubrics, or a combination. Rubrics are a general way of transparently defining what quality, value or achievement would look like. Rubrics are a good way to ensure that community cultural and organisational values are incorporated into evaluation and can help build a shared consensus about what is valuable.

It is recommended that the Partnership Forum consider seeking technical assistance to develop some targets and rubrics at the end of the first and subsequent 3-year Action Plans and end of strategy outcomes as an integral part of the monitoring process.

Key evaluation criteria and key evaluation questions

In order for the Dhelk Dja Partnership Forum to make a judgement about progress and success of the strategy, the MEAP needs to be framed by key evaluation criteria and questions that match the intended purpose of the strategy and meet audience needs. The following criteria and questions, are proposed. The recommended timing for these questions is specified in Table 1 Dhelk Dja MEA data plan, in Section 6.

Key framing criteria and corresponding key questions

Key Criteria

Criterion 1: Effectiveness and impact – To what extent were the 3 year actions implemented as intended

A measure of the extent to which the Dhelk Dja Agreement attains its outputs and outcomes; and the positive and negative changes produced by the strategy, directly or indirectly, intended or unintended

Key questions

• To what extent were the 3 year actions implemented as intended?
• To what extent were the intended (short, medium and long-term) outcomes achieved?
• What were the positive, negative and unexpected impacts of the strategy for the intended beneficiaries – ACCOs, mainstream CSOs, government, Aboriginal communities?

Key criteria

Criterion 2: Contribution to change

An assessment of the extent to which the Dhelk Dja Agreement contributed to the observed changes

Key questions

Drawing on the analysis of the effectiveness criterion:

• (1) To what extent did the Dhelk Dja 3-Year Action Plans and the Dhelk Dja Agreement as a whole, contribute to the outcomes achieved?
• (2) What other factors contributed to the outcomes achieved (external political, social, financial and other factors)?

Key criteria

Criterion 3: Relevance and strategy

The extent to which the Dhelk Dja Agreement is suited or aligned with the priorities of the Dhelk Dja Partnership Forum; how relevant the strategy outcomes are to policy context and needs; an assessment of the underpinning theory of change

• (3) To what extent are the identified outcomes of the Dhelk Dja Agreement still relevant to the needs of Aboriginal Victorians in relation to family violence prevention and response?
• (4) How should future action plans or strategies respond to this change in priorities/desired outcomes?
• (5) Where the 3 year actions were implemented as intended, to what extent did the key assumptions of the theory of change hold true and why? What gaps were there in the key assumptions?
• (6) Are there other system levers that have been overlooked in the assumptions underpinning the Dhelk Dja theory of change?
• (7)What actions could be prioritised to improve the effectiveness of the Agreement in achieving the desired outcomes?

Key criteria

Criterion 4: Governance and management

How effectively the Dhelk Dja Partnership Forum manage the Dhelk Dja Agreement

• (8) How effectively did the Dhelk Dja Partnership Forum work together to govern and deliver the Dhelk Dja Agreement? What were the enablers and barriers to governance and delivery?
• (9) To what extent was the Dhelk Dja Partnership Forum able to adjust and adapt to the emerging social and policy context in which it works?
• (10)To what extent, and how effectively, was monitoring and evaluation activity undertaken as intended?

5. Dhelk Dja outcome and indicator set

The following section contains the set of outcomes and corresponding indicators that have been selected against the short, medium and long-term outcomes as specified in the Dhelk Dja Theory of Change.

(The full set of outcomes and corresponding indicators, proposed measures and data sources is contained in Appendix 4).

All stakeholders to the Partnership – government, ACCOs and mainstream CSO’s – will be expected to report against those indicators that apply to them.

Outcomes and corresponding indicators for the Dhelk Dja Agreement

Aboriginal individuals, families and communities outcomes

Short term outcomes and indicators

• Aboriginal people have greater access to family violence prevention and early intervention initiatives (P2).
  • Increased and long-term funding is allocated to Aboriginal-led prevention initiatives (P2).
  • Increased and long-term funding is allocated to healthy relationship education initiatives for children and young people (P2).
  • Prevention activities reach more Aboriginal people across more settings.
• Aboriginal people, especially women and children, have greater access to culturally responsive, holistic family violence services (P3).
  • Increased and long-term funding is allocated to culturally responsive, holistic family violence services, including therapeutic support and healing services (P3).
  • Funding is prioritised to Aboriginal-led family violence services (P3) (3.3) [implementation of the DFFH Prioritised Funding Policy].

Medium term outcomes and indicators

• Aboriginal people are benefiting from Aboriginal-led initiatives that strengthen connection to culture, country and community (P2).
  • Increased and long-term funding for programs that embed connection to culture, country and community (P2).
  • Increase in Aboriginal people who are connected to culture, country and community.
• Aboriginal people are engaged in and benefiting from Aboriginal-led, trauma informed family violence prevention and early intervention (P2).
  • Increase in Aboriginal children and young people participating in healthy relationship education initiatives (P2) (5.5).
  • Increase in Aboriginal adults participating in prevention initiatives.
  • Aboriginal people make healthy relationship decisions and choices (P2).
  • Aboriginal people understand their rights and do not accept or tolerate controlling or violent behaviours (P2).
  • Increase in Aboriginal children and young people participating in healthy relationship education initiatives (P2) (5.5) - repeat indicator.
• Aboriginal people are safe to express their cultural identify without fear of racism, violence and discrimination (P2).
  • Increase in people willing to identify as Aboriginal when receiving family violence services (P2).
  • Decrease in incidence of racism and discrimination when receiving family violence services (P2).
• Aboriginal people who use family violence are held accountable for and change their behaviour through engagement in safe, culture and trauma informed services (P3).
  • Increase in participation rates for Aboriginal people who use violence in behaviour change programs.
  • Increase in behaviour change programs that are culture and trauma informed.
  • Increase rate of compliance with statutory orders by Aboriginal people.
• Aboriginal victim survivors, especially women and children, receive culturally responsive services across the family violence system that keep them safe and help them heal (P3).
  • Reduction in Aboriginal children and young people experiencing family violence.
  • Reduction in Aboriginal people, especially women, experiencing family violence.
  • Increased safety for Aboriginal people, especially women and children, who have experienced family violence.
  • Reduction in re-presentation of Aboriginal people, especially women and children, in the family violence system.
  • Increase in Aboriginal clients reporting that ACCO and mainstream family violence services are culturally responsive.

Long Term Outcomes andIndicators

• Aboriginal people have improved social emotional and cultural wellbeing and safety (P2).
  • Improved physical, social, emotional and cultural wellbeing across the diversity of Aboriginal people (5.1).
  • Individuals and families are empowered by the system to advocate and supported in making positive life choices (5.6).
• Aboriginal people at risk of, or experiencing family violence, especially women and children, are safe and rebuild their lives (P3).
  • Increase in Aboriginal-led family violence services with the specialisation and expertise to increase the safety, healing and wellbeing of victims, children, young people and families.

Family violence system outcomes

Short term outcomes and indicators

• Culturally safe case management, risk assessment and information sharing practices are in place across all service providers.
  • Increase number of mainstream family violence service providers instituting cultural safety/competency training as a core requirement for workers (P4).
  • Increase number of mainstream family violence service providers with organisational cultural safety plans in place.
  • Tools and infrastructure that support culturally safe case management, risk assessment and information sharing (MARAM) are implemented across FV services (P4).
• Aboriginal client voice is informing policies and practices to improve service delivery across the system.
  • Increase in family violence services for Aboriginal people using client feedback mechanisms.
  • Increase in consultation and co-design of policies and programs with Aboriginal people.
  • Aboriginal-led resources are guiding and informing family violence primary prevention.
• Aboriginal-led tools and resources that support culturally responsive primary prevention are in place.
• Aboriginal people are represented in all aspects of family violence policy, implementation, monitoring and evaluation.
  • Funding is prioritised for education and prevention initiatives to promote healthy and respectful relationships across genders, the life-span and generations.
  • Increased and long-term funding is allocated to Aboriginal-led prevention initiatives (P2).
  • Increased and long-term funding is allocated to healthy relationship education initiatives for children and young people (P2).

Medium term outcomes and indicators

• The Aboriginal family violence workforce is expanded, skilled and recognised for its practice wisdom.
  • Processes for recognising the equivalence of work experience and qualificationsare in place (P3).
• Increase in the number of Aboriginal specialist family violence workers employed.
  • Increased retention rates for Aboriginal family violence workers (P3).
  • Aboriginal workers have the necessary skills to assess and respond to family violence risk.
• Strong partnerships and collaboration exist between Aboriginal and non-Aboriginal family violence services to share expertise and co-design culturally safe services.
  • MOUs and other formal partnership agreements exist between ACCOs and mainstream organisations.
• Mainstream organisation workers have the skills and confidence to deliver culturally responsive family violence services.
  • Increase number of mainstream family violence organisations instituting cultural safety/competency training as a core requirement for workers (P4).
  • Increase in family violence workers reporting feeling confident to deliver culturally responsive services for Aboriginal people.
• Increase in Aboriginal clients reporting that ACCO and mainstream family violence services are culturally responsive.

Long term outcomes and indicators

• The service system for Aboriginal people atat risk of, or experiencing family violence is integrated, holistic and culturally safe.
  • All Aboriginal people at risk of, or experiencing family violence, receive a service that meets their needs holistically.
  • Increase in coordinating mechanisms for sharing information, MOUs or other formal agreements to foster integration.
• Aboriginal organisational infrastructure and workforce development to support expanded family violence service delivery is in place.
  • Aboriginal workers have increased access to education, training and skills development and career pathways.

Family violence organisation outcomes

• Aboriginal organisational infrastructure and workforce development to support expanded family violence service delivery is in place.
  • Aboriginal Family Violence Industry Strategy is developed and implemented.
  • Aboriginal organisations providing family violence services receive funding to support workforce development.
• Aboriginal organisations receive prioritised funding for delivery of holistic family violence services.
  • Funding is prioritised to Aboriginal-led family violence services (P3) (3.3) [implementation of the DFFH Prioritised Funding Policy] - repeat indicator.
• Aboriginal organisations are funded to undertake monitoring and evaluation and demonstrate family violence programs that work.
  • Increase in funding to Aboriginal organisations providing family violence services to support monitoring and evaluation and showcase best practice.

Medium term outcomes andindicators

• Organisations are accountable to Aboriginal communities for ensuring their services are culturally responsive.
  • Increase in organisations publicly reporting client feedback on cultural responsiveness.
  • Increase number of family violence service providers with Aboriginal cultural safety plans in place.
  • Increase number of family violence service providers instituting cultural safety/competency training as a core requirement for workers (P4).
• Aboriginal organisations can sustainably deliver culturally responsive family violence services, including therapeutic support and healing services.
  • Increase in ACCO capacity to meet family violence service demand.
  • Increased retention rates for Aboriginal family violence workers (P3) [repeat indicator].
  • Increase in length of funding term for family violence services and programs that work for Aboriginal people (3.2).
  • All ACCOs providing family violence services meet and maintain accreditation standards.

Long term outcomes and indicators

• Sustainable Aboriginal organisations are the primary providers of family violence and healing services for Aboriginal people.
  • New and existing funding is prioritised to Aboriginal-led family violence services (P3). [implementation of the DFFH Prioritised Funding Policy] (3.3).
  • Increase in length of funding term for family violence services and programs that work for Aboriginal people (3.2).
  • All ACCOs providing family violence services meet and maintain accreditation standards.
  • Increase in ACCOs achieving intended client outcomes.
  • Increase in ACCO capacity to meet service demand.
  • Increased retention rates for Aboriginal family violence workers (P3).

Aboriginal-led and informed innovation, data and research outcomes

Short term outcomes and indicators

• Aboriginal-defined measures inform data collection, reporting and planning by Dhelk Dja Partnership Forum members
  • The Aboriginal Health, Wellbeing and Safety Evaluation Framework is complete and implemented
  • The Dhelk Dja Monitoring, Evaluation and Accountability Plan is complete and implemented
• Aboriginal-led initiatives are building the evidence base for family violence prevention and intervention
  • The DFFH Evidence Menu includes exemplars of Aboriginal success in family violence prevention and response
  • Annual Aboriginal-led prevention showcase instituted
• Aboriginal-led ethical protocols are improving the design and conduct of research and evaluation in Aboriginal family violence
  • Aboriginal communities have led the development of ethical protocols to underpin culturally safe research and evaluation processes and methodologies
  • Increased uptake of ethical protocols for Aboriginal family violence research and evaluation (government, ACCOs, researchers)
• Workforce development and capacity building for Aboriginal-controlled structures to lead research and evaluation in family violence is in place
  • Increased government investment in evaluation capacity building for Aboriginal organisations and community groups

Medium term outcomes and indicators

• Aboriginal communities have control over collection, ownership and application of data
  • Formal agreements are in place and guiding Aboriginal data governance and Intellectual Property
• Best practice Aboriginal approaches are enabling better decision making and more effective resource allocation by the Dhelk Dja Partnership Forum
  • Dhelk Dja Partnership Forum decision-making and resource allocation is transparently informed by best practice and evidence
• Aboriginal community-controlled structures and organisations are well-resourced to monitor, evaluate and lead evidence-based practice in family violence responses

Long term outcomes and indicators

• Aboriginal-led capture and sharing of data and evidence is driving family violence system reform, policy, practice and innovation for Aboriginal people

Aboriginal culture and leadership outcomes

Short term outcomes and indicators

• Aboriginal-led governance and leadership is evident across the breadth of the family violence service system.
  • Increased participation from ACCOs and community members on family violence reform governance groups (P1).
  • Increased government funding and investment in Aboriginal controlled structures, governance and leadership across family violence reform.
  • Increase the diversity of Aboriginal representation to reflect diversity of the community and local needs (P1).
• Local Aboriginal communities have the capacity and capability to participate in governance and leadership.
  • Increased capacity of Dhelk Dja Action Groups to participate in governance across the family violence system.
• Aboriginal current and emerging leaders are supported to lead and influence decision-making and system design.
  • Increase in opportunities for young leaders, including those with lived experience, to build their leadership skills (P1).
  • Increase in opportunities for young Aboriginal people to be involved in governance and decision-making.

Medium term outcomes and indicators

• Cultural determinants of health, wellbeing and safety are embedded in policy and program planning, delivery and monitoring.
  • Increase in family violence programs, policies and services that incorporate cultural determinants of health, wellbeing and safety (3.8).
• Aboriginal-led service and program design is reflectedacross the family violence system.
  • Increased participation of Aboriginal community members (including practitioners and people with lived experience) in service and program design.
  • All family violence services and programs respond effectively to Aboriginal community needs.
• Aboriginal leadership and governance is effectively influencing investment and resourcing, policy and system wide improvement.
  • Government investment matches community and ACCO needs and expectations.
  • Government processes around decision-making, investment and policy are accountable and transparent to Aboriginal community.

Long term outcomes and indicators

• Aboriginal communities and controlled structures are driving family violence solutions and innovations for their communities
  • Transfer of power/decision-making to Aboriginal communities and organisations (refer to self-determination principles in VAAF)
• Across the family violence service system, Aboriginal culture is respected and understood to be integral to health, wellbeing and safety

6. MEAP data plan

The following section sets out the proposed MEAP data plan, including key evaluation questions, sub-questions and corresponding data collection methods.

KEQ 2: To what extent did the strategy contribute to the observed impact?

Key questions

• (1)To what extent were the 3 year actions implemented as intended?
• (2) To what extent were the intended (short, medium and long-term) outcomes achieved?
• (3) What were the positive, negative and unexpected impacts of the strategy for the intended beneficiaries– ACCOs, mainstream CSOs, government, Aboriginal communities?

Method for routine monitoring (ongoing)

• MEAP indicators/ measures collated annually (selection of indicators only).
• MEAP indicators collated every 3 years (all indicators).
• Progress against 3-year Action Plan reported annually.

Method for evaluative judgements (annually)

• Partnership Forum reflection workshop to review 3-year Action Plan progress report.

Method for evaluation study (at end each 3-Year Action Plan and/or end of entire 10-year agreement)

• Quantitative (trend) analysis of collated annual monitoring data for indicators (as against agreed performance standards/targets/ rubrics).
• Assessment of progress against 3-year Action Plan (as against agreed performance standards/ targets/rubrics).
• Focus groups with key stakeholders (Dhelk Dja Action Groups, Chairpersons, ACCOs, mainstream service providers, Aboriginal communities).
• Partnership Forum reflection workshop at end of each 3-year Action Plan for ‘sense making’.
• Partnership Forum reflection workshop at end of 10-year agreement for summative ‘sense making’.

KEQ 3a: How relevant is the strategy to policy context and needs?

Key questions

• (4) To what extent did the Dhelk Dja three-year action plans and the Dhelk Dja Agreement as a whole, contribute to the outcomes achieved?
• (5) What other factors contributed to the outcomes achieved (external political, social, financial and other factors)?

Method for evaluative judgements (annually)

• Analysis for Partnership Forum of known contextual changes and emergent developments (political, social, financial or policy changes).
• Partnership Forum reflection workshop.

Method for evaluation study (at end each 3-Year Action Plan and/or end of entire 10-year agreement)

• Focus groups with key stakeholders (Dhelk Dja Action Groups, Chairpersons, ACCOs, mainstream service providers, Aboriginal communities, government departments).
• Partnership Forum reflection workshop at end each 3-year Action Plan for ‘sense making’.
• Partnership Forum reflection workshop at end of 10-year agreement for summative ‘sense making’.

KEQ 3b: How robust is the underpinning theory of change and its assumptions about how change will happen?

7. Next steps

Proceed with monitoring where data exists

This Dhelk Dja MEAP identifies indicators that already have existing data sources and corresponding measures, and those for which there is currently no available data.

The Dhelk Dja Partnership Forum is advised to commence monitoring pragmatically where data is available as soon as possible.

Determine what new data should be collected and how

For those indicators without an existing data source, the Dhelk Dja Partnership Forum needs to determine that these are important enough to warrant collecting new pieces of data.

A data development plan will need to be created.

Possible solutions include building in an additional data item/ measure to existing information management systems, devising a new data instrument, or periodic special data collection queries.

This has implications not only for Government, but also for ACCOs and mainstream organisations in terms of resourcing and supporting additional data collection and reporting responsibility. Further implementation planning by the department is recommended as a next phase of the Aboriginal Research and Evaluation Framework project.

Planning for data analysis and sense making

The Dhelk Dja Partnership Forum will need to decide how best to resource and undertake data collation, analysis and sense-making for ongoing monitoring and periodic evaluation.

As already noted, the resourcing available to undertake both the annual reporting against monitoring data, and the periodic evaluations is, as yet, unspecified. Normally, the resources, including human and financial, are determined before planning the approach and methods, along with available time.

This MEAP may therefore need to be adjusted once resourcing becomes clearer.

Appendices

Appendix 1: Method for development of Dhelk Dja MEAP

How was the Dhelk Dja theory of change developed?

This theory of change was developed iteratively in the following way:

• desktop development of a draft theory of change by the project team (DFFH, FSV and VACCA), built upon the Dhelk Dja Agreement and 3-year Action Plan
• testing of the draft theory of change with the Dhelk Dja Priority 5 Sub-Working Group
• further testing one-on-one with government and non-government members of the Dhelk Dja Partnership Forum
• review of the revised theory of change and corresponding indicators (see Section 4 below) by the Dhelk Dja Priority 5 Sub-Working Group for final substantive changes and recommendation for endorsement by the Dhelk Dja Partnership Forum
• delivery of the final draft MEAP to the Dhelk Dja Partnership Forum for final review and endorsement.

How were indicators generated?

The first step in indicator generation was to consult widely. As part of the Aboriginal Research and Evaluation Framework project, VACCA conducted state-wide consultations with the Dhelk Dja Partnership Forum, ACCOs and local advisory groups involved in Aboriginal family violence prevention, response and recovery across Victoria. Participants identified outcomes and possible indicators of ‘success’ or progress from an Aboriginal perspective for the Dhelk Dja Strategic Priority areas.

Participants were also invited to identify outcomes and possible indicators for the six themes identified by the University of Melbourne as common to all four DFFH health, wellbeing and safety agreements. The University of Melbourne team also drew on potential outcomes and indicators from authoritative Aboriginal-led research sources. A significant number of potential indicators was collected and mapped by the University of Melbourne team against the Dhelk Dja Agreement. The full ‘bank’ of outcomes and indicators has been provided to the Dhelk Dja Partnership Forum for its information separate to this document.

Please refer to Appendix 2 for further information about the consultation process.

How were indicators selected for inclusion?

Once the Dhelk Dja theory of change was developed, the next step was to choose from the bank of indicators that were generated and gathered for best fit with the set of outcomes. There were three key indicator sources to draw on from the original consultation process conducted by VACCA across Victoria:

• the large pool of indicators developed specifically for Dhelk Dja (from consultations and literature)
• the smaller pool of indicators that were developed for the common themes that cut across all four health, wellbeing and safety strategy agreements (see next section)
• indicators developed for the Victorian Government Family Violence Outcomes Framework and the Victorian Aboriginal Affairs Framework (to ensure alignment where possible).

Selection was based on application of a set of agreed criteria, including:

• 'control' versus 'influence' versus 'context'. This rates whether the Dhelk Dja Partnership Forum can be expected to have significant leverage over an outcome indicator (control) or only partial leverage (influence), or whether the outcome indicator is a matter of context beyond the Forum’s reach but helpful in understanding the outcome (context). Control and influence are more useful to prioritise over contextual indicators because they offer some degree of leverage.
• 'relevance' to outcome being measured (H= high; M=medium);
• 'availability of data' (data is currently collected and readily available) (Y=yes; N= No)
• 'burden' of data collection (H=high; Medium=medium; L=low;
• 'costs' associated with new data collection and analysis (H=high; Medium=medium; L=low).

FSV and the VACCA evaluation team undertook the first round of indicator selection. The Dhelk Dja Strategic Priority Five Sub-Working Group then reviewed the indicators along with the draft MEAP.

Appendix 2: Consultation process

Overall, the consultation process involved 241 people, of which 89% were Aboriginal (Table 1). Five key informant interviews were also conducted.

Table 4: Consultation sites and number of participants

Consultation notes and audio transcriptions were uploaded to the software NVivo, where material was coded by The University of Melbourne. Themes were largely deducted from the four Aboriginal health, wellbeing and safety strategies, namely:

• governance and leadership
• cultural safety
• system and service reform
• workforce
• data and knowledge
• individuals, families and communities
• evaluation capacity building
• distinctive features of ACCOs
• self-determination
• cultural determinants of health.

After all material had been coded, indicators of ‘success’ were developed. Further indicators were developed and/or sourced from the following: The University of Melbourne’s literature review on the cultural determinants of Aboriginal and Torres Strait Islander health, wellbeing and safety; previous evaluation frameworks to improve Aboriginal and Torres Strait Islander health by Kelaher et al. (2018), referred to in the overall evaluation framework as the ‘Lowitja Evaluation Framework’; and those for Aboriginal and Torres Strait Islander health and social programs by Williams (2018), referred to in the overall evaluation framework as the ‘Ngaa-bi-nya Evaluation Framework.

Appendix 3: The five prioritised outcomes of the Dhelk Dja Theory of Change

Outlined below is a narrative version of the five prioritised outcomes that the Dhelk Dja Partnership Forum expect to see in the short, medium and long term as a result of delivering their 3-5 year actions. There are a number of assumptions, gaps and key stakeholders identified for each strategic priority area in the theory of change presented in Figure 4 of the '3. The theory of change underpinning the Dhelk Dja Agreement' section of this plan.

These key assumptions, gaps and stakeholders will need to be monitored to ensure that progress is being made towards the intended outcomes and the overall success of the strategy.

Aboriginal culture and leadership outcomes

If actions under the five Strategic Priorities are delivered in line with the principles of self-determination, community-led, self-management and leadership, then the following outcomes will be achieved in the short term:

• Aboriginal-led governance and leadership is evident across the breadth of the family violence service system
• current and emerging Aboriginal leaders are supported to lead and influence decision-making and system design
• local Aboriginal communities have the capacity and capability to participate in governance and leadership.

Once achieved and with sustained efforts, this will lead to the following medium-term outcomes:

• cultural determinants of health, wellbeing and safety are embedded in policy and program planning, delivery and monitoring
• Aboriginal-led service and program design is reflected across the family violence system
• Aboriginal leadership and governance is effectively influencing investment and resourcing, policy and system wide improvement.

Building upon the medium-term outcomes two important long-term outcomes will result by the ten year mark:

• Aboriginal communities and controlled structures are driving family violence solutions and innovations for their communities
• Across the family violence service system Aboriginal culture is respected and understood to be integral to health, wellbeing and safety
• Aboriginal-led and informed innovation, data and research outcomes.

If actions under the five Strategic Priorities are delivered in line with the guiding principles, then the following outcomes will be achieved in the short-term:

• Aboriginal-led ethical guidelines are improving the design and conduct of research and evaluation in Aboriginal family violence Aboriginal-led initiatives are building on the evidence base for family violence prevention and intervention
• workforce development and capacity building for Aboriginal-controlled structures to lead research and evaluation in family violence
• Aboriginal-defined measures inform data collection, reporting and planning by Dhelk Dja Partnership Forum members.

If these short-term outcomes are achieved, then this will lead to the following outcomes in the medium term:

• Aboriginal community-controlled structures and organisations are well-resourced to monitor, evaluate and lead evidence-based practice in family violence responses
• best practice Aboriginal approaches are enabling better decision-making and more effective resource allocation by the Dhelk Dja Partnership Forum
• Aboriginal communities have control over collection, ownership and application of data.

Achieving the short and medium-term outcomes in this pillar will pave the way for reaching this long-term outcome by the ten-year mark:

• Aboriginal-led collection and sharing of data and evidence is driving family violence system reform, policy, practice and innovation

Family violence organisation outcomes

If actions under the five Strategic Priorities are delivered in line with the guiding principles then the following outcomes will be achieved in the short-term:

• Aboriginal organisational infrastructure and workforce development to support expanded family violence service delivery is in place
• Aboriginal organisations receive prioritised funding for delivery of holistic family violence services
• Aboriginal organisations are funded to undertake monitoring and evaluation and demonstrate family violence programs that work.

Once these outcomes are achieved, then this will lead to the following outcomes in the medium-term:

• organisations are accountable to Aboriginal communities for ensuring their services are culturally safe
• Aboriginal organisations can sustainably deliver culturally informed family violence services, including therapeutic support and healing services.

The achievement of these medium-term outcomes will result in one long-term outcome by the ten-year mark:

• Sustainable Aboriginal organisations are the primary providers of family violence and healing services for Aboriginal people.

Family violence system outcomes

If actions under the five Strategic Priorities are delivered in line with the guiding principles then the following outcomes will be achieved in the short-term:

• culturally safe case management, risk assessment and information sharing practices are in place across all family violence service providers
• Aboriginal client voice is informing policies and practices to improve service delivery across the system
• Aboriginal-led resources are guiding and informing family violence primary prevention
• Aboriginal people are represented in all aspects of family violence policy, implementation, monitoring and evaluation
• funding is prioritised for education and prevention initiatives to promote healthy respectful relationships across genders, the lifespan and generations.

If these short-term outcomes are achieved, then it will lead to the following outcomes in the medium-term:

• the Aboriginal family violence workforce is expanded, skilled and recognised for its practice wisdom
• strong partnerships and collaboration exist between Aboriginal and non-Aboriginal family violence services to share expertise and codesign culturally safe services
• mainstream organisation workers have the skills and confidence to deliver culturally responsive family violence services.

Once these medium-term outcomes are realised, then the long-term outcome that will follow by the ten-year mark is:

• the service system for Aboriginal people, at risk of, or who are experiencing family violence is integrated, holistic and culturally safe.

Aboriginal individuals, families and communities outcomes

If actions under the five Strategic Priorities are delivered in line with the guiding principles, then the following outcomes will be achieved in the short-term:

• Aboriginal people are championing everyone’s right to safety and wellbeing
• Aboriginal people, especially women and children, have greater access to culturally responsive holistic family violence services
• Aboriginal people have greater access to family violence prevention and early intervention initiatives.

If these short-term outcomes are achieved, then this will lead to the following medium-term outcomes:

• Aboriginal people are benefiting from Aboriginal-led initiatives that facilitate greater connection to culture, country and community
• Aboriginal people are engaged in, and benefiting from Aboriginal-led, trauma informed family violence prevention and early intervention
• Aboriginal people make healthy relationship decisions and choices
• Aboriginal people are safe to express their cultural identity, without fear of racism, violence and discrimination
• Aboriginal people who use family violence are held accountable for and change their behaviour through engagement in safe, culture and trauma informed services
• Aboriginal victim survivors, especially women and children, receive safe and culturally responsive services across the family violence system that keep them safe and help them heal.

Long-term outcomes that build on these medium-term outcomes will be in place by the ten-year mark, including:

• Aboriginal people have improved social, emotional and cultural wellbeing and safety
• Aboriginal people at risk of, or experiencing family violence, especially women and children, are safe and rebuild their lives

Ultimately, when the long-term outcomes are achieved successfully across the five outcome pillars, the intended impacts and vision will be attained.

Glossary of terms

Monitoring

Generally used within evaluation to describe an ongoing or routine process of collecting agreed data. Monitoring data feeds into evaluation.

Evaluation

Generally used to describe a systematic process of gathering and analysing data in order to make evidence-based judgements about the merit or worth of the project/program/ strategy/policy or other initiative against agreed specific criteria or value.

Process evaluation

Process evaluation is concerned with evidence of activity, and the quality of implementation. The questions in a process evaluation focus on how, and Process evaluation is concerned with evidence of activity, and the quality of implementation.

Impact evaluation

An impact evaluation is an assessment of the actual results of a change. The term implies a broad evaluation that considers unintended consequences and contribution/attribution.

Summative evaluation

Summative evaluation is one that takes place at the end of the evaluation cycle. It is a type of evaluation that judges the worth of the task by the end of program (or strategy) activities. The main focus of the summative evaluation is based on the outcomes.

References

Brown, A. D., Morrissey, M. J., and Sherwood, J. M. (2006). Uncovering the determinants of cardiovascular disease among Indigenous people. Ethnicity and Health, 11(2), 191-210. doi:10.1080/13557850500485485

Brown, N. (2013). Culture is an important determinant of health: Professor Ngiare Brown at NACCHO Summit Retrieved from https://blogs.crikey.com.au/croakey/2013/08/20/culture-is­an-important-determinant-of-health-professor-ngiare-brown-at-naccho-summit/

Kelaher, M., Luke, J., Ferdinand, A., Chamravi, D., Ewen, S., and Paradies, Y. (2018). Evaluation Framework to improve Aboriginal and Torres Strait Islander Health

Markwick, A., Ansari, Z., Sullivan, M., Parsons, L., and McNeil, J. (2014). Inequalities in the social determinants of health of Aboriginal and Torres Strait Islander People: a cross-sectional population-based study in the Australian state of Victoria. International Journal for Equity in Health, 13

National Aboriginal and Torres Strait Islander Health Council (NATSIHC). (2003). National Strategic Framework for Aboriginal and Torres Strait Islander Health. Retrieved from Canberra, ACT: http://www.health.gov.au/internet/main/publishing.nsf/Content/health­oatsih-pubs-nsfatsih-context

National Aboriginal Health Strategic Working Party (NAHSWP). (1980). In: A National Aboriginal Health Strategy
Rootman, L., Goodstadt, M., Hyndman, B., McQueen, D., Potvin, L., Springett, J., and Zoglio,
E. (Eds.). (2001).

Evaluation in Health Promotion: Principles and Perspectives (Vol. 92). Copenhagen: WHO Europe
Williams, M. (2018 ). Ngaa-bi-nya Aboriginal and Torres Strait Islander program evaluation framework Evaluation Journal of Australasia, 18(1), 6-20