Establishment and governance

To what extent have establishment planning and operationalisation activities enabled implementation of The Orange Door foundational service model?

In evaluating the contribution that establishment have made to early implementation of The Orange Door model, we have examined a number of factors including: establishment processes, activities and tools, timeframe for foundational work, recruitment and on-boarding of staff, access to guidelines and policy, physical infrastructure and partnership formation.

Establishment processes, activities and tools

Extensive foundational work within a compressed timeframe

As a landmark reform representing significant transformational change in service delivery, an extensive array of establishment activities and processes were required to set-up and operationalise The Orange Door. The Orange Door model represents fundamental changes to ways of working, practice, culture and service delivery, as well as new organisational and governance structures, requiring extensive design, coordination, planning and support to enable roll-out. The government’s initial goal to open in five areas at the end of 2017 was ambitious, and created challenges and urgency for FSV – as a newly formed agency[1] – which ultimately led to delays in rollout.

Overall, FSV has been responsible for ‘establishing The Orange Door and associated governance mechanisms, as well as designing the service model and overseeing statewide delivery.’[2] This was to ensure ‘statewide fidelity and quality in the model, and to ensure The Orange Door and local services were supported to come together and work effectively.’[3] The amount of work to set-up and prepare The Orange Door has been unprecedented, with FSV managing a range of establishment activities, including:

• Development of statewide design, policy, frameworks and guidance to support integrated practice (for example Support and Safety Hubs Statewide Specifications, Interim Integrated Practice Framework and Operational Guidelines), as well as pre-requisite check-list for service commencement.
• Collaboration with local governance and stakeholders to understand the range of local stakeholders’ area specific needs, and to test elements from the statewide design in relation to the local context[4].
• Recruitment of Hub Managers to provide strategic management and oversight of their area of The Orange Door and locally-based support staff to facilitate operations, partnerships and connection to government and the broader service system.
• Coordinating infrastructure arrangements for each area including locating, leasing and designing premises, site fit-out, IT requirements and works.
• Workforce induction, training and development program including an induction training program for all workers in The Orange Door[5].
• Development of Partnership Agreements to formalise the partnership between all partner agencies, including government. The agreement outlined the requirements for working in partnership at The Orange Door, and was signed between all parties, except DHHS.[6]
• Formation of local establishment and governance groups: to monitor and plan for the local context.[7] This included: Hub Leadership Groups (HLGs), Hub Operations Leadership Groups (OLGs), Local Hub Establishment Groups and Aboriginal Advisory groups.[8] [9]
• Communications and engagement: to ensure consistent branding, messaging and feedback with a broad range of stakeholders, including the public, sector stakeholders and across government.

Establishment processes that were managed by the partnership agencies as part of The Orange Door included:

• recruitment of practice leaders and practitioners
• operationalising the model
• internal change management in preparation for services and service delivery
• communications and engagement with their workforce and stakeholders

Establishment activities for the launch areas commenced when FSV was formed in July 2017, following development of the Support and Safety Hub Statewide Concept (released in the same month)[10] and considerable joint work between FSV and stakeholders on service model development. While significant work and consultation was undertaken by the Department of Premier and Cabinet (DPC) to design and develop the statewide concept and secure funding allocation across one year,[11] activities to establish the model occurred within a comparatively compressed timeframe of under 18 months while FSV continued to engage with stakeholders across the state during this establishment phase[12]. Evaluation evidence indicates that timeframes for establishing The Orange Door did not allow for the extensive range of operationalisation activities required to set up the foundational model, and had to be implemented rapidly.[13] The requirement to locate suitable premises and recruit an experienced workforce were key challenges that contributed to delays in the planned timeframes.

A significant volume of change for the sector

The amount of change for the sector – and particularly practitioners – has been unprecedented. Transitioning to The Orange Door service model has necessitated a completely new way of working, in conjunction with learning how to navigate new systems and technology (eg the CRM system), adjust to new processes (ie workflows), a new team model (ie colocation), practice change (ie from case management to intake and assessment) and operational changes arising from bringing together some services that had not previously worked together and aligning to a consistent service response. This has placed immense pressure on staff as they adapt to their new roles and new systems while providing client support.[14] The volume of change in almost every aspect of practice needed over a short period of time has complicated initial implementation – despite the workforce bringing skills, expertise and existing practice and frameworks to enable service to be provided to clients from commencement. Evaluation feedback from external stakeholders was that more a sequential or phased rollout would have supported smoother implementation, to allow for the adoption of new systems and ways of working.[15]

The transition has demanded a level of flexibility and agility from the workforce in order to work within uncertainty as the model, systems and processes have been implemented, refined and updated. This level of change has been significant for the sector without a corresponding focus on change management needs. The nature of the model means that this change management focus needs to be coordinated and progressed by both FSV and partner organisations.

'… what was overlooked was change readiness, there needed to be a phased approach to change management… The scale of the change was just too great, we needed time to prepare the staff for winding up their current practice and then merging their practice to the new system.' – HLG evaluation interview

Establishment activities

The amount of change associated with implementing the numerous components of The Orange Door model, timing considerations and change readiness of the sector are overarching complexities that complicated establishment and initial service delivery of The Orange Door. Figure 3 highlights five key establishment activities – noting this is not an exhaustive list – each of which is discussed in the following section.

Recruitment

Recruitment of The Orange Door workforce community service organisation funded positions across practice leadership and practitioners) began in late 2017. The allocated funding for full-time equivalent units (FTE) was based on outputs rather than pre-determined inputs to allow agencies flexibility to recruit to meet their output targets.[16] Despite early recruitment efforts, none of The Orange Door areas had full contingent of staff at service commencement.[17] Estimates provided to the evaluators suggests that in one area, there was less than half of staff expected on day one, 'despite 100 per cent of L17s[18] coming in. It destroyed morale.'[19] Evaluation evidence indicated the workforce and agencies did not have sufficient preparation time for recruitment, which may be due to delays to ‘recruitment products,’ and 'underestimation of the time required for end to end recruitment.'[20] It is also possible that the predominant functions of intake and assessment may not appeal to all people working within the broader family violence and family services workforce. While this may be the view of some practitioners interviewed as part of this evaluation, it is important to remember that The Orange Door undertakes a range of additional functions.

Similarly recruitment for FSV employed positions[21] has been challenging, and continued after service commencement in most areas. [22] Vacant positions in the initial periods tended to be the Service System Navigator and Operational Support officer[23]. The impacts of these vacancies were felt by The Orange Door workforce. In particular, the absence of an Operational Support Officer in some areas meant that operational processes and infrastructure issues had to be dealt with by the leadership team (eg OLG or HLG teams), distracting their focus from strategic leadership[24].

In addition, subsequent rounds of recruitment have been necessary due to staff-turnover. While we are unable to quantitatively verify what the rates of staff attrition have been, interviews during fieldwork indicated that attrition rates have been higher than expected across most The Orange Door areas, and are likely due to a number of factors:

• Position descriptions that did not accurately reflect actual roles. A consistent theme through the evaluation was that the expectations of a role based on the position description did not align to the reality of the role once operations had begun.
• Role ambiguity for some positions at The Orange Door. Some workers did not have a clear understanding about their role and responsibilities even after a number of months working at The Orange Door.
• Working at The Orange Door has demanded a high level of adaptability to work within the uncertainty that the innovation of service delivery in The Orange Door involves. For many workers this was unexpected; some did not have the skills or willingness to work within this degree of uncertainty, others embraced the innovation. This finding needs to be balanced with the evaluation finding that there has been, and remains significant excitement by many in The Orange Door for the new model.
• Some workers felt that they did not have a choice to move to The Orange Door as the employing agency positions were subsumed into The Orange Door.
• Some workers reported that they have struggled to make the transition to a narrower role of intake and assessment from broader case management and others do not believe they have skill on entry to The Orange Door to work immediately within an integrated service model[25].

It is our understanding that recruitment is now being tailored to these needs by FSV and partner organisations.

There was also some suggestion from the workforce during the evaluation that the anticipated Full Time Equivalent (FTE) allocations for each discipline needs refinement. Some areas, for example may require more men’s workers due to an increasing demand.[26] Quantifying the adequacy of FTE across specialties was beyond the scope of the evaluation, nor could it reasonably have been expected to be predicted with accuracy ahead of the model being implemented given that this model is a world first initiative in service delivery.

Induction training

FSV facilitated a four-day induction training program to introduce staff to the service model and practice framework, and to provide foundational knowledge on working with family violence, children and families and perpetrators.[27] It aimed to equip workers to commence service delivery from the first day of operations, and guide integrated and safe practice.[28] Content delivered during the induction training program included:

• The Orange Door service model
• Interim Integrated Practice Framework
• use of the CRM platform
• the 'Client Experience Toolkit'
• family violence practice
• children and family practice
• working with child protection
• perpetrator accountability and visibility
• risk assessment and management for adults and children, Tools for Risk Assessment and Management (TRAM), including contextualising The Family Violence Multi-Agency Risk Assessment and Management Framework (MARAM) The Orange Door
• information sharing within The Orange Door context

Modules were delivered by FSV, Domestic Violence Resource Centre Victoria (DVRCV), No To Violence (NTV), DHHS Child Protection and the Centre for Excellence in Child and Family Welfare (CFECFW). Aboriginal self-determination and people’s agency and intersectionality and diversity were not included in detail as part of the initial training program, noting that these materials are to be developed and delivered post service commencement as they need to reflect and be tailored to the local area and community.

Each area initially received two rounds of training to minimise disruption to service delivery and allow partner agencies to send half their workforce to either of the training dates. Further rounds of induction training have been run post service commencement as new staff have been recruited.

FSV conducted an evaluation of the training program primarily through feedback forms which focused on the content. Overall, the evaluation found that while a positive impact was seen in terms of building 'on existing skills and knowledge of the participants across all training modules,' participants wanted further information on how The Orange Door would operate in practice in order to obtain a better understanding of their role.[29] This finding was supported with our evaluation data: and in particular, further clarity about integrated practice and how to operationalise this at The Orange Door was required. As The Orange Door continues to operate and it becomes clearer how the model is operationalised in practice, FSV are well placed to build on and tailor these early induction programs to meet the emerging needs of the workforce.

Evaluation evidence indicated that for some, the training was pitched at a low skill level for practitioners for their own speciality but did not provide them with sufficient understanding of disciplines outside of their experience. Training content is discussed further in Section 6, although it is important to acknowledge that FSV is refining the induction training program based on findings from its evaluation, as well as developing a Workforce Strategy for The Orange Door which includes 'effective approaches to workforce induction and on-boarding for new workers.'[30]

Guidelines and policy

FSV produced a comprehensive range of frameworks and practice guidance to ensure 'a consistent, statewide approach to operating The Orange Door across the launch areas.' Guidance materials were intended to be read in conjunction with other resources, which are outlined in Table 1.[31]

Table 1: Foundation documents

During evaluation fieldwork, many practitioners admitted that they had not read the guidance materials. While this was identified in the orientation training, when investigated, The Orange Door staff indicated it was due to the amount of documentation that had been produced, the length of the content and time pressures.[32]

Those who had read the documentation did not find the content useful for preparing them to work at The Orange Door as they expected more definitive practice based guidance including examples of how the model would or could work in practice. For example, while a vison of integrated practice was outlined in the Interim Integrated Practice Framework, there were no clear benchmarks or detail about how to operationalise this vision. As a result, people reverted back to how they previously operated.[33] Several Hub Leadership Group (HLG) representatives reported that they were continuing to work through what integrated practice looks like in The Orange Door six months after service commencement. It is important to note that though these foundational documents and guidance materials do acknowledge that the model is intended to be informed by evidence, and it seems reasonable that initial documentation would need to be iterated in response to ‘user’ feedback (in this case through use by The Orange Door workforce).

The Orange Door service model was never intended to be 'static or fixed, but rather informed by emerging needs and evidence,' and evolve and mature in response to 'community needs, co-design, and practice learnings.'[34] Despite this, however this evaluation found that greater parameters and prescription were needed in the service model to provide a sufficient starting-point for establishment (particularly in the absence of sequenced and ongoing change management processes which will necessarily need to be a joint responsibility of FSV and partner organisations), including more definitive guidelines on how to operationalise The Orange Door.

Additionally service model requirements (including those in the service specifications) were considered by many in the workforce to be too high-level to underpin operationalisation. Instead practitioners wanted practical operational guidelines to help mobilise operations. In time this may be developed by the OLG, however at the time of fieldwork, these issues were still being worked through. While practitioners reported that workflow needed to be tailored to the specific area, they requested a baseline to give them confidence that they were progressing in the right direction.[35] It is unclear if they expected FSV or local HLGs and OLGs to provide this direction.

There was confusion about the level of consistency of operational requirements with the guidelines in Support and Safety Hubs Service Model. Some leadership members indicated that a certain level of prescription was required to facilitate understanding of their operational requirements, and for FSV to clearly denote aspects which could be tailored to the local context and which required state wide standardisation. We understand that FSV are aware of these tensions and are currently considering how to address this issue. Tensions between statewide consistency and localised flexibility are likely to remain an ongoing issue in a reform of this scale and complexity. As further areas are established, we could expect to see clarity emerge about which requirements need statewide consistency.

Systems and tools

In addition to practice guidance, new Information Technology systems were developed to support service delivery, including the CRM, CIP and TRAM[36]. The CRM has undergone planned continual development[37] since service commencement to improve its functionality and to support service delivery. Practitioners reported that the initial functionality of the CRM was difficult to operate, time consuming and had provided limited support for their work. This is expected in any first iteration of a new IT system. In addition, there was a lack of supporting guidance or system workflows to help practitioners navigate the system, largely due to the ongoing development of the CRM.[38] The delay between induction training – where practitioners received CRM training – and service commencement further meant that some practitioners reported that they had to re-learn the system. The evaluation found that the workforce has varying levels of competency and confidence using the CRM which was confounded at that time by a range of IT functionality issues in The Orange Door.[39]

Evaluation evidence found that the issues with functionality of the CRM have impacted service delivery. The time practitioners have spent trying to navigate and overcome issues with the CRM compromised their ability to work through cases efficiently in these early stages. These difficulties compounded an already challenging transition to The Orange Door. It is however important to note that during the second round of evaluation fieldwork many practitioners reported that subsequent updates to CRM had already improved functionality.

'…the CRM went live on day one. A new data system on day one was not ideal. There should have been a phased approach to learning the systems.... we’re falling behind in cases' [40] – Practitioner interview

While the time pressures associated with opening The Orange Door created challenges to CRM deployment, and likely impacted on the readiness of the system at service commencement, the workforce did report the benefits of an evolving IT system and iterative FSV upgrades. For example, practitioners have seen tangible changes in the functionality of the CRM as a result of feedback they provided, and are beginning to see a more operational and fit-for-purpose system emerging. Practitioners were also complimentary of the CRM trainers, and appreciated them coming to The Orange Door to provide on-the-ground support.[41]

Similarly, the CIP has taken time to become fully operational, although these issues have not impacted The Orange Door workforce to the same degree as CRM, given the system sits outside The Orange Door. For example, establishment challenges for CIP included:

• on-boarding of agencies took longer than anticipated
• recruitment of suitably qualified data custodians
• early CIP reports taking longer to prepare than in the initial testing (nearly three hours compared to one hour respectively). Since ‘go live’, modifications were made to the platform to improve efficiency, and data custodians have become accustomed to the platform.[42] However, demand remains high in terms of the time it takes to prepare the CIP report. This may remain an operational challenge into the future

Despite the above challenges (many of which are reasonable given the innovation that CIP represents), there is strong evaluation evidence that CIP has been valuable for obtaining information to help practitioners more accurately assess and manage risk. Commencement reports (produced for each area) demonstrate that practitioners were using the CIP in the first months of service delivery, and this was verified during the fieldwork. While there is variation in the amount of reports that were requested across The Orange Door areas, this may be due to the different timeframes in which data was recorded.[43] See Section 5.3 for a timeline of CIP reports by The Orange Door area.

Physical premises

The Victorian Government committed to working with local communities, and local and statewide service providers, to establish the location of each physical site of The Orange Door.[44] FSV conducted extensive work to ensure client-centric design of The Orange Door physical site (see Chapter 5 for further discussion). The physical site of The Orange Door was an important part of establishment as the facilities functioned to bring together different workforces under one space, and colocate practitioners to support multidisciplinary practice. There were significant challenges in finding appropriate premises for the physical site for The Orange Door, as well as completing the fit-out within the timelines that FSV needed to work within.[45] Additionally, it is our understanding that minimum viable requirements for ‘go live’ were still being reviewed with each HLG leading up to service commencement, leaving insufficient time for areas to comply with the new criteria.[46] It is also our understanding that this will not be repeated in subsequent rollouts of The Orange Door and FSV have adjusted sequencing and go live criteria for future sites.

Given these time pressures, a number of issues with the physical premises were encountered at service commencement for several areas. While we understand there have been adjustments to the sites since then (and following our fieldwork) to rectify a number of these issues, it is important to acknowledge the problems and highlight how they impacted establishment and initial service delivery. An overview of these issues and their impacts on the workforce, culture and service delivery at the time of the evaluation are summarised in Table 2.

Table 2: Issues encountered with physical premises at establishment

Notwithstanding the above challenges, the workforce recognised The Orange Door site – as a key enabler for bringing together disparate agencies, and colocation was a foundation for building integrated practice:

'People sitting next to each other means they can learn from one another.' [48] – practitioner interview

It is our understanding that a number of the issues identified have subsequently been addressed by FSV or by The Orange Door area.

Partnership formation

The Orange Door is underpinned by a partnership approach to:

• ensure collective leadership
• harness specialist skills and expertise from a range of specialties to deliver the service model functions and
• offer a holistic response to individuals and families[49]

The Orange Door partnerships comprise of government and non-government agencies across different sectors. To formalise this, Partnership Agreements were put in place to 'lay the foundation for collective leadership of the Hubs.'[50] Partnership Agreements served to create a legal mechanism across the organisations and key government agencies in the Hubs to 'embed collective leadership, set down a shared vision, goals and principals, support integration of services, formalise governance mechanisms and effectively deliver Hub functions.'[51]

Evaluation evidence indicates a perception by many within the HLGs and OLGS that the Partnership Agreements and Service Agreements did not include the level of detail they were expecting to provide a suitable foundation for the partnership at The Orange Door. This is to be expected for such a novel service delivery model, and was predicted through the inclusion of a 12 month review in initial Partnership Agreements. It is also unclear if the Partnership Agreements and Service Agreements would have been sufficient at any point in The Orange Door service model, or if a lack of understanding or specificity in these documents reflects the point in time and need to refine these as the service model began to be operationalised and resourcing/funding allocation refined within the first few years of operations. Key requirements of the Partnership Agreements include: obligations of The Orange Door partners including resource and funding contributions (in the Service Agreements); establishment of the HLG (including its functions and expected terms of reference); and details of how complaints are to be managed as a collective.[52]

While Partnership Agreements provided a solid starting point for establishing and defining governance arrangements and operations, further discussion will be needed with the agencies to ensure that there is sufficient detail within these agreements to embed leadership and governance within the new partnerships. This was particularly relevant in the context of the matrix model which adds a layer of additional complexity to establishing governance arrangements (see Section 3.2). In addition, some leadership groups discussed the need for Hub Leadership Group terms of reference (included in the Partnership Agreement) to be reviewed and updated. The need for revisions, further detail or updates were reported to evaluators as covering areas such as: the role of the Operations Leadership Group (OLG), data collection and communication management at The Orange Door. It is our understanding that these aspects are currently being reviewed and endorsed by FSV.[53]

Collective leadership in these initial stages has been impacted by a perceived imbalance of contributions from partner agencies at The Orange Door. Some leadership groups reported that while there are variances in the FTE contribution and funding that each community service organisations receives attached to their service agreement, the contribution they are expected to make to The Orange Door has not as yet been adjusted to reflect the contribution. For example, some community service organisations with smaller funding contributions who provide staff with specific expertise are finding that early indications of demand mean that they do not have sufficient workers in The Orange Door. It is unclear if this is a point in time issue, or one that will remain once operations stabilise after the initial implementation phase. This needs to be considered in the light of the existing relationships and networks in the areas The Orange Door has been established. It also needs to be considered in the context that collective leadership takes time to be established and embedded.

Besides Partnership Agreements, FSV has been instrumental in forming local establishment and governance groups to plan and implement The Orange Door at the local context. We understand that FSV and some HLGs facilitated a number of workshops (including local establishment group forums and broader sector forums) to bring sectors together and consider ways of working in collaboration, as well as to try to further define some operational aspects prior to service commencement.[54] While this was considered imperative for establishing the group and initiating co-design, a number of stakeholders reported that more lead time for the HLG and OLG would have been valuable prior to service commencement, specifically to test and refine systems and processes.

A further contributing factor to slowing down the operating rhythm of leadership groups was a number of 'false starts'[55] associated with the opening of The Orange Door.

'Open day was a key challenge, having a date which changed many times and then confirmed at short notice… we had not prepared our staff and then had three to four weeks before we moved in.'[56] – OLG member interview

This has been addressed by FSV in considering what they are able to control in terms of start-up responsibilities and in a new sequencing and ‘go live’ criteria for establishing a new The Orange Door area.

A key enabling factor to collaboration – and particularly the HLG – has been various pre-existing alliances such as the Child First and Family Services Alliances, Family Violence Regional Integration Committees and Indigenous Family Violence Regional Action Groups (now Dhelk Dja Regional Action Groups). For a number of Orange Door areas, these existing networks have served as the platform for partnerships to build from, and they strengthened the leadership groups through prior working relationships and arrangements. Indeed these leadership groups reported that it would be very difficult to form partnerships so rapidly in the absence of their past working relationships.[57]

While this evaluation has focused on partnerships between community service organisations that comprise The Orange Door, further investigation is required to understand establishment of The Orange Door within the broader service sector, given the role of The Orange Door in connecting clients to the right services and coordinating these connections. It is important to understand the progress in developing and sustaining these collaborations, both operationally (i.e. establishing referral pathways and practices to and from The Orange Door) but also in terms of perception of The Orange Door and its role within the system. Limited information is available at this point in time for the evaluation and it was not within scope for this evaluation to examine the role of The Orange Door within the broader service system, however a number of indicative findings have emerged which may need to be explored and validated in future evaluations. These findings include:

• the need to understanding how The Orange Door aligns within the family violence, family and child wellbeing and the broader social service sectors
• the need for supporting systems and infrastructure to support referrals, noting that this was a work in progress during our fieldwork (ie workflows)
• recognition from The Orange Door areas that establishing engagement with the sector is an ongoing, long term activity, and a key role of the Service System Navigator (SSN)

Partnerships and networks enable information sharing and referral practices

Collaboration is central for information sharing to connect disparate agencies and sectors, and share critical information about clients. Despite expected challenges with establishing these connections in the early stages of The Orange Door, there is evidence that information sharing is an aspect of The Orange Door at present – particularly in relation to information transfer and collation within The Orange Door and information sharing between The Orange Door and external referral agencies. It is also important to note that there is a lack of consistency across The Orange Door areas in relation to referral processes and time frames.

Information sharing in The Orange Door has been facilitated by Part 5B of the Family Violence Protection Act which streamlines the sharing of confidential information as it treats community service organisations within The Orange Door as one organisation or partnership.[58] The Act allows practitioners to collect, use and disclose confidential information to other practitioners only within The Orange Door so long as the purpose is related to the provision of The Orange Door services. In particular it allows practitioners to share information with a colleague to draw from the expertise of a worker from a different specialisation.[59] Practitioners recognised the relevance and importance of this to their practice at The Orange Door.[60]

As The Orange Door evolves, the various components of ‘information sharing’ will become clearer in terms of the operational environment. Distinctions will be able to be made between:

1. Information sharing to support service delivery within The Orange Door.
2. Information sharing between external parties and The Orange Door (either collecting and/or disclosing information as part of a referral) or for assessment and risk management.
3. Information sharing from The Orange Door to a client to manage risk or provide service.
4. Information sharing to support monitoring and performance of The Orange Door.

The nature of the developmental evaluation early in the establishment of The Orange Door meant the focus on information sharing was limited to sharing within The Orange Door (and The Orange Door partner agencies 1).

In addition to this legislated mechanism[61], there are a number of factors that currently support information sharing, and the provision of services, within The Orange Door:

• Colocation of practitioners from different disciplines is important for information sharing and working collaboratively. Many practitioners acknowledged that information sharing is easier and faster than if they were situated in their employing agencies. However, it was reported that information sharing occurs in an inconsistent manner, is often supported by previous relationships and occurs more easily between certain disciplines (for example specialist victim survivor family violence services and child and family wellbeing services). This is to be expected at this early stage and information sharing is likely to be further embedded as The Orange Door evolves.
• Multidisciplinary teams and meetings such as cluster teams[62] or function (ie triage) meetings comprising of practitioners from each discipline. This has embedded a more structured approach to information sharing than colocation and has promoted more consistent information sharing. This was seen particularly in ‘function teams’ (such as initial screening teams) which were underpinned by an agreed and operational workflow process.[63] Operational workflow processes could be embedded in future areas of The Orange Door from the outset.
• Access to employing agency’s client record systems to obtain information from a range of different sources to better understand the risks and needs for an individual or a family. Practitioners retain access and use of their employing agency system so they are able to search for relevant information about a case. The partnership in this respect facilitates quicker access to a more comprehensive range of information. The benefits of access to partner information were widely regarded by the workforce and stakeholders.
• The CRM is intended to assist with gathering information and record management to facilitate sharing of client information between practitioners at The Orange Door.[64] Teething issues with the CRM functionality at the time of the evaluation currently limits the ease of information sharing at The Orange Door. We understand that a number of updates have subsequently been made and functionality has improved. The system has the potential to be an invaluable repository of information to support The Orange Door workforce, as it consolidates new and existing information from across the partnership.

Further discussion about information sharing protocols is discussed in Section 7.

Further discussion about information sharing challenges beyond The Orange Door is detailed in Section 8, and includes:

• supporting secure data transfer between siloed systems
• formal feedback loops for oversight of client outcomes
• clarity in terms of the amount of information practitioners should provide about and/or to a client

In relation to referral practices, evaluation evidence suggests that existing networks are instrumental for setting up referrals to The Orange Door. Having a foundation of existing referral pathways to work through has been useful, and practitioners have maintained their connections to services they are familiar with, including their employing agency to facilitate both referrals to and from The Orange Door. Going forward, it will be important that the CRM collects granular data on referrals from The Orange Door to allow for a more detailed understanding of the referral network within which The Orange Door exists in each area.

It is however unclear to what extent practitioners are referring to services outside of the partnership at this point in time.[65] This may largely be due to a focus and prioritisation on internal process development and operations at The Orange Door in the early stages of implementation. During evaluation fieldwork we saw the refinement and development (and later endorsement) of a referral form and process at one area that was focused on the partner services. Workers hoped that this template would improve information sharing and consistency and formalise referral pathways within the partnership. At this stage of The Orange Door’s evolution, it is to be expected that attention would be on internal development, and efforts to implement robust systems and processes that can be replicated and rolled out more broadly will follow.

As such, referral practices and processes remain a work in progress. Our evaluation meetings with leadership teams confirmed that further work is required to develop referral pathways and relationships with universal services (such as schools and hospitals).[66] This requires concerted and ongoing effort, particularly as many stakeholders believe that the role and function of The Orange Door is not widely understood at present. There is some limited evidence of inappropriate referral pathways into The Orange Door occurring because of the perceived lack of understanding about The Orange Door. This is already anticipated and will need to be monitored by FSV and partner agencies to ensure that The Orange Door is not seen as a ‘catch all’ social service.

'Education with the broader system needs to be done, as currently we’re seen as a one stop shop.'[67] – Practitioner interview

Leadership teams also reported that local services and referral pathways need to be mapped, and that this gap is particularly challenging for The Orange Door areas which service a large geographic area or where there are a large number of service providers.

For further information on referrals in and out of The Orange Door, refer to Section 4.1.2 and 4.2.1.

Key finding 1:

Extensive foundational work to operationalise The Orange Door was completed within a compressed timeframe. The timeframe negatively impacted on a range of establishment and operational activities. Evidence of the impact of this compressed timeframe was still being seen at The Orange Door at the time of this evaluation, despite each area managing to provide the full suite of foundational services

Key finding 2:

Significant change management to support the implementation of The Orange Door service model is needed by both FSV and partner organisations. Initial focus has been on establishment and operational activities in a compressed timeframe with a need to shift the focus now to change management support

Key finding 3:

Each area of The Orange Door areas was not at full contingency of staffing during the fieldwork. There have been a range of challenges in the recruitment of appropriate staff and in retaining those staff. This will be addressed by FSV through a workforce strategy

Key finding 4:

Roles and responsibilities need increased clarification and standardisation across partner agencies. This is important to facilitate effective matrix management

Key finding 5:

Initial induction training was provided but the content of induction training will need to evolve in response to feedback. It will also need to be tailored to meet specific workforce needs

Key finding 6:

FSV’s foundational documents provide overarching direction for The Orange Door, however they are not well understood or used by The Orange Door practitioners. Further operational ‘translation’ of these documents is needed to help the workforce understand how to operationalise the model

Key finding 7:

A range of physical infrastructure issues impacted operations and colocation of the workforce in some areas at service commencement. These have been or continue to be addressed by FSV

Key finding 8:

Collaboration is central for information sharing within The Orange Door. Despite early stage challenges with establishing strong collaboration, information sharing is already happening within The Orange Door primarily through the CRM, colocation and multidisciplinary working arrangements, and through access to employing agencies’ client record systems.

To what extent are governance arrangements for each Orange Door site operating effectively?

Governance of The Orange Door

Governance of The Orange Door is a combination of specialist service delivery expertise (from discrete service sectors) as well as government management, oversight and systems planning. A ‘hybrid model’ was conceived to support transition of The Orange Door as a new feature in the service system.[68]

The model is operationalised at The Orange Door with FSV providing oversight and leadership facilitation, and the partner agencies leading local implementation, management and service delivery of The Orange Door. Both FSV and partner agencies come together under a partnership agreement to foster collective leadership.

FSV employs Hub (The Orange Door) Managers at The Orange Door area to provide strategic and operational oversight. Hub Managers are supported and report to Assistant Directors in the Support and Safety Hubs Operations branch based at FSV.

Hub leadership groups (HLG) have been formed for each area of The Orange Door, to lead local implementation and management of the model along with an Operations Leadership Group (OLG) to manage operational processes and policies. Both the HLG and OLG comprise of a representative from each partner agency, and others (for example the Department of Health and Human Services and Victoria Police).[69]

Governance arrangements contribute to The Orange Door effectively delivering its functions

The Orange Door governance model for the areas is supported by terms of reference, Partnership Agreements (discussed in Section 3.1) and 'Service Agreements between core agencies and the Department of Health and Human Services (DHHS)' that 'may evolve over time.'[70] These form a formal basis for governance at the outset of establishment of The Orange Door.

A collective approach to governance was designed to give 'government and non-government agencies a stake in the success of The Orange Door, support collaboration, and promote shared responsibility and mutual accountability for establishment and implementation.[71]

Bringing together numerous organisations across a variety of settings has been ambitious, particularly within the challenging timeframes discussed earlier. Establishing expansive governance arrangements at The Orange Door has had its challenges, particularly in terms of coordination, embedding roles and responsibilities and providing a cohesive and united approach to implementation.

'How do you bring a group of different agencies together to conceptualise how they will think about this in a coherent way?'[72] – HLG member interview

It is important to recognise that some leadership groups are further progressed and have clearer roles and responsibilities than others. Evaluation evidence indicates that The Orange Door areas that had previous working relationships or alliances were able to leverage from these, and 'hit the ground running' when it came to service commencement. This is to be expected in a reform of this scale, given the scope of change for the sector and interests involved. Additional lead time for the OLG groups to establish as an operational group prior to commencement of a new area are recommended for future rollouts.

'It is not clear what the local governance arrangements for The Orange Door are. Before we have the risk of people falling between the gaps of services, and now we have the risk of people falling through the gaps in coordination – local coordination.'[73] – Stakeholder interview

In the early establishment phases, the lack of clarity/understanding of the governance arrangements at The Orange Door was raised in relation to the division between strategy, management and operations. While the service model, service specifications, Partnership Agreements (and HLG terms of reference included in the Partnership Agreements) provided information on the intended role of governance groups, interviewees stated that there was a need for these documents to be more prescriptive about the role of the HLG and OLG.[74] This is a challenge in the early phase of the roll out as the balance between statewide consistency and local flexibility will be evolving. It could however be expected that some iterations will be needed to governance structures and arrangements after the initial implementation phase of reforms of this scale.

At some areas, the lack of clarity about governance arrangements during the intense early implementation phase has resulted in the conflation of roles and responsibilities between the HLG and OLG, particularly during the initial months of operations. The focus for both these groups during this time was establishment and operational issues, with no clear boundary between what operational issues needed to be contained to the OLG and their decision-making authority. While greater consideration of operational matters could be expected in the early stages of implementation, HLGs now need to refocus their efforts to move beyond operational matters to more strategic matters. The lack of reporting and accountability of the OLG to the HLG has also been unclear, contributing to the HLG having detailed involvement in operations.[75] It is our understanding that most areas have now conducted workshops with their OLG and HLGs to further define governance roles and decision making and that FSV is working with partner organisations to review the Partnership Agreement.

In those areas where governance roles were experienced as being unclear, we have identified that this resulted in delays to operational decision-making. For example, there have been delays to endorsing operational processes and workflows. At some areas the large number of agencies represented at the HLG and OLG has delayed decision-making with the desire to gain consensus from all partner agencies. In comparison, The Orange Door areas with smaller HLGs were seen to be more efficient in making decisions. Pathways for escalation and identification of risks were also unclear.[76]

Governance arrangements embed collective leadership of The Orange Door

As discussed in Section 3.1, there have been challenges to establishing collective leadership quickly. This was particularly the case when community service organisations perceived that there were inequitable contributions of FTE and funding attached to service agreements. Collective leadership and the establishment of a well-functioning ‘operating rhythm’ was also challenging for community service organisations that had not previously worked together.

Collective leadership of The Orange Door and how this is structured in terms of operational management of services is within a matrix model of management. In a matrix model, the management structure has employees who report to both a ‘functional’ manager and a ‘product’ manager. While such lateral coordination can facilitate joint responsibility – a key aim of Partnership Agreements – it is also acknowledged to be one of the most complex management structures in terms of implementation and sustainability.[77] This complexity appeared to be amplified in larger partnerships of The Orange Door. Leadership teams are still trying to determine how collective leadership can work in some areas and it may require concerted effort to work through these challenges at each area.

'We’re having a leadership meeting this afternoon where [an employing agency] have got an external facilitator to work out how the matrix model works.'[78] – Practitioner interview

As efforts to establish an effective matrix model continue, the complexity of the arrangement has been experienced at The Orange Door in a number of ways:

• The authority of Hub Managers to direct The Orange Door operations. While Hub Managers are tasked with providing strategic and operational leadership, in reality this is challenged in a matrix model as all partners have shared governance of The Orange Door. As a result, no one individual or organisation can steer or guide the operations, and this can delay decision-making. Research on matrix models suggests that a lead or coordinating role is critical to success of these models.[79] Evaluation evidence indicated that the workforce looks to the Hub Manager for direction but the Hub Manager role is structured to influence rather than direct operations.
• Co-ordination in people management. In the matrix model at The Orange Door, practitioners in effect ‘report to’ or receive direction from three ‘managers’: their team leader at The Orange Door, their employing agency (for professional development) and the practice leader (for case related queries), all of which could be from different agencies. This presents logistical challenges for even basic functions, such as scheduling holiday arrangements or attendance at training sessions, complicating backfilling of positions to ensure sufficient staff contingent at The Orange Door.
• Consistency of practice due to employing agencies’ practice background. While The Orange Door has been working to adjust and clarify roles and responsibilities for staff in a formal and informal manner, some practitioners are unclear as to the roles and responsibilities of their colleagues. This has at times led to some staff reverting to pre-Orange Door practices and aligning with the roles and responsibilities of their employing agencies. For example, some practitioners reported they were undertaking more ‘case management’ type activities.

Governance arrangements support integrated practice and integration

While a vision of integrated practice was outlined in FSV’s Interim Integrated Practice Framework, the central challenge in establishing integrated service delivery in The Orange Door will be to articulate and realise a shared understanding of what this looks like on the ground.

'… nobody said, ‘this is what integration means’, so all agencies made up their own definitions'[80] – Practitioner interview

The added complexity of supporting integrated practice and integration results from the different ideological and practice frameworks that family violence, perpetrator services and family service sectors bring to The Orange Door. Services within these sectors have historically operated independently and in different ways. This complexity presents challenges for embedding integrated practice and operating cohesively in a complex matrix arrangement where leadership groups are still trying to determine how integrated practice should be operationalised. This is to be expected to some degree in the early stages of implementation and while there are many case examples of integrated practice, a clearer definition is needed on what this means in terms of staff retaining their specialisation. In particular, there are different practice approaches which have been ‘brought’ over to The Orange Door, which shape how they conduct practice, and potentially undermine integration between disciplines. Having said this, the evaluation also found a strong commitment by the workforce to integrating disciplines within The Orange Door – indicating that working through these challenges was a goal the workforce shares.

'We’ve still had cultural issues with workers saying they’re not doing women, men or children’s work'[81] – Practitioner interview

A number of leadership positions have been identified as key to driving integrated practice, specifically the Integrated Practice Lead. Currently, roles and responsibilities for this position lacks clarity at an operational level, and will need to be determined more specifically in the future.

At the time of the evaluation, no publicly identifiable access points had been formally established. The Statewide Concept outlines that satellite access points, or ‘spokes’ will be established at alternative physical locations 'where people can access support offered by The Orange Door.'[82] Spokes may be tailored for targeting specific access needs, such as for Aboriginal people or people in rural and remote areas, or perpetrators of family violence. As ‘spokes’ are established, they will deliver core functions of The Orange Door, such as 'multi-disciplinary screening and triage, needs assessment and support planning.'[83] [84]

At present, staff work from Colac (associated with The Orange Door in Barwon) and Swan Hill (associated with The Orange Door in Mallee). These have not formally commenced as publicly identifiable ‘access points’, and integration is ongoing. Notwithstanding, evaluation evidence noted differences in how Colac and Swan Hill are currently interacting with The Orange Door in Barwon and The Orange Door in Mallee respectively:

• Staff in Colac are situated within Colac Area Health, which is an integrated health service that provides medical and allied health services as well as community services. Colac Area Health operates independently of The Orange Door in Barwon, and has built on an existing service model which encompasses the colocated services at Colac Area Health including family violence case management services, family counselling, and mental health services.
• The Orange Door staff located in Swan Hill are 2.5 FTE workers and participate in the morning screening meeting held at The Orange Door in Mallee. The physical location is situated within the offices of the Mallee Domestic Violence Service (MDVS). There is some question over how this location influences the intended interdisciplinary nature of The Orange Door. Swan Hill practitioners consult with practice leads (in Mallee) as necessary. [85]

Key finding 9:

Pre-existing working relationships mobilise effective governance arrangements, as well as an agreed sense of vision and direction for The Orange Door. This can be complicated if there are numerous employing agencies involved in the partnership with competing interests and expectations of The Orange Door

Key finding 10:

The matrix management model has been challenging to implement in the context of an innovative service model and new partnership structure. As clarity in roles and responsibilities, accountability and decision-making are defined and operationalised in each area of The Orange Door, the matrix management model may become more effective for the workforce

Key finding 11:

The integration of alternative access options requires implementation in order for these to work effectively. As FSV formally establishes publicly identifiable access points, a focus will be needed on integration with the primary physical site

[1] FSV was established in July 2017.

[2] DHHS, ‘Support and Safety Hubs – flexible funding and brokerage’, 2018.

[3] FSV, Hubs Service Model Slide Deck.

[4] FSV, ‘Co-design and establishment of The Orange Door Factsheet’, November 2018.

[5] FSV, ‘Communique’ July 2018.

[6] FSV, ‘Partnership Agreements for the Support and Safety Hubs Factsheet’, July 2018.

[7] FSV, ‘Support and Safety Hubs: Statewide concept’, July 2017.

[8] FSV, ‘Co-design and establishment of The Orange Door Factsheet’, November 2018.

[9] At the time of fieldwork, Aboriginal Advisory Groups for The Orange Door areas were still being established. There was however representation of Aboriginal services on the HLGs.

[10] The Support and Safety Hubs: Statewide concept was released in July 2017. Hub Leadership Groups were convened in February 2017.

[11] This included: co-design of The Orange Door through a series of Community Conversations in Victoria led by Department of Premier and Cabinet in September 2016. This was followed by consultation workshops held in in Geelong, Pakenham, Horsham, Swan Hill, Shepparton, Sale and Craigieburn, as well as with Aboriginal and Diverse Community working groups to shape thinking on the statewide model, design principals and core elements of the model (see co-design and establishment of The Orange Door Factsheet November 2018). In early 2017, a series of workshops with an expert design group were conducted to consider the functional model for The Orange Door and interface with the broader service system. From July 2017. FSV worked with members of the Victim Survivors’ Advisory Council to obtain insights on the functions and features of The Orange Door from service users perspectives.

[12] Most of The Orange Door areas noted that while some early establishment workshops occurred in early 2017 (such as ways of working together), more formal establishment activities to operationalise the model at the local level occurred from July 2017

[13] Evaluation interview – HLG, November 2018.

[14] FSV, Communique, July 2018.

[15] FSV, ‘The Orange Door Commencement report’, May – June 2018

[16] FSV, ‘The Orange Door – commencement report’, 2018.

[17] Within the first period of operations, Barwon, Bayside, NEMA and Mallee had recruited 78.1%; 56.3%; 71.4% and 87.7% of all funded positions respectively. See The Orange Door – commencement reports. However, it is important to acknowledge that the delay in NEMA opening by six weeks meant that this workforce was at a higher level of recruitment. This then led to different workforce challenges in this site (for example, where community service organisations could place their workers in the absence of an operational physical site) prior to commencement.

[18] L17 reports are referrals from Victoria Police Victoria Police officers who attend family violence incidents make referrals (L17s) to specialist family violence services, family services and child protection as necessary

[19] Evaluation data from fieldwork

[20] FSV, ‘Lessons Learned – Tranche 1 Support and Safety Hubs’, 2018.

[21] FSV staff roles include Hub Manager, Service System Navigator, Operational Support Officer, Strategic Planning and Reporting Officers, Client Support and Administration Officer

[22] 56.5%, 45.2%, 58.8% of FSV positions filled in Barwon, Bayside and Mallee after one month of operations. NEMA had a full complement of FSV staff in August 2018

[23] It is unclear if the vacancies in these position was due to difficulties recruiting appropriately qualified personnel into these positions or the condensed time frame for establishment.

[25] It was beyond the scope of the evaluation to do a full workforce profile, an assessment of workers skills to work in an integrated manner was not able to be made. Evaluation judgements are made based on reporting of concerns about skill levels reported by multiple practitioners across multiple sites.

[26] Qualitative evaluation evidence from fieldwork

[27] FSV, ‘Communique July 2018’.

[28] FSV, The Orange Door Induction Training program. Evaluation report, 2018.

[29] FSV, ‘The Orange Door Induction Training program. Evaluation report’, 2018.

[30] FSV, ‘Workforce Strategy for the Orange Door – project outline’, 2018.

[31] This is in addition to other supporting resources such as Client Relationship Management (CRM) system guidelines (2018) Central Information Point guidelines (2018), Family violence information sharing guidelines (2018), and Family violence risk assessment and risk management framework (September 2018).

[32] Qualitative evaluation data from fieldwork

[33] Qualitative evaluation data from fieldwork

[34] FSV, ‘Support and Safety Hubs: Statewide concept’, July 2017.

[35] Information provided by The Orange Door OLG, November 2018.

[36] Please note, not all workers in The Orange Door use CRM (for example, child protection community based practitioners continue to record data using the DHHS CRIS client case record system). Please note, TRAM was initially called ICRAT.

[37] At the time of writing, CRM Release 5 training session was being rolled out.

[38] FSV, ‘CIP-CRM Lessons Learned Register’, July 2018.

[39] FSV, ‘Communique’, July 2018.

[40] Qualitative evaluation evidence from fieldwork

[41] Qualitative evaluation evidence from fieldwork. See also FSV, Communique July 2018.

[42] FSV, internal document: ‘CIP-CRM – Lessons Learned Register’, 18 July 2018.

[43] For example, NEMA requests were collected over 7.5 weeks and later in the CIP’s development as opposed to 7 weeks for Bayside, 4.5 weeks for Barwon and Mallee respectively earlier in the CIP rollout

[44] FSV, ‘Support and Safety Hubs: Statewide concept’, July 2017.

[45] We understand similar challenges have delayed opening of The Orange Door in the inner Gippsland area.

[46] FSV, ‘CIP-CRM Lessons Learned Register’, 18 July 2018.

[47] Barwon Support and Safety hub risk register May 2018

[48] Qualitative evaluation evidence from fieldwork

[49] FSV, ‘Support and Safety Hubs. Service Specifications.’ April 2018.

[50] FSV, ‘Support and Safety Hubs. Service Specifications.’ April 2018.

[51] FSV, Hub Partnership Agreement Template, October 2018

[52] FSV, Hub Partnership Agreement Template, October 2018

[53] FSV, ‘Table of governance discussions – 30 Day Review’, 2018

[54] Qualitative evaluation evidence from fieldwork

[55] HLG and OLG reported ‘false starts’ of dates for commencement, however commencement dates were always tentative and in constant reviews as physical infrastructure and premises were established.

[56] Qualitative evaluation evidence from fieldwork

[57] Qualitative evaluation evidence from fieldwork

[58] FSV, ‘Support and Safety Hubs. Interim Integrated Practice Framework’, April 2018.

[59] FSV, ‘Support and Safety Hubs. Interim Integrated Practice Framework’, April 2018.

[60] FSV, ‘30 Day review report’, 2018.

[61] The family violence information sharing and child information sharing schemes are being rolled out alongside the MARAM reforms. The family violence information sharing scheme commenced in February 2018 with a select group of workforces, including The Orange Door, permitted to share information with each other relevant to assessing or managing family violence risk. The scheme was expanded to further organizations and services in September 2018 along with the commencement of the MARAM reforms and the child information sharing scheme allowing information to be shared to promote the wellbeing or safety of a child between broadly similar workforces. These reforms will include further workforces in 020 and is expected to improve the consistency and amount of information sharing between services.

[62] For example, a multidisciplinary team supervised by a team leader who manages capacity of workload across practitioners in the team

[63] For example intake and triage at Mallee where practitioners were in an established operating rhythm

[64] FSV, ‘Support and Safety Hubs. Interim Integrated Practice Framework’, April 2018.

[65] Data on referrals was unavailable to corroborate this.

[66] Respondents to the 30 Day Review survey also identified priority service interfaces as: core family violence services and family services to facilitate allocation and referrals, crisis services, adolescent family violence services, housing and homelessness services, alcohol and drug services, mental health services, general health services, courts, police, legal services, specialist community sector organisations supporting Aboriginal, and culturally and linguistically diverse community members.

[67] Qualitative evaluation evidence from fieldwork

[68] Victorian Government, ‘Support and Safety Hubs: Concept paper’, July 2017.

[69] Description of each leadership group, membership and roles will be provided as an appendix in the final report.

[70] FSV, ‘Support and Safety Hubs. Service model’, April 2018.

[71] FSV, ‘Support and Safety Hubs. Service model’, April 2018 p11.

[72] Qualitative evaluation evidence from fieldwork

[73] Information provided during stakeholder interview. October 2018.

[74] Qualitative evaluation evidence from fieldwork

[75] FSV, ‘Lessons Learned Tranche 1 Support and Safety Hubs’, 2018

[76] FSV, ‘Lessons Learned Tranche 1 Support and Safety Hubs’, 2018

[77] Hanover Research 2013; Lawson & Gobeli 1987; Turner, Utley and Westbrook, 1988: Katz & Allen, 1985; Sy, 2005

[78] Qualitative evaluation evidence from fieldwork

[79] Hanleybrown, F. Kania, J. & Kramer, M., ‘Channeling change: Making collective impact work’ in Stanford Social Innovation Review, January 26, 2012. ANROWS ‘Meta-evaluation of existing interagency partnerships, collaboration, coordination and/or integrated interventions and service responses to violence against women’ Landscapes: State of knowledge paper. September 2015, p13.

[80] Qualitative evaluation evidence from fieldwork

[81] Qualitative evaluation evidence from fieldwork

[82] Qualitative evaluation evidence from fieldwork

[83] Qualitative evaluation evidence from fieldwork

[84] It should be acknowledged that the establishment of access points for The Orange Door was not identified as an initial priority in the first roll out of The Orange Door in the foundational documents.

[85] Qualitative evaluation data from fieldwork