Foundation service model

To what extent is The Orange Door operating as intended, and making use of systems and processes?

The suite of foundational services being offered

The Orange Door is designed to mobilise the service system for women, children and young people experiencing family violence and families who need support with the care, development and wellbeing of children.[1] The service model documents the suite of services and responses intended to be delivered by The Orange Door in the initial establishment and the role of The Orange Door in connecting people to services. A summary of the service model is shown in Figure 4.

Figure 4: PwC Summary of The Orange Door service model

Source: PwC analysis and interpretation summary based on FSV (2018), ‘Support and Safety Hubs: Service Model’.

Prior to the establishment of The Orange Door, most people experiencing family violence were referred to individual services (predominantly by police) and/or needed to navigate the service ‘system’ to find services to meet their needs. The role of Child FIRST has been to assess the needs of families and refer them to a service that can assist - it provides an entry point to an integrated local network of family services for children and families experiencing difficulties, which The Orange Door has replaced in areas where it has been established.

Figure 5: PwC explanation of client pathways prior to establishment of The Orange Door

Source: PwC summary and explanation adapted from FSV, Support and Safety Hubs: Statewide Concept, July 2017.

The Orange Door receives L17 reports, referrals from professionals (such as child protection practitioners) or the community, as well as provides an access point for self-referrals. As the key access point into the service system, The Orange Door assesses client needs and connects them to services that can provide appropriate interventions.

Importantly, people are able to access services directly if they are appropriate or desired, except for people referred via police L17 reports, who are referred to the relevant area.

Figure 6 provides a visual representation of the critical role The Orange Door performs in assisting clients to access the services they need.

Figure 6: Client pathways in The Orange Door

Source: Adapted from FSV, Support and Safety Hubs: Statewide Concept, July 2017.

These diagrams demonstrate the significant refocus of client pathways that The Orange Door represents in the service system. To this end, we would expect very significant investment in defining, establishing and implementing new processes and systems at almost every stage of the client pathway through the system. This evaluation found evidence that The Orange Door areas involved in the evaluation had spent their early months of establishment defining and developing these new processes.

The Orange Door is largely delivering against the suite of foundation services outlined in the service model, with some elements of the model stronger than others (for instance, CIP and crisis response by The Orange Door). A focus is now needed on other elements of the model including:

• Communications and engagement with the community to promote visibility of The Orange Door: Evidence from fieldwork indicated that practitioners reported that the role of The Orange Door was not clear to many parts of the sector and to clients. This was demonstrated by providers in the broader services system directing clients to The Orange Door whose needs could be better met elsewhere (e.g. by homelessness services). This is to be expected so early in the implementation of such an innovative model as The Orange Door, but focused effort is now needed to improve the communities understanding of the model.
• Assessment and planning for perpetrators: Evaluation evidence indicated that often the default response by The Orange Door for men who perpetrated violence was to direct them straight into behaviour change programs in most cases[2]. As a practitioner stated:

'If we’re not case managing these men, we’re not creating the cycle of change. There needs to be some case management like at [name of Aboriginal service] so that they can build somewhat of a relationship…instead of just offering them one phone call' – Practitioner interview

• Outreach: The service model and specifications outline the requirements for delivery of outreach and home visits. We found that discipline-based differences (stemming from previous experience and guidance from their employing agency as well as their practice philosophy) meant that this was being undertaken inconsistently – with family violence-focused practitioners often more reluctant to undertake home visits in these early stages of implementation. Practitioners reported that they need greater clarity about when outreach was suitable and should occur.[3]
• Review and monitoring of client experience: FSV and The Orange Door have made attempts to understand client experience during or following their service from The Orange Door, including client interviews as part of this evaluation. Uptake from clients to participate in interviews as part of this evaluation however, has not been high and the limited qualitative data collected from interviews represent a small sample size of client experience. As such, experience of the service model in the early stages of implementation by clients is unable to be confirmed without validation with clients which is an area of work that FSV is currently progressing.
• Documented feedback loops with external services: Formal feedback loops do not as yet exist between The Orange Door and external services. Qualitative evidence from fieldwork indicated that there were no consistent methods to seeking feedback from external services, nor to determining whether those services had provided satisfactory service after their service had been allocated to a client or a client was referred to them. In an interview, a client called for greater ‘exit planning’ to ensure that the service responses are appropriate – in this case, the client interpreted ‘exit planning’ as the process of referring them to an external service for supports.

'I think a lot of the time, and for me too, women are afraid to even ask for [help] because, and when you’ve been in an abusive relationship and particularly like me when you’ve been in one for a very long time you lose so much sense of yourself that to actually ask or to say that look I’m not really happy with this worker I need you to refer me somewhere else is perhaps something that you’re not going to do because you’re, you don’t even know how to stand up for yourself. You lose that sense of self … A bit of more follow up I think. If that’s at all able to be done would be good.' – Client Interview

• Practice level interfaces: Recognising the breadth of service interfaces that will contribute to the effective delivery of The Orange Door, FSV is taking a phased approach to developing statewide guidance.[4] For example, the interface specifies that The Orange Door and the Victims Support Agency (VSA) will have a close working relationship, and will develop protocols to support referrals, information sharing and where appropriate, joint-agency risk assessment, needs assessment and planning, particularly in instances where the ‘primary aggressor’ is incorrectly identified[5]. Evidence from fieldwork indicates that this does not appear to have occurred at this point in time. At a local level, The Orange Door practitioners perceive that accountability for ensuring interfaces are developed lies with the System Service Navigator. However, given that the VSA is a statewide agency this may be challenging to coordinate on a local level. Service interfaces will require a focus in the next twelve months to progress.

Referrals to The Orange Door

As specified in the service model, the referral sources/access to The Orange Door include direct contact (on-site, via phone or online or through outreach), L17 reports from Victoria Police, statewide after hours services, and professional referrals (from Child Protection, maternal child health services, legal services etc). While there is data to estimate demand for some parts of The Orange Door service offering, there is not sufficient data for a comprehensive estimate of the overall demand for The Orange Door. Each data source reflects caseload for a subset of potential clients for The Orange Door. This is to be expected given the innovation of The Orange Door bringing together clients who are victim survivors of family violence, family violence perpetrators and families and children who require support.

Demand forecasts and workforce planning was undertaken by DPC and FSV prior to establishment. These forecasts used data relating to the number of L17 referrals and referrals relating to child and family wellbeing in order to help determine resourcing in each area of The Orange Door. Given that The Orange Door is a new service model, this appears to have been a practical way to estimate referrals in the absence of an existing evidence base on consolidated demand.

The evaluation found that referrals to The Orange Door are largely generated from the expected sources. Baseline analysis undertaken by FSV[6] following the commencement of The Orange Door benchmarked referral sources against their equivalent referral types prior to the establishment of The Orange Door. This analysis indicated that L17 numbers stayed relatively stable pre and post-establishment, while referrals from other sources increased compared to the number of Child FIRST cases in the previous year. Overall, most of the demand for The Orange Door came through L17s and other professionals such as teachers and health workers at the time of the evaluation. Direct contact by clients seeking assistance (comprising calls to the 1800 numbers and walk-ins) are around a tenth of all contacts. Third party referrals from members of the public make up less than one per cent.[7]

More detailed data collection and longitudinal tracking will be required to identify any precise changes or trends in relation to referral sources over time.

Demand for The Orange Door

Prior to the formation of FSV, DPC completed work on the potential demand for family violence services following the RCFV. Once established, FSV commissioned Deloitte to undertake a demand-forecasting analysis.[8] Deloitte based demand forecasts on caseload targets (which employing agencies determined using their service use data), FTE's, and data modelling – including using previous reports and predicted growth patterns. The analysis focused on service provision prior to establishment of The Orange Door and the government used this information to inform funding for community service organisations. Community service organisations then individually determined their practitioner workforce needs for The Orang Door.[9]

Box 1: Sources of demand for The Orange Door

Qualitative evaluation data from The Orange Door workforce indicates a perception that the volume of cases The Orange Door has serviced has been greater than anticipated (with the exception of Mallee).[10] We are unable to confirm the accuracy of this perception without access to prior quantitative forecasts. This is further complicated by a number of factors regarding the context of the early roll out of The Orange Door areas:

• Qualitative data from the evaluation suggests that some staff in The Orange Door are working with clients for a greater time than anticipated by FSV and the service model and specification documents,[11] meaning that cases are taking longer to progress through The Orange Door than practitioners believe they should. This appears to be primarily due to a lack of shared understanding amongst practitioners about what ‘intake and assessment’ means in practice and what types of activities they need to do and at what point service should cease from The Orange Door and be provided by an external service. Some practitioners reported that they would ‘hold’ a case if they were not confident that a client could be connected to an appropriate service (for example, if a support worker was absent) – this is however in line with the service model. Further discussion of active engagement and risk monitoring (often termed as ‘active holding’ by practitioners) is in Section 4.2.2.
• Practitioners and employing agencies reported some concerns about the capacity of external agencies to service referrals from The Orange Door. This creates a ‘bottleneck’ where The Orange Door does not allocate services or make referrals due to a perceived lack of capacity in the service system. Stakeholders interviewed, perceived that these capacity constraints are exacerbated by workforce shortages in the sector with the transition of skilled workers from these agencies into The Orange Door. The capacity of the service sector to respond to referrals from The Orange Door was not within scope for assessment as part of this evaluation but should be a key consideration of FSV and partner agencies in future evaluations of The Orange Door.
• Qualitative data from the evaluation suggests that some cases take longer to progress than others at this point in time. A number of practice leaders and practitioners reported that while family violence-focused cases could, in theory, be delivered quickly (that is, ‘screening, identification and triage’, ‘assessment and planning’ and ‘connection to service’ within a few weeks),[12] child wellbeing-focused cases are inherently longer to assess than family violence victim survivor cases and require more detailed and different assessments, including a home visit. The resourcing differences, which lead to these cases remaining in The Orange Door for longer, include the need to create profiles for all family members and the greater number of required checks.[13] Practitioners at some Orange Door areas also suggested that there is a much broader range of services and interventions associated with child wellbeing cases, which can be difficult to find and connect clients to, particularly in areas that service a large population or geographic area.[14] Current default data within CRM is unable to accurately verify the length of time cases are ‘held’ at The Orange Door at this stage; therefore, evaluation findings were not able to be quantitatively verified as to the different length of time different cases remained within The Orange Door.
• The number of workers in The Orange Door in the initial months following commencement was lower than the desired FTE levels, particularly in Barwon and BPA. The staffing vacancies in the early months of operations (particularly leading into the high demand time of Christmas) likely contributed to backlogs in case processing. Combined with issues of accessing and using the CRM, this reinforced the view among practitioners that demand was much higher than anticipated.
• Most practitioners who came into The Orange Door from Child FIRST brought with them an existing caseload, whereas those who came from family violence agencies waited for new cases to come in through the L17 portal. This has created differences in workload between practice disciplines at the outset. However, this cannot be verified by quantitative data at this point in time.

Efficiencies in processing some referrals through the L17 portal were not achieved in the early months of operations. At the same time that L17s went ‘live’ for The Orange Door, practitioners and team leaders reported needing to spend time on developing standard operating processes and procedures before L17 referrals could be responded to appropriately. This necessarily led to an immediate ‘backlog’ of referrals that were unable to be processed with the same efficiencies as experienced prior to The Orange Door. It is probable that this would not be experienced to the same degree when future areas of The Orange Door are established as these areas will be able to leverage existing processes developed by the first four areas to achieve efficiency relatively quickly.

Box 2: Quantitative data on demand

Demand for The Orange Door is primarily represented through the number of referrals and number of cases created for clients. Cases represent the number of incidences; that is one client can have multiple cases in any given period (but not at once). As shown in Figure 7, the number of referrals screened in The Orange Door areas in BPA and NEMA has been considerably larger relative to their larger catchment populations and trending higher than in the Barwon and Mallee areas. This is particularly pronounced in NEMA, where The Orange Door commenced operation in July 2018.

Figure 7: Total screenings (referrals) recorded on the CRM by The Orange Door area, July – December 2018[15] [16]

Source: PwC analysis of CRM data provided by FSV. Note that The Orange Door commenced in Barwon, BPA and Mallee in May 2018 and in NEMA in July 2018.

Screening, identification and triage

Screening, identification and triage in The Orange Door determine whether an immediate emergency response is required and/or if The Orange Door can provide the most appropriate service response. The process considers the following questions:

• Is there an immediate risk to safety for any family member?
• Is there or has there been family violence?
• Are there concerns about a child’s safety, wellbeing or development?
• Who are the people associated with the referral or contact?
• What is the priority of action needed?

In addition, the screening process includes entering referrals into the CRM, conducting history checks, identifying family members and associated people to the case or referral, with particular emphasis on discovering whether there are any children or young people involved (especially for L17 referrals). Where associated people are identified, the screening questions are applied to consider their needs individually.

Triage helps to determine the priority and urgency of The Orange Door’s response to the person, and results in internal assignment of the client(s) to a practitioner within The Orange Door.

It is important to note that each area of The Orange Door undertakes screening, identification and triage slightly differently. This may reflect a need for local flexibility or the absence of clear direction about statewide standardised processes however we have been unable to verify the underlying reason and it may be too early to determine this. Table 3 describes the processes for screening, identification and triage at each Orange Door area.

Table 3: Summary of typical processes in each Orange Door areas as of end 2018

Source: Information provided by The Orange Door workforce in fieldwork. Note that Barwon and Mallee have similar intake structures so have been grouped together, as do BPA and NEMA.

Evidence from fieldwork indicated that assessment of some cases was delayed for months at a time, especially where demand was high or where a backlog existed. This was exacerbated by the prioritisation of self-referrals which led to third-party referrals being continually 'overtaken in the queue' in the very early stages of implementation.[17] This was verified through a client interview (below), however it is important to note that initial interview information from clients would indicate clients have limited (if any) understanding of the different stages of The Orange Door process. This actually reflects a seamless service from the perspective of the client and is a positive early finding of The Orange Door experience. This could be further explored through FSV’s current program of work on client experience and feedback.

'The hardest thing that’s happened is I’m still under the intake team [after around four months], waiting to progress on to the next service that I’m supposed to go to. So, like I’m still dealing with the original person that I dealt with who kept saying: ‘I’m only the temporary worker, I’m only the temporary worker.’ And, which I’m so lucky, she’s fantastic. But I think that when you’re like in this sort of situation and like this is the first time I’ve actually felt safe and settled that sometimes you want a bit, you know a bit more solid thing. Like this is my worker. I’m going to be able to tell her this stuff and I’m not going to have to repeat it again in a month or so when I finally get a worker type thing. But she’s been really good, and she’s been all over everything …. It’s just that next step to more permanent. To help you settle in yourself'. – Client Interview

'Don’t take forever. I mean my situation is not urgent, but it might be different if there’s women out there who need like urgent help. Just don’t f*** around on the poor lady. [Provided specifics about delays in housing and legal processes.] Just like get things done for families in general.' – Client Interview

Despite the time delays experienced by some clients, even in this early stage of implementation there are indications that most clients are effectively screened, triaged and provided with a response to their immediate risks and needs.

Apart from the number of screenings recorded on CRM, quantitative data on the screening, identification and triage stage of the client pathway is not readily available at present but would be expected to become available as the CRM matures in function.

Key finding 12:

The Orange Door offers all functions required to operate and provide service to clients. While to be expected at this very early stage of implementation, some of these functions require refinement and/or standardisation to work effectively and efficiently for all clients. However, early data from clients indicate that some are experiencing ‘seamless’ services while they transition between functions within The Orange Door

Key finding 13:

The ‘review and monitoring’ function articulated in the service model is not established yet and will mature over time as The Orange Door rolls out across the state

Key finding 14:

The ‘connection to services’ and ‘service response’ functions in the service model are not yet fully matured in The Orange Door areas. This is to be expected at this early stage of implementation

Key finding 15:

There are distinct differences in the way ‘walk-in’ and phone referrals enter The Orange Door compared to L17s, emails and Child Protection referrals. Evaluative judgements on the impact of these differences on service quality and client experiences are unable to be made at this point in time

Key finding 16:

A combination of high demand, staffing vacancies and a lack of clarity by the The Orange Door workforce about how to operationalise appropriate screening, identification and triage processes, resulted in some clients experiencing significant wait times for support and referrals in the early stages of implementation. This will need to be monitored to determine if this is a point in time or systemic issue

To what extent are clients connected to the right services at the right time?[18]

Service responses are tailored to need and risk

The Orange Door is intended to work with clients to tailor their safety plans and/or risk management plans, provide targeted interventions and connection to services based on individual needs, as identified through screening, triage, identification and assessment.[19]

The two primary determinants of whether service responses are tailored to need and risk are:

1. Whether the risk and needs assessments have been undertaken effectively (see Section 4.3) to inform tailored responses.

Evaluation evidence indicates that risk and needs assessments appear to be undertaken effectively within The Orange Door. The majority of The Orange Door workforce appear to have sufficient experience to recognise risks to their clients and what they need to manage those risks. Please refer to Section 4.3 for a more in-depth discussion of risk assessments.

Given the limited data on risk and needs assessment recorded on the CRM at this point in time, it is difficult to make more in-depth conclusions about the quality of risk assessments and if responses to these assessments are effectively tailored to meet the needs of clients.

2. The capacity of the service system to provide tailored responses.

The Orange Door workforce has limited awareness of the service system apart from individual professional knowledge from prior work experience. Automated systems are not yet in place for practitioners to determine capacity or waiting times for services to which they may wish to refer – meaning practitioners are often reliant on a range of, often time consuming processes that have been developed with core agencies to support allocations. This limits practitioner’s ability to ensure that the support provided is tailored to an individual client’s risk and needs. As detailed in Section 4.1.1, the review and monitoring aspect of The Orange Door model and the collaboration and connections with the broader sector system is still being established and would not reasonably be expected to be established at this early stage of the statewide The Orange Door roll out.

Clients interviewed for this evaluation raised concerns with the appropriateness of service responses, as the following anecdotes highlight:

• A client reported feeling unsure about whether they were able to provide feedback to The Orange Door if their relationship with their case manager was not providing what they needed.
• One client reported that the service response represented 'tinkering around the edges' rather than addressing fundamental factors in their situation.
• One client did not receive support from the service they were referred to, even after following up with The Orange Door practitioner they were working with.
• One client reported that they believed The Orange Door workers were not necessarily aware of the services available in the broader system.

As with risk and needs assessments, data recorded on the CRM relating to connection to services is limited at this point in time. However, available data recorded on the CRM does indicate that around half the cases in The Orange Door areas are closed after the client had engaged with the service system or had their needs met.[20] This is a positive outcome early in the roll out of The Orange Door.

There are however, a significant proportion of Orange Door clients whose cases are closed without them having accessed services or received support from The Orange Door. For example, around ten per cent of clients refuse services and a large proportion in each area are not contactable (however we do not have any further details about these clients at this point in time). The closure of cases where contact has not been made may or may not be appropriate for any given case. No evaluative judgement is made here relating to the appropriateness of case closures.

Similarly, data from client interviews as part of this evaluation were limited and may be influenced by negative bias (when people are more likely to recall the things they didn’t like in comparison to what they did like). These factors make it difficult to draw firm evaluation conclusions about whether service responses are being tailored to need and risk at this early stage of The Orange Door roll out.

Active management and support

Following assessment, allocations of services to clients or referrals to services can occur immediately. However, if this is not possible or appropriate,[21] active engagement and risk monitoring (often termed ‘active holding’) responses must be put in place.[22]

The line between limited interventions as part of the assessment, active engagement and risk monitoring stages and case management is often blurred in practice. At a broader level, a number of FSV staff and The Orange Door workforce reported uncertainty about whether practitioners were undertaking more active engagement and risk monitoring (active holding) than is intended in The Orange Door model. Practitioners gave a range of reasons why this lack of certainty might be the case:

• Many practitioners reflected that they had been hired from roles where they undertook case management and it is their established practice to support clients through difficult periods. In addition, they reported that their position descriptions and expectations set by employing agencies did not always reflect the intake and assessment role of The Orange Door and they had understood their roles would have more substantive case management responsibilities.
• Confusion about the role of The Orange Door practitioner in the overall service to the client. For example, some practitioners reported that they understood that the service model called for case managers in external services to be the primary conduit with schools for a client’s case, however, they were criticised by these external case managers if The Orange Door had not already contacted the school as part of the assessment phase. The service model does not state that this needs to occur, but clearly there is a misalignment in the expectation of external case managers and the role of The Orange Door in these cases.
• Some practitioners reported that it was difficult not to become emotionally invested in certain cases, particularly where the victim may be at immediate risk of harm, and they were inclined to use any resources available to assist. This lengthened practitioners engagement with the client beyond immediate assessment, planning and rapid referral.
• Some clients develop a personal bond for the first person they engaged with for assistance, which is often The Orange Door practitioner. One practitioner described a client calling her months after she first entered The Orange Door and asking for help or advice. While there is deliberate flexibility built into the model to accommodate client preferences, it was unclear to practitioners at this early stage, how they could make decisions about which aspect is appropriate to vary for particular client needs.

In very few cases did practitioners mention that their active engagement and risk monitoring (‘active holding’) was coordinated in collaboration with core services. In most cases, it appeared that they were supporting clients using their own knowledge and expertise.[23]

Interviews with clients revealed mixed attitudes towards greater involvement by The Orange Door in their cases:

• One client wanted The Orange Door to keep their case open in order to be able to follow-up to ensure the service response provided was appropriate and satisfactory.
• Client interviews revealed a desire by clients for more assistance to both manage and remain alert to the various support options available:

'The coordination….like if Orange Door could sort of oversee bits, to make sure things are being done. Like following up. And letting you know everything that actually is available out there ….'[24]. – Client Interview

One client felt that the scope of The Orange Door was too narrow:

'I like to think of an iceberg. You know, their initial tip is like really good. And you know you’re sort of balancing up on this tip and you’re like working it all out, doing it and it’s supportive and then, but when it gets harder as you go along, which is usually the case when you’ve got like in-depth trauma and all stuff the family law issues and everything else at the bottom of the iceberg is so much bigger. And all you’re doing is just rocking that iceberg. You’re not chiselling at it. And I think that, that’s what if Orange Door is going to help you with that top bit I think it needs to continue down at least sort of to the water line... And like I know there’s so many women and so many families and the men that go through this it’s going to be difficult to spread it around, especially for such a new service, but I think that looking at the mission and thing of Orange Door like I hope going forward that it’s going to be that thing that can make you chip that iceberg down. You know, not just leave you floundering like everybody else does.'[25] – Client Interview

Evaluation data from both practitioners and clients highlight that understanding of The Orange Door and its role is only just emerging within the community. This is to be expected this early in the implementation and roll out of The Orange Door. Clearer messaging at the intake stage may be required to establish client expectations regarding the role of The Orange Door.

Key finding 17:

The capacity of the system to provide tailored service responses is limited by The Orange Door workers’ knowledge of services in the sector but will be enhanced once the ‘review and monitoring’ function is fully developed

Key finding 18:

Data limitations mean that we are unable to make firm evaluative judgements about whether service responses are tailored to need and risk at this point in time

Key finding 19:

Clients who are not able to be immediately allocated or referred to services are provided with active management and support

Key finding 20:

The Orange Door workers would benefit from clearer guidance using examples from practice about where their role in screening, intake and assessment ends, including ‘active holding’, and where case management begins

Key finding 21:

Many of the (limited sample of) clients interviewed as part of the evaluation expressed a desire for greater support from The Orange Door, even after services had been allocated to them, or they had been referred out to services

To what extent is The Orange Door effective in assessment and management of risk and need?

Risk and needs assessments

Risk assessment in The Orange Door aims to understand any child wellbeing concerns, family violence risk to all family members and the risk posed by the perpetrator. The Statewide Concept envisages that The Orange Door workers use consistent processes and tools to assess risk.

In practice and at this point in time in the early operations of The Orange Door, consistent approaches to risk assessments are only emerging. The evaluation found:

• Individual practitioners reported that they used family violence risk assessment in different ways. Some found the risk assessment structure helpful while others considered them mainly as extra paperwork. Evidence from fieldwork indicated that some practitioners do not upload all their risk assessments to the CRM (either as a scanned copy or by inputting information to TRAM) at this point in time.
• Each of The Orange Door areas appears to have produced its own template for child and family wellbeing risk assessments, apparently due to the need to shorten the standard template. Some practitioners also stated that they found the Best Interests Assessment (BIA) used in their area was not well structured.[26] This means that risk assessments of child wellbeing concerns may be undertaken using different considerations and focusing on different factors in each area (which is consistent with what was occurring prior to the establishment of The Orange Door).

There appeared to be inconsistency about whether a multidisciplinary lens was applied to risk assessments. One practitioner stated that:

'…a lot of practitioners don’t understand the risk assessments coming out of the triage team…they often look at it from a family violence or child wellbeing perspective…not from a combined perspective'[27]. – Practitioner interview

There was some confusion about when the RAMP should become involved and where it should sit (that is, within The Orange Door structure or outside it). An example of this was the description of the screening meeting that occurred each morning in Mallee (attended by team leaders and practice leaders) as a 'mini-RAMP' where multidisciplinary risk assessment occurred.

Figure 8 describes the numbers of risk assessments (specifically the Interim Comprehensive Risk Assessment Tool (ICRAT) undertaken in July and August 2018. ICRAT was updated and renamed TRAM so the figure also shows TRAM following its introduction in September 2018). These started relatively high at the start of 2018-19 and fell in subsequent months, with the exception of Barwon, which had the opposite trend. While there is not a clear explanation for this, it may be because practitioners were not recording risk assessments in CRM, were not using the tool, or were finding it challenging to implement the data collection methods required.

Figure 8: Total number of risk assessments by The Orange Door area, July – December 2018

Source: PwC analysis of CRM data provided by FSV. Note that The Orange Door commenced in NEMA in July 2018. Note that ICRAT was replaced by TRAM in late-August 2019. ICRAT included four assessment types: Adult Victim Survivor, Child Victim Survivor, Adult Perpetrator, and Adolescents Who Use Violence. TRAM includes two assessment types: Adult Comprehensive Tools and Child Comprehensive Tools. The total shown in the figure above is the total of all types of ICRAT/TRAM risk assessments.

One way to gauge the level of risk faced by clients of The Orange Door is to use the categories assigned to them in risk assessments reported on the CRM. As shown in Figure 9, in all areas except for Mallee, around a third of risk assessments are classed as ‘elevated risk’, and at least five per cent are classed as ‘requires immediate protection’. In Mallee, these proportions are lower. Given that around a third of risk assessments are kept ‘blank’ in the CRM in order to finalise the assessment, it is difficult to determine whether this is due to practice differences or poor data recording at this early stage of implementation.

Figure 9: Level of risk identified, October – December 2018

Source: PwC analysis of CRM data provided by FSV. Note that as around a third of risk assessments are recorded on CRM as ‘(blank)’, these proportions may not be an accurate reflection of actual risk as assessed by practitioners.

The use of CIP in risk assessments is detailed in Section 5.3.1.

Client choice

Ensuring choice and control is a fundamental part of The Orange Door model – and the family violence reforms more broadly – to put people at the centre and in control of their lives. The concept predicates that The Orange Door will provide accurate information and support to enable people to make informed choices about their own needs and priorities and decide what they need to be safe and to recover. The Orange Door, as outlined in the Statewide Concept, is designed to specifically support the choice and self-determination of Aboriginal people, including referrals to an Aboriginal service or support by an Aboriginal worker.[28]

Within The Orange Door, evidence from fieldwork indicates that there appeared to be a strong commitment to a client-centric approach. However, practitioners typically discussed how they took client preferences and choice into account during the intake and ‘connection to services’ stages rather than the assessment and planning stage. During the assessment and planning stage:

• There was a greater focus on acquiring information to inform risk assessments, for which practitioners were more directed by the TRAM templates than client choice. Nonetheless, client voices were an important part of safety planning, as practitioners needed to determine what was realistic for the victim-survivor or family.[29]
• Aboriginal clients’ voices appeared to take a stronger role during the assessment and planning phase, with practitioners and Aboriginal workers in particular recounting how the support of the community[30] was an important factor to consider in risk assessment and planning.
• Practitioners reported that it is relatively easy to deliver on immediate client requests and choices in crisis situations involving family violence. Responses to requests for lock changes or installation of security equipment appeared to occur relatively fast and practitioners were able to give many examples of immediate support like this. In general, practitioners always followed requests by victim-survivors not to contact perpetrators, however, evidence from fieldwork indicated that some practitioners had been directed by their employing agency not to wait for a victim-survivor to give their permission before contacting the perpetrator.[31] There is some variability across areas in relation to gaining the approval of the victim survivor ahead of contact being made with the perpetrators. This may reflect different practices within the sector and may require further consideration to support the development of a consistent approach. There was insufficient data from the evaluation to determine if this is a widespread practice in The Orange Door.
• One area where practitioners reported that they found it challenging to take clients choices and preferences into account was in relation to non-family violence child wellbeing cases – particularly where there may be the potential for Child Protection involvement. In contrast to family violence cases where an adult (most frequently a woman) is usually a victim-survivor, in child wellbeing cases, no adults may be victim-survivors of family violence. While child wellbeing-focused practitioners are highly skilled at engaging with children, clients under 18 years of age have less legal and financial agency to make their own choices.

Feedback from client interviews indicated that while clients did not distinguish between assessment, planning by The Orange Door and connection to external services, some expected ongoing engagement with the practitioner at The Orange Door:

One client was satisfied with the assistance that they received, but wished The Orange Door practitioner was able to ‘go deeper’ into the fundamental issues causing them difficulty.

A client recounted how The Orange Door consulted with them before reaching out to their violent partner and kept them informed throughout the engagement.

Clients expected The Orange Door workers to be proactive about suggesting potential solutions (such as service responses) and supporting choices about services and solutions.

While data is limited it is a very positive early finding that some clients are experiencing ‘seamless’ services within The Orange Door and have a positive relationship with The Orange Door workers. Further exploration is needed to better understand if and how client choice is incorporated into assessment and planning functions at The Orange Door.

Key finding 22: Qualitative data from fieldwork indicates that risk and needs assessments appear to be undertaken somewhat inconsistently within The Orange Door. Consistency in recording of risk assessments require monitoring by FSV to ensure that quality and consistency of data recording continues to grow as service delivery continues and the service model evolves. Validation through client feedback is required to determine if risk and needs assessments lead to responses that are tailored to the needs of the client

Key finding 23: The Orange Door practitioners have access to a range of tools to assist them to assess risk. Most of the family violence specific tools are standardised and in early stages of embedding in practice. The child wellbeing tools are not yet standardised (nor was there a standardised tool prior to the establishment of The Orange Door)

Key finding 24: The Orange Door practitioners report a strong commitment to a client-centric approach to practice

Key finding 25: There is some evidence that at these early stages, the voices of Aboriginal clients play a strong role in service choice and in determining the support provided to them

Key finding 26: Client preferences and choice appear more likely to be taken into account in relation to referral rather than assessment and planning, and more likely for adult clients of The Orange Door

[1] FSV, ‘Support and Safety Hubs: Statewide concept’, July 2017.

[2] The FSV Perpetrator Practice Guidance provides a range of service responses including crisis support and referral to MBC or men’s case management as options for responding to perpetrator risk.

[3] Qualitative evaluation data from fieldwork

[4] Current interface guidance has been developed for the interface between Magistrates Court of Victoria, Victim Support Agency, Child Protection and Integrated Family Services, Men’s Referral Services, RAMP, Victoria Police and Sexual Assault services.

[5] FSV, Statewide concept: service model, April 2018.

[6] Baseline Analysis conducted by FSV, December 2018.

[7] PwC analysis based on CRM data for July – December 2018 provided by FSV in January 2019.

[8] These forecasts were not provided to PwC to inform the evaluation. As such, no conclusions on the appropriateness of demand forecasts or method of calculation have been made in this report.

[9] Information provided by FSV, January 2019.

[10] Mallee experienced steady demand for its services following commencement, however, due to the higher FTE staffing at this time, practitioners reported that they perceived they were better able to deal with this caseload.

[11] Information provided by FSV, November 2018.

[12] Practitioners and FSV staff refer to these functions as intake, assessment and referral out.

[13] Qualitative evidence from fieldwork

[14] Practitioners noted the array of interventions and services that could be appropriate for child wellbeing cases, such as playgroups, maternal and child wellbeing, etc. which can be difficult to keep a current view of.

[15] CRM data for Barwon, BPA and Mallee in May and June are not shown as data collection and validation in these periods was inconsistent among The Orange Door areas.

[16] CRM data used throughout this report was extracted progressively from October 2018 to January 2019.

[17] Qualitative evaluation data from fieldwork

[18] Aligns with the Family Violence Outcomes Framework system domains. Indicators to inform this domain may be refined based on findings/available data from this developmental evaluation of The Orange Door

[19] FSV, ‘Support and Safety Hubs: Service model’, April 2018.

[20] PwC analysis of CRM data for October – December 2018 provided by FSV, January 2019.

[21] Based on qualitative evaluation evidence from fieldwork, this primarily appears to be where service capacity is not available, or the appropriate response is not yet clear.

[22] FSV, ‘Support and Safety Hubs: Service model’, April 2018.

[23] Qualitative evaluation evidence from fieldwork

[24] Qualitative evaluation evidence from fieldwork

[25] Information provided by a client during fieldwork

[26] Qualitative evaluation data from fieldwork

[27] Qualitative evaluation data from fieldwork

[28] FSV, ‘Support and Safety Hubs: Statewide concept’, July 2017.

[29] Information provided by The Orange Door workers during fieldwork.

[30] Community in this instance refers to community from an Aboriginal service system point of view

[31] Practitioners who work with perpetrators stated that at the commencement of The Orange Door, they had waited for the family violence-focused worker to inform them that the victim-survivor agreed that their partner could be contacted, however, after this policy resulted in a sharp reduction in referrals to men’s behaviour change programs, it was agreed that the respondent on an L17 could be contacted immediately. Refer to the Interim Integrated Practice Framework and Perpetrator Practice Guidance for further discussion on the sharing of information related to risk.