The Family Violence and Sexual Assault Workforce Pulse Survey was a survey of family violence and sexual assault specialists working in funded agencies and The Orange Door and included child wellbeing specialists at The Orange Door. It was conducted between September and October 2022.
Compared to the Family Violence Workforce Censuses conducted in 2017 and 2019,[21] the Family Violence and Sexual Assault Workforce Pulse Survey covered a narrower set of questions and did not extend to prevention or broader non-specialist workforces. It was also the first time a specialist workforce survey was conducted directly by Family Safety Victoria.
Similar to prior Censuses, the survey utilised a non-probability convenience sampling approach. A sampling frame based on type of employer was developed to better define the target population and track survey performance.
Target population
Understanding a survey’s target population can be important for a range of reasons. Defining the actual or probable size of the target population means a sampling method can be developed and/or the performance of a survey can be tracked according to sample size. For this survey, the target population was defined to include any worker in Victoria who provided services to clients through a:
- specialist family violence services (victim survivors) organisation
- sexual assault support services organisation
- Aboriginal Community Controlled Organisation
- Family Violence Perpetrator Interventions organisation (including specialist family violence services for adults using violence and Men's Behaviour Change Program providers)
- specialist services through The Orange Door network, including child wellbeing specialists.
The target population was defined to exclude workers employed:
- by a peak body, such as Safe and Equal
- from the Primary Prevention workforce
- to deliver administration services at a family violence or sexual assault organisation
- by Victoria Police, the Victorian Courts System or Corrections Victoria
- at Family Safety Victoria
- in a primary human services role that did not have a family violence or sexual assault intervention and/or support focus (e.g. employed by a homelessness service).
Family Safety Victoria used internal data to identify a headcount of workers employed by TOD network. Beyond TOD network, Family Safety Victoria used a formula to estimate the family violence and sexual assault workforces employed across non-government agencies based on funding allocated by the Victorian government to funded agencies in the 2021-22 financial year. This formula generated a headcount estimate, that was further qualified by approximate administration costs and vacancy rates.
Furthermore, the projected headcount for workers at funded agencies was reviewed against historical agency workforce data, as collected by the Australian Charities and Not-for-Profit Commission (ACNC) and Workplace Gender Equality Agency (WGEA).[22]
This process provided Family Safety Victoria a basis to track the performance of different target sub-populations and target communications while the survey was in-field. An example of its use is demonstrated below in the segmentation of the sample as a proportion of estimated target subpopulations (Table 36).
Table 36: Survey response segmented by estimated subpopulations
| Sector | Survey sample | Estimated population | % Estimated population |
| TOD network, including child wellbeing specialists | 289 | 1,028 | 28% |
| SFVS organisation | 426 | 1,079 | 40% |
| SASS organisation | 162 | 347 | 47% |
| FVPI organisation | 110 | 277 | 40% |
| ACCO | 62 | 272 | 23% |
| Grand total | 1,049 | 3,003 | 35% |
Source. WP Survey 2022
Response rate
At the close of the survey on 21 October 2022, 1,681 users had accessed the survey, with a completion rate of 62.4%. The raw data was cleaned to deidentify open text responses and improve data quality.
The final survey sample is 1,049 responses, representing a response rate of approximately 35% of the target population.[23] The final sample is composed of a combination of total and partial completions. Partial completions were included if 80% or more of the survey was completed. We note that due to the nature of the collection method, the response rate does not precisely reflect the workforce actually reached.
The age distribution for the survey sample is portrayed below (Figure 1).
Figure 1: Population pyramid for the Workforce Pulse Survey 2022
Validity assessment
While response rate is commonly considered the most important measure of a survey’s validity, research demonstrates there are conceptual and functional limitations for using response rate as an exclusive benchmark, and sample validation has emerged as relevant to understanding whether a survey sample is high quality, appropriate and representative.[25]
An alternative to focusing on whether a response rate is either high or low is to assess whether the responses are ‘functional’ or ‘dysfunctional’. The WP Survey, while not subject to random sampling, applied a number of measures to assess sample quality including clear eligibility questions within the survey, an extensive data cleaning process and data quality checks post-survey.
In order to assess sampling validity, Family Safety Victoria also undertook data quality checks by comparing characteristics of the survey sample to a workforce proxy derived from the 2021 ABS Census[26] and population data available from other comparable workforce surveys.[27]
On the basis of age distribution, the sample was broadly consistent with comparable population surveys or data, although a concentration of younger workers were reflected in the 2022 Workforce Pulse Survey sample (Figure 2).
Figure 2: Age distribution of various workforce surveys
On the basis of geographic distribution, the sample reflects a diverse range of geographic locations for service delivery. In particular, the survey was well-represented by workers in regional Victoria, with 38% of the sample delivering services in a regional Local Government Area.[28] The sample distribution on the basis of the region where services were delivered also reflected a very high level of diversity, with a Simpsons Diversity Index score of 0.92.[29]
Based on distribution across employers, the sample similarly reflected a high level of diversity. While respondents in the sample did not identify coming from almost one quarter of funded organisations, the 738 individuals who identified where they worked were employed across 101 organisations, providing a very high Simpsons Diversity Index score of 0.98.
We caution there are certain areas where a true enumeration of the workforce might contribute stronger insights into the limitations in sample quality. For instance, analysis based on 2019 ACNC data for a proportion of funded agencies indicates there may potentially be an under-representation of part time and casual workers.
While this analysis does not lead Family Safety Victoria to make a categorical claim about the representativeness of the sample, the process provides greater assurance that the survey methodology was appropriately targeted at the family violence and sexual assault workforce, that there was limited sampling frame error, and therefore that both the survey methodology and findings are valid.
Communications and survey tracking
A communications strategy was developed to focus communications across the target population while the survey was in field. This strategy focused on contact with segmented populations through employers and peak bodies, through social media, as well as direct engagement with some of the target population through internal email lists and communications channels.
The communications strategy enabled the survey team to track and analyse in situ the impact of individual communications tactics, by reviewing on a periodic basis survey completions and relevant landing page views. It also enabled the survey team to modify and test appropriate language for the target audience.
Our broad observation is that a survey campaign over a period of time is required to build the target sample, given that the family violence and sexual assault workforces appear time poor.
Furthermore, periodic direct engagement via Family Safety Victoria mailing lists had the largest measurable impact upon the response rate and represented the most effective method of access to the target population within this campaign. This mode of contact was enhanced by engagement from the peak bodies and other sector intermediaries. In future surveys, alternative methods of direct engagement with the target population planned in advance of the survey’s delivery will likely have the largest effect on increasing the response rate.
Notes
[21] Family Safety Victoria, Family violence workforce census webpage, State Government of Victoria, accessed 15 February 2023.
[22] Australian Charities and Not-for-profits Commission, ACNC registered charities database webpage and Workforce Gender Equality Agency, WGEA data webpage, Australian Government data.gov.au, accessed 1 August 2022.
[23] Given the difference between the number of people who accessed the survey and the final sample size, future surveys may include further questions following the eligibility question to provide a clearer understanding of those who commenced but did not proceed through the survey.
[24] The self-described category included the following self-identified gender identities: non-binary, genderqueer, trans, agender, gender fluid and no specification.
[25] H Brooks Y Baruch, H Aguinis and G Ballinger (2002) “Survey response rates: Trends and a validity assessment framework”, Human Relations, Vol. 75, Issue. 8, pgs 1423-1643.
[26] Noting that there are no specific occupational (ANZSCO) or industry (ANZSIC) codes that identfy workers involved in family violence and sexual assault service delivery, a cross-tabulation of codes at the 4 and 6 digit level were used to reflect the breadth of relevant job roles or occupations specific to the target population for the WP Survey (for example, by excluding aged and disability carers, general administrative roles, registered nurses, etc). The 4-digit ANZSIC classes used were Other Health Care Services nec (8599); Other Allied Health Services (8539); Other Social Assistance Services (8790) and Religious Services (9540). The 6-digit ANZSCO occupations were Chief Executive or Managing Director (11111), Policy and Planning Manager (132411), Health and Welfare Services Managers nec (134299), Counsellors nfd (272100), Drug and Alcohol Counsellor (272112), Clinical Psychologists (272311), Social Worker (272511), Welfare Worker (272613), Community Worker (411711), Youth Worker (411716), Family Support Worker (411713), Program or Project Administrator (511112). This established a broad population proxy of approximately 18,000 workers in Victoria.
[27] UNSW Social Policy Research Centre, National survey of domestic violence and sexual assault workforces webpage, UNSW, accessed 1 August 2022; Victorian Public Sector Commission, Employee demographics and diversity webpage, Victorian Public Sector Commission, accessed 19 December 2022.
[28] This is compared to 33% who lived in a regional VIC Local Government Area according to the 2021 ABS Census.
[29] The Simpson’s Diversity Index is a way to measure the diversity of species in a community, with a value range between 0 and 1. Guajardo suggests a score range between 0.81 and 0.99 indicates a high degree diversity (heterogeneity). See SA Guajardo (2015) “Measuring Diversity in Police Agencies”, Journal of Ethnicity in Criminal Justice, Vol. 13, Issue 1, pgs 1-15.
Updated