How representative are Community Views results?

Community Views is designed to build a sufficiently large, diverse and balanced evidence base through sample design, targeted recruitment and weighting, rather than assuming that a large response count alone makes results representative. This article explains how that works and what it means when you read your results, whether you are answering a question in a briefing or deciding how much weight to put on a finding.

Key concepts

Sample size and representativeness are different questions

There are two different questions behind 'can this survey speak for the whole community?': whether enough people have participated, and whether the people who participated adequately reflect the broader community. You do not need to survey everybody to obtain a reliable picture of a population. Once a sample reaches a reasonable size, its reliability depends much more on the size and composition of the sample than on what proportion of the total population has been surveyed. That is why national opinion polls can tell us something meaningful about millions of Australians from samples of only a few thousand people. Increasing a sample from, say, 2,000 to 20,000 improves statistical precision, but nowhere near tenfold.

Sample design and recruitment

At .id, we work with each commissioning organisation to set minimum sample size goals for each local area (your neighbourhood, ward or community of interest), so that results support place-based decisions. Local areas are aggregates of suburbs, and the size of the estimated resident population is considered so the target sample is feasible. For example, a study with five local areas might aim for a minimum of 200 responses in each and an overall sample of 1,000. Respondents are recruited through social media advertising on Facebook and Instagram, and through the organisation's own channels where it chooses to use them. Digital recruitment is geographically targeted and designed to reach residents with different demographic characteristics, rather than relying only on people who are already engaged.

Weighting

Responses are typically weighted by age, gender and local area, decided case by case for each study. Weighting targets are based on Australian Bureau of Statistics population data for the geographical area in scope. We monitor the composition of the responding sample and use weighting where required, so groups that are over-represented or under-represented do not have a disproportionate influence on the overall results.

Margin of error

The margin of error depends on the sample size of the particular study. For example, for a sample of 1,300 responses, the maximum margin of error is +/-2.7% at the 95% confidence interval: if 50% of respondents gave a particular answer, we could be 95% confident that between 47.3% and 52.7% of the adult population in that area would hold the same view. These figures are an illustration, not a description of any one study.

Self-selection

Like most community engagement, Community Views involves an element of self-selection: people choose whether to participate. No community survey can completely eliminate non-response or self-selection bias. The methodology is designed to reduce that risk through broad recruitment, monitoring of sample composition, weighting where required and analysis of results by geography and demographic group, because an overall result can otherwise conceal quite different experiences among younger and older residents, renters and homeowners, different neighbourhoods and other parts of the community.

Minimum samples and significance testing

The minimum reportable base for main charts on the Community Views platform is 50 respondents, so results are not reported from very small groups. Significance testing at the p<0.05 level is shown with up and down arrows, which tells you whether two results are meaningfully different: a difference without an arrow falls within the survey's margin of error and should be read as broadly similar. Results can also be explored by local area and demographic group, so you can check whether an overall result conceals different experiences across the community before you act on it.

Frequently asked questions

Q: Can a few thousand responses really speak for the whole community?

A: Yes, when the sample is well designed. Reliability depends much more on the size and composition of the sample than on the proportion of the population surveyed, which is why national opinion polls draw meaningful conclusions about millions of Australians from a few thousand responses. The more important question is whether the sample reflects the community, which is what sample design, recruitment, weighting and subgroup analysis work together to address.

Q: Does everyone in the community get surveyed?

A: No, and they do not need to be. Minimum sample size goals are set for each local area (your neighbourhood, ward or community of interest) so results support place-based decisions, and respondents are recruited through geographically targeted social media advertising and the organisation's own channels.

Q: What happens if one group answers more than others?

A: The composition of the responding sample is monitored, and responses are typically weighted by age, gender and local area against Australian Bureau of Statistics population data, so over-represented or under-represented groups do not have a disproportionate influence on the overall results.