Inspiring Australia (NSW) is committed to ensuring that we support partners in connecting scientists to the community through programs that are effective and seek to have a long-term impact. Measuring our success in supporting community partners to deliver quality public engagement is important, particularly in the context of public funding, scarce resources and volunteer-led programming.
Currently, we measure our progress in supporting our network partners by documenting outputs:
- Counting – numbers of events, audiences, participating and presenting partners, participating researchers and STEM professionals.
- In-kind contributions – of cash, free access to venues and infrastructure, professional support like marketing and publicity and volunteer labour.
- Diversity – of topics, presenters, audience members, geographic location, event types.
- Quality of science engagement experience – via audience and partner satisfaction, professionalism of event delivery, scientific content presented by or verified by appropriately qualified professionals.
- New audience reach – how programs target underserved communities.
- Media and social coverage – online content that extends reach of event.
- Participation by civic leaders – including endorsements and participation by politicians, local government, prominent scientists and business leaders.
- Enduring partnerships – length of time partners have worked together, ability of partners to both nurture and expand their local networks.
In 2018, NSW grant funds of $317,000 leveraged more than $1 million co-investement from over 900 partners to deliver 779 science events across the state that reached more than 297,000 audience members.
While we collect qualitative data to seek information from partners about the effectiveness of their programs and audience engagement through evaluation surveys and feedback, we are limited in our ability to report on outcomes and the longer-term impacts that may occur as a result of all this activity.
This information is key to learning whether our programs increase scientific literacy and/or lead to behaviour or attitudinal change and is an area we would like to develop.
Over the coming year, we’ll therefore be asking partners and grant recipients for new information that can assist us to track the long-term impacts of the science engagement programs we support. We want to demonstrate that we’re using public funds to support the delivery of meaningful and mutually beneficial programs that respond to community needs and interests and extend the reach of science engagement activity.
We’d like to develop better ways to prove that the programs we support achieve the overarching program goal of creating a society that understands and appreciates the relevance of science to everyday life and its value to Australia’s economic and social wellbeing. A big focus for Inspiring Australia (NSW) will be to continue to engage with new audiences, not just those who are already interested in science.
We’d also like to know more about is what happens as a result of partner interaction with Inspiring Australia (NSW) – for example success in raising additional funding, significant invitations and awards, new partnerships and/or the development of legacy programs.
Tracking these kinds of outcomes will help us prove the value of nurturing a network through collecting evidence that demonstrates the benefits of Inspiring Australia (NSW)’s funded activity.
Measuring participatory engagement
The question of how effective science engagement offerings are in improving scientific literacy was keenly debated at the recent Australian Science Communicators national conference.
Various methodologies are used by academic researchers to assess the effectiveness of common approaches, with some arguing that the majority of science engagement programs fall into the so-called “information deficit model” of science communication that, over decades, has been proven to be ineffective.
The deficit model is an idea developed almost 50 years ago. It assumes that gaps between scientists and the public are a result of a lack of knowledge and way to close this gap is to provide information from experts in the hope that this will change individuals’ attitudes, beliefs, or behaviours. Among assumptions of this approach are that non-experts are ignorant or have negative attitudes towards science and technology.
In our experience, science engagement programs supported by Inspiring Australia (NSW) and delivered in community settings are almost always created by multidisciplinary teams. They typically offer opportunities for audience participation, particularly when events are held outdoors, in bars or clubs or involve the creation of an artwork, theatre piece or participation in a citizen science activity.
Dialogue is part of the event experience, from the planning to the delivery, and scientists often report being challenged in new ways by their presenters and community audiences.
While community events often include a short talk by an expert, they are unlikely to be “deficit” in their approach due to the environment in which they are delivered. Further sharing of information, podcasts and videos via social media is another means of sparking ongoing engagement and dialogue.
The community groups and regional networks we work with play a huge role in creating programs that enable public participation. It’s also usually the case that scientists work with non-scientists to develop participatory programs that resonate with local audiences who feel welcome to voice their ideas. Our partners report these approaches work well, but how can we prove they do?
A focus for Inspiring Australia (NSW) next year will be to gather deeper insights into audience engagement experiences. We’d also like to document what approaches our partners find are most effective in connecting scientists with community members to explore and debate scientific knowledge.
Such information will help improve future programs and also build a stronger case for public engagement. It will also potentially show how programs supported by Inspiring Australia (NSW) add value not only to audience members, but also to the participating scientists.
Find useful evaluation resources on the UK-based National Coordinating Centre for Public Engagement website.