The network effect

What happens when you provide a small amount of funding to a group of skilled volunteers to enable them to present regular public programs in partnership with enthusiastic scientists? Propelled by small grants, buckets of goodwill and reciprocal networks, Inspiring Australia NSW supports the delivery of creative science engagement experiences, sparking lasting action that ripples through communities.

Demonstrating the difference made over eight years of fostering networks and supporting hundreds of community partners to deliver quality science public engagement experiences to a broad public is no mean feat. To help us articulate the true value of Inspiring Australia NSW’s network effect and the trust invested by hundreds of partners in our collaborative model, we recently held a stakeholder discussion about developing a holistic impact measurement framework.

Jeremy Thorpe, PWC’s chief economist and Jackie Allender from Deloitte’s sustainability and climate change team led an insightful conversation about capturing the social impact of our state-wide STEM engagement efforts.

Our metrics

During our COVID-19 year in 2020, Inspiring Australia NSW’s grant funded projects achieved the following results:

  • 409 partners created 440 mostly online event segments
  • 700 scientists and science communicators were involved
  • More than 685,000 audience members attended live program presentations, with most events continuing to attract engagement online.
  • NSW grant funding of $182,000 leveraged at least $519,000 co-investment – excluding volunteer labour.

These numbers are incredible and similar results have been delivered year on year – but they don’t demonstrate the long-term impacts of Inspiring Australia’s influence in NSW.

How we report

Inspiring Australia NSW is required to submit detailed reports to its program funding body, AusIndustry, twice a year. We provide data on numbers of events, numbers of audience members and which hard-to-reach audiences may have been engaged. We provide names of participating and presenting partners and names and disciplines of participating scientists. AusIndustry also asks us for event presenter postcodes.

In NSW, we gain a deeper understanding of our value by collecting additional data that includes:

  • Estimate of value of in-kind contributions – of cash, free access to venues and infrastructure and professional support
  • Quality of science engagement experiences – grant recipients are asked to gather data documenting audience, presenter and partner satisfaction
  • Stories – we publish stories about the events, the scientists involved and the audience participation and response
  • Professionalism of event delivery – we assess the capability of presenting partners to deliver a high-quality STEM engagement experience
  • Longer term outcomes – we investigate what happens as a result of various collaborations with high-value partners – including new audience channels and further opportunities for scientists.

This work involves fostering trusted relationships, staying in close contact with our many grant recipients and being open to feedback about the effectiveness of our approach. We continuously ask how we might better support our presenting partners and encourage them to survey their audiences.

NSW Regional Science Hub members get together for two-day forums every two years for professional development and extensive discussions about local impacts. These sessions are always invigorating and help us identify new ways to extend the reach of our science engagement efforts at a state and local level.

Other things I take note of across the program include:

  • Diversity – of topics, presenters, audience segments, geographic location, event types
  • Media and social coverage – reports about our projects and the scientists involved
  • Participation by civic leaders – how support from politicians, local government, prominent scientists or business leaders helps raise the profile of a community project or the associated STEM researchers.

Gathering this kind of information is straightforward. However, data collection involves a lot of time, and Inspiring Australia NSW is limited in how much it can collect and interpret, given the huge breadth of activity covering an entire state.

Over the years, I’ve witnessed a network effect that’s been exponential, and this is an impact story worth sharing. If more resources were available, I’d capture more information about what happens as a result of the dynamic relationships I nurture.

Our social impact

At our most recent Stakeholder Briefing held online on 22 March 2021, we investigated whether our impact is far greater than what we measure.

Our guest presenters Jeremy Thorpe, PwC’s chief economist, and Jackie Allender from Deloitte’s sustainability and climate change team, led a conversation about techniques for capturing the social capital value of STEM engagement and how we might apply this lens to programs supported by Inspiring Australia. They were joined by representatives of three Regional Science Hubs.

Listen to a recording of the discussion

Listen to a recording of the Stakeholder Briefing Discussion on social impact frameworks. 1 hr 22 min.

Jeremy Thorpe began by stating that Inspiring Australia NSW is not alone in trying to determine its true value. Increasingly, all kinds of organisations, from NFPs to business and governments, are looking for credible ways to measure their value and relevance, often through a dollar figure or some kind of numerical metric.

Jeremy spoke of the many ways to define value – it can be in a dollar sense, or it might be value through delivering a service, a saving or simply by building awareness. The important thing to note is that there is no one measure for value.

Jeremy said that being clear about who an audience is can help you define the most appropriate determination of value for a project. He then explained some of the common frameworks used by economists to measure social return on investment, and how a theory of change model can help demonstrate how an activity impacts on someone – and what beneficiaries come from this.

“A program logic is useful to show how there’s a situation, a need and a methodology for meeting that need,” Jeremy said.

Describing inputs and outputs is easy. Where the challenge lies is finding a way to capture outcomes and long-term impacts.

“These are longer term measures and it usually takes some time to make a judgement about whether a program or policy is successful or not.”

This is particularly challenging for Inspiring Australia’s state programs whose managers are engaged on short term contracts located in host institutions and whose budgets do not extend to covering the costs of evaluation.

Jeremy noted that often the base case for an intervention is overlooked by program designers. He argued that a strong sense of what’s happening in the present prior to a program being established is needed as a first step. Ideally this would entail both a reflection on not only what is the status quo, but specifically, what was the status quo in the absence of the intervention being introduced.

“The biggest challenge is the time horizon – it may be years before the solution to the challenge you are seeking to overcome is delivered,” he said.

Economists make a lot of plausible assumptions when they conduct evaluations, usually based on other data about what has happened before. Jeremy said that like scientists, economists are always thinking creatively about changes that are likely to happen in the longer term in order to predict how actions today will influence outcomes in the future. Unlike scientists, however, they are prepared to make informed assumptions without necessarily having hard evidence to prove their claims.

He warned that despite decision-makers wanting short and sharp summaries rather than in-depth analyses, big numbers can be misleading and even obscure key messages about impact. Instead of relying on the provision of ever-increasing numbers, Jeremy recommended using case studies and personal anecdotes, as these can be more powerful in conveying incremental social impact.

“Stories matter. Show how someone’s life was changed because of something that happened as a result of their interaction with your program,” he said.

Jackie Allender from Deloitte continued the reflection on the many different types of value that can be delivered depending on the target audience of a program.

She suggested that a broader way Inspiring Australia NSW could look at how its programs are providing value could include describing:

  • The breadth of networks and statewide relationships
  • In-kind value, including human capital and leveraged contributions like marketing, talent, labour and venues
  • Jobs created by the presentation of community programs
  • Intellectual capital created including through artistic creations, new event formats, audiences being educated and people being encouraged to be more open to science and innovation
  • Manufactured capital – any new inventions created as a result of a program
  • Financial capital – commercial outcomes and local expenditure at events.

Jackie spoke about the incredible labour and skills that volunteers provide, and the value of work and services contributed by local businesses and venues.

In addition to tracking the immediate outputs created by a program, Jackie explained that a useful way of tracking longer term outcomes is to use a social impact framework.

Referring to a recent evaluation Deloitte conducted of Gippsland’s Neighbourhood House program established to benefit people experiencing social isolation or disadvantage, Jackie explained how a number of quantifying measures were used including: 

  • Health and wellbeing, and an estimate of healthcare costs avoided
  • Community connections  
  • Resilience
  • Independence
  • Productivity.

“What we are doing here is working on a theory of change in the short, medium and longer term, through the prism of the various audiences that use this service.”

The overall value of the initiative was calculated as being $2.78 economic and social benefits leveraged for every $1 investment.

Jackie agreed with Jeremy that impact analysis usually involves making informed assumptions. She also recommended looking at other sources of information to back up claims, such as other research data pertaining to the issues you are trying to solve.

Making assumptions about the proportion of change to which your initiative has contributed is another common way to articulate value. In terms of volunteering, for example, you could investigate increased wellbeing though social connections, local networks and reduced social isolation. You could also assume huge value through new skills and knowledge – including by contributing to an innovative culture.

To keep impact reporting affordable, Jeremy recommended using surveys to collect data, and also to identify one key benefit that arises as a result of an activity by doing a break-even analysis.

“If you can identify the cost of the program versus the cost of delivering just one key beneficial outcome, you could show how the program investment pays for itself.”

For example, ‘If we only got one more scientist as a result of all these events …. If we reduced single use plastic in one town in our region, we would save …’  

Regional perspectives

NSW Regional Science Hubs have been established by Inspiring Australia NSW to develop and deliver high-profile science, technology, engineering and maths (STEM) events in regional NSW. With links to multiple communities, their efforts are changing public perceptions about science and technology.

Representatives from three Science Hubs were invited to share their perspectives from delivering year-round events and activities that promote the value of STEM in local communities in partnership with business, local government and the environment, arts and cultural sectors.

Doug Reckord from the Bega Valley said that the Sapphire Coast Science Hub unites a huge number of organisations that now regularly work together. Benefits of their ongoing collaboration since 2013 include increased visibility for science, new audiences, major collaborative projects and greater awareness of the region’s unique biodiversity.

“It is marvellous that as a small enterprise, Inspiring Australia NSW’s funding and professional learning opportunities have been so significant, not only allowing us to meet people from other regions but also to develop skills,” said Doug.

“Even today, hearing from Deloitte and PWC has given me ideas that I can use in my own planning, and having access to the Virtual Excursions training last year was crucial to our success in delivering science week online.”

Doug said that while collecting evidence takes a lot of resources, simply by looking at the photos of people at events, you can see that they are having a good time – and how the community events open people’s eyes to what science can be.

“Without that external support from Inspiring Australia NSW, it would be a struggle in rural Australia to keep this kind of work going.”  

Kirsti Abbott from New England spoke of the benefit to her region’s Science Hub of having access to staff and students at the University of New England.

“We are an agricultural and environmentally focussed community and the benefits for us have been enormous, especially having that long-term investment over 8 years,” she said.

The New England North West Science Hub is led by volunteers that have worked together for a long time alongside people that come and go, bringing with them different skills.

Kirsti described the Science Hub’s many and varied partnerships, including with community groups and the art museum. She said that huge benefits flow to the Science Hub from having access to so many different networks.

“The networks have been fantastic for us – it also means we can link out to people in other Science Hubs too. Bringing those regional engagement people together has also been invaluable, and we are always looking at what each other is doing as well.”

Kirsti praised the Science Hub model that allows regional partners to be creative and specific to their local context while also addressing at commonalities.

“Our demographics are very specific – so we appreciate being able to use the grant funds to do what we need to do in our area. An off the shelf model would not work as well,” she said.

Kirsti spoke of micro values – such as understanding what drives an individual to attend an event and how this is likely to be different to what motivates families. There is also the value to communities and regions to consider, and a global value to science engagement as well.

“The impact on our own confidence as science communicators has been enormous. I feel like we are part of a community and I can truly say that we are science engagement experts: this is what we do!” she said.

Joining from the James Theatre in Dungog, Jane Richens and Brian Doherty from the Paterson Allyn Williams Science Hub echoed Kirsti’s reflection on micro values.

“Our approach to providing a social benefit to our community is based around bringing science awareness around really topical issues such as renewable energies, biodiversity loss, the NBN and sustainability,” said Jane.  

With 42 events presented in the last five years, the Paterson Allyn Williams Science Hub seeks to explore topical ideas suggested by community members.

Jane said that consistency is important for building trust, making connections and having impact. The group appreciate how year-round funding makes it possible for them to provide ongoing activities, something they see as being critical to the Science Hub’s success and growth, and its provision of meaningful social interactions.

“The structured, social interactions we build into our events usually lead to quite a few other events and programs around the town,” said Jane, who describes the group’s regular events as innovation incubators in themselves.

Science Hub gatherings have sparked all kinds of behaviour change action, from the introduction of a Boomerang Bag program and a Sustainability Spotlight Expo of local businesses demonstrating sustainable practices to the creation of a plastic free initiative for the Dungog Shire and a Blue Planet Status awards system to recognise local businesses that adopt sustainability practices.

“Spin offs of our STEM engagement have led to many new sustainability initiatives being established in our region. Community building and seeding ongoing activities are a result of what we do as a Science Hub,” said Jane.

Brian said that the great part of ensuring scheduled time for social interaction is allowing the opportunity for chance meetings to occur– trust develops and ideas flow.

“People have the opportunity to creatively explore ideas that arise as a result of their interaction with a scientist,” he said.

“The opportunity to be part of the Science Hub network has also been invaluable – all the professional learning opportunities provided by Inspiring Australia NSW are really useful,” added Jane.

Where to from here?

It may be impossible to track all the long-term impacts of the Inspiring Australia initiative across NSW – such as the creation of intellectual capital, professional skills development, social networks and the agglomeration of new initiatives created as a result of regular events presented across the state. But by sparking interactions that are fun and engaging as well as being educational and seeding longer-term action, it’s clear that STEM engagement certainly contributes to the social fabric of a town. The fact that the state-wide Regional Science Hub network continues to thrive is another testament to the initiative’s success in NSW.

How we continue to track and document the lasting benefits that flow from STEM engagement funding and our strategic decision eight years ago to invest primarily in sustaining long-term partnerships is a work in progress, but one that remains a priority for Inspiring Australia NSW.

Thanks to all those that participated in this discussion about social impact, in particular Jeremy Thorpe, Jackie Allender, Doug Reckord, Dr Kirsti Abbott, Jane Richens and Brian Doherty. Thanks also to our funders AusIndustry and the Office of the NSW Chief Scientist & Engineer and our host institution, the University of Sydney.

By Jackie Randles, Manager Inspiring Australia NSW.