Author Archives: erindubitably

Stop doing evaluation!

Anybody who has even dipped a toe into the waters of public engagement recently will know what I mean when I talk about the dreaded ‘i’ word.

Impact.

It seems to be everywhere – in funding applications, at conferences, even (for those of us fortunate to work in higher education) in the REF case studies. Impact is the word of the day, and proving that you have it is everybody’s goal. After all, why fund something that isn’t having an appreciable effect? Why spend time and resources embedding something into your practice if it isn’t going to change hearts and minds?

The problem, of course, is how to measure this. Evaluation is impact’s much talked-about but highly misunderstood little sibling. Sure, we need to evaluate our projects, but not just any evaluation will do. This is why I have massively stepped back the evaluation I do of my programmes, all but eliminating the usual gamut of questionnaires and surveys that used to be a must-have for any robust initiative.

Performance Evaluation Form Feedback

Think about it this way: have you ever ever gotten a truly surprising answer to ‘did you enjoy this activity/event/project?’ Most people will have done, a few people didn’t, and that tells you… precisely nothing. Sure, if you’re developing something particularly new or experimental it might be worth checking if your audience enjoyed it, but nine times out of ten you’ll be able to tell how enjoyable something was without asking.

Same with ‘did you learn anything today?’ The facts and figures people might be able to recall and parrot back five minutes after finishing your event are all but worthless in measuring whether you had a real impact on their knowledge. I can memorise a phone number that I need to call – that doesn’t mean I learned it or that I’ll remember it tomorrow, much less in a year’s time.

True evaluation of impact is going to take a lot more effort and a lot more care than what we’re used to. We need to look at long-term changes, all the while understanding the many complex and intersecting factors at play when it comes to affecting people’s attitudes about science. Groups like the British Science Association and Wellcome have started undertaking studies into longer-term impact of STEM projects, among other things, but it will still be many years before we have the data we need to know what makes a good, impactful project.

Despite the click-baity title this isn’t a call to stop all evaluation ever. But think about the questions you’re asking and what they’re telling you. Are they really informing best practice and proving impact, or are they just a waste of your audience’s time – and yours?

So: what questions are worth asking, and what impact should we be aiming for? That will be the subject of future blogs but I invite you to continue the discussion below!

Advertisements

Why do public engagement? Diamond Nine activity

This week I had the pleasure of attending the BIG ‘How to train researchers’ event at Newcastle University. It was an excellent and thought-provoking day and I took away a lot of useful tidbits and ideas.

As our focus was on improving how we support researchers to do public engagement one of my favourite sessions was the ‘activities circus’ where we were able to visit various brave souls who had activities or exercises they have tried and tested for our perusal. One of the ones I enjoyed was the ‘Diamond Nine’ of public engagement – a hands-on tool for facilitating discussion around researchers’ reasons for doing public engagement.

I was so inspired I’m incorporating it into my next training session; I’m looking forward to finding out what the researchers prioritise and hoping that the conversations will spur some introspection on their part!

I’ve included a print-out of the activity and instructions below – if you give it a try let me know what you think!

PDF: Why Do Public Engagement Diamond Nine

The Best Question I’ve Ever Been Asked

The man in the audience was shifty-eyed. I thought my talk about swamp-dwelling leeches, while slightly gruesome, had been going well but he seemed to have other things on his mind. I liked enthusing people about the diversity and creative adaptations of the species that made the swamp their home. I finished up, eliciting the usual ‘eww’s and ‘cool’s, and my audience crowded around the table to see the artifacts I had brought along. The alligator skull was a perennial favourite, and it took several minutes for the crowd to disperse. The shifty-eyed man remained, however, along with a friend. He approached.

“Excuse me, I was just wondering… is there such thing as a real Pokémon?”

“I’m sorry?” I got a lot of strange and interesting questions working as an education assistant at a zoo, but that one was new to me. “A what?”

“A real live Pokémon. You know, like in the game.”

“I… er…” Working in outreach makes you good at thinking on your toes, and I’m happy to say I didn’t stare at him in bemusement for more than a second. Maybe two. “You know, it’s certainly not something I’ve heard of existing in the real world.”

“But aren’t you a zoologist? Don’t you know about things like this?”

“I do, but I’ve never heard of a real live Pokémon. I don’t know if such a thing exists.”

“Well,” he said, eyes darting from side to side before he leaned closer, “I think it does, because I was listening to the news earlier and they said Paris Hilton had been bitten by some sort of Pokémon. So I just wondered if you knew about that.”

“I haven’t heard anything about it,” I said, quite honestly. “But that’s interesting. I’ll have to go look it up!”

He seemed satisfied at this even if I couldn’t confirm for certain, and left not long after. After sharing the story with some coworkers we decided we definitely had to look up what had befallen Paris Hilton.

It turns out she had been bitten by her pet kinkajou. I suppose it does sound an awful lot like Pikachu…

Psychologists highlight academic terms to avoid

Fifty psychological and psychiatric terms to avoid: a list of inaccurate, misleading, misused, ambiguous, and logically confused words and phrases

Six decades ago, two prominent psychiatrists bemoaned the tendency of writers to use “jargon to blur implausible concepts and to convey the impression that something real is being disclosed” (Cleckley and Thigpen, 1955, p. 335). We hope that our article offers a friendly, albeit greatly belated, corrective in this regard.

What science communicators can learn from #ThatDress

If you were on Twitter, Facebook, or any type of social media last night you were probably inundated with one of two things: the live llama chase in Arizona or #ThatDress. As fascinating as I find camelids, I’m going to talk about the more polarising of the two memes.

that dress

So if you haven’t seen it, there it is. What colour is it? Families, classrooms, and friends feeds are divided on the answer, and it’s sent countless people onto the Google results page for colourblindness. Various experts have weighed in with their opinions and technology has been rolled out to tweak, correct, and perfect the picture. Already there are comprehensive scientific articles and videos about why we perceive the colours differently and the fascinating subject of how humans process and interpret light hitting the back of our eyeballs.

What I find interesting about the phenomenon is the completely organic way it came about, and how a few smart cookies jumped on it to do some science communication. The original post went up on Twitter and within hours it was circulating around the world, drawing comments and creating debates among friends and strangers alike. ‘Experts’ (who have now been proven wrong) shared their opinions and amateurs griped and argued in the comments section of various articles. So what made this such a widely-spread discussion?

Firstly and most importantly, I think it was because it was something everyone could have an opinion of. If you could see the picture, you could form an opinion (based on what the rods and cones in your eyes told you), and that was all you needed to join the fray. Experiences that relate to inherent human perception are great jumping-off points for science communication because they are shared among many backgrounds and profiles. There’s a reason sex, death and food are common popular science subjects – they’re rather unavoidable topics and ones we all share!

Secondly, the potential opinions were so different from one another. Blue and black or white and gold are quite distinct, hardly the difference between ‘is this reddish-pink or pinkish-red?’. Once you stated your opinion you were firmly in one camp or another, and you needed no prior knowledge or expertise to back it up. The debate was accessible, something everyone felt comfortable taking part in. Sometimes science engagement can require a level of knowledge of a subject that is off-putting to non-experts. #ThatDress has no such problem.

So what can we learn from this? I’d say it’s the fact that the most gripping science communication is relatable, approachable, and adaptable. It needs to have a topic that interests people, that relates to experiences or knowledge they already have. It needs to be something they feel comfortable forming opinions and talking about (engagement is a two-way street, after all, and a conversation is far better than a lecture!). And most importantly, it needs to keep its finger on the popular pulse, ready to jump on the zeitgeist of the moment. Sure, a sustained campaign of building interest is important, but there’s nothing like hooking into a meme to reach millions of people who might otherwise not have time for a bit of science communication.

Link

The Fermi Paradox

The Fermi Paradox

For a great example of science communication (and some fascinating thought puzzles) take a look at Wait But Why’s The Fermi Paradox, which addresses the question: where is all the other intelligent life out there?

(NSFW language in the article)

Link

Scientists don’t need to wear a white lab coat to talk about science

Scientists don’t need to wear a white lab coat to talk about science

I had a scientist request a bunch of lab gear to take into a classroom just this week (that they don’t normally use in their job). It can be fun playing dress-up, certainly, but it’s interesting to think about the repercussions of having such an authoritative uniform on public perception