Stop doing evaluation!

Anybody who has even dipped a toe into the waters of public engagement recently will know what I mean when I talk about the dreaded ‘i’ word.

Impact.

It seems to be everywhere – in funding applications, at conferences, even (for those of us fortunate to work in higher education) in the REF case studies. Impact is the word of the day, and proving that you have it is everybody’s goal. After all, why fund something that isn’t having an appreciable effect? Why spend time and resources embedding something into your practice if it isn’t going to change hearts and minds?

The problem, of course, is how to measure this. Evaluation is impact’s much talked-about but highly misunderstood little sibling. Sure, we need to evaluate our projects, but not just any evaluation will do. This is why I have massively stepped back the evaluation I do of my programmes, all but eliminating the usual gamut of questionnaires and surveys that used to be a must-have for any robust initiative.

Performance Evaluation Form Feedback

Think about it this way: have you ever ever gotten a truly surprising answer to ‘did you enjoy this activity/event/project?’ Most people will have done, a few people didn’t, and that tells you… precisely nothing. Sure, if you’re developing something particularly new or experimental it might be worth checking if your audience enjoyed it, but nine times out of ten you’ll be able to tell how enjoyable something was without asking.

Same with ‘did you learn anything today?’ The facts and figures people might be able to recall and parrot back five minutes after finishing your event are all but worthless in measuring whether you had a real impact on their knowledge. I can memorise a phone number that I need to call – that doesn’t mean I learned it or that I’ll remember it tomorrow, much less in a year’s time.

True evaluation of impact is going to take a lot more effort and a lot more care than what we’re used to. We need to look at long-term changes, all the while understanding the many complex and intersecting factors at play when it comes to affecting people’s attitudes about science. Groups like the British Science Association and Wellcome have started undertaking studies into longer-term impact of STEM projects, among other things, but it will still be many years before we have the data we need to know what makes a good, impactful project.

Despite the click-baity title this isn’t a call to stop all evaluation ever. But think about the questions you’re asking and what they’re telling you. Are they really informing best practice and proving impact, or are they just a waste of your audience’s time – and yours?

 

Art and Science in the Arctic

I just returned from two weeks aboard a sailing ship filled with artists and scientists. Your first question might be, why?

I heard about the Arctic Circle residency program during a transitional time in my life. I was weeks from unemployment and had been applying for jobs for several months already, and I didn’t really know what was coming next for me. I had been living in Ireland for more than four years and I loved it, but was it better to stay or to go back to the US? And I loved my work as a nanoscience researcher, but I had also become very active in science communication and various forms of public engagement, so would I be better off making a career transition? Was it possible to live in a way that I could do everything I loved doing, or would I have to pick and choose?

In the midst of all of this, the idea of a science/art experience, aboard a ship in the Arctic, was like a dream. Not a research expedition, not a creative hermitage, or perhaps both of those things and more. True interdisciplinarity, in a creative and inspiring environment.

And what an environment! I love the wilderness, the mountains and the sea, but the Arctic has long held a special fascination for me. It’s such a stark environment: brutal and yet full of life and beauty too. The stories of Scott, Amundsen, Nansen, and Shackleton are inspiring and terrifying in equal measure. While I have no desire to freeze to death, I wanted to see the edge of the world, to listen to nature and search for humanity.

So I wrote to the Arctic Circle, talked about projects I could do on board as well as my history of science communication and science/art collaborations. I was delighted to be selected for a 2017 expedition, to take place over the summer solstice during the season where the sun never sets. It was somewhat strange to have 15 months to think about and prepare for such an incredible journey, and in the meantime I got a new job, moved to a new city in Ireland, and came to a very different place than I was in when I first seriously thought about going to the Arctic.

I’ll be writing more about the trip, which one of the other participants pointed out was like an iceberg: the part that’s visible, the trip itself, is only a small fraction of the total. It was amazing but will take a long time to process and sort through. But to start out, I did some vlogs (a first for me so they are pretty raw), and you can watch the first one, from the day we set out, here:

Know the rules so you can break them

Whenever you get up to speak in public, the audience has certain expectations about what you are going to say and how you’re going to say it, based on context: where you are, what you look like, who the audience themselves are. This is perfectly satirized in this meta-TED talk:

Everything about the TED format is pointed out and executed perfectly. It’s almost difficult to watch TED talks after watching this talk, because so many of them are cast from the same mold. It reminded me too of this classic example of a meta-academic talk:

There is only one word in this talk. And yet the tone of voice, the graphs and bullet points, and the story arc of it are all clear and very familiar if you’ve ever sat through a research talk. (This is also a common improv game of scenes done in gibberish, which shows you don’t need words to tell a story.) In both cases, the speakers are showing that the performance element of their talks, the delivery (from vocal inflection to props to body language) can be completely divorced from content. Delivering material in a certain style tells the audience what to expect.

This is as true in comedy as it is in academia. Experienced comics will tell you that you can write a brilliant joke but if you don’t say it in a way that tells the audience to laugh, or if you talk right through them as they laugh, it won’t land. There’s a style of presentation in standup comedy, outside of actual humorous content, that tells the audience what they can expect.

At a first pass, if you’re looking to give a good talk, or tell a good joke, or communicate basically anything to any audience, it helps to be aware of the norms around how material is delivered. Basic storytelling and conversational tools are important too, of course.

But I think these rules are also made to be broken. Going outside the norm when you’re giving an academic talk projects confidence and mastery. Well crafted comedy can be used to discuss tough real-world subjects in memorable ways. Taking tips from performers on timing, stagecraft, and the many ways an idea can be explored helps you to not only understand the expectations of an audience, but also to surpass them and create something new.

Diversity in Science Communication

I have been thinking lately, about how much public engagement is aimed at people who already have access to science and education (too much), about the importance of science in public debate (see: March for Science), and about whose voices are routinely excluded from those discussions. So with thanks/apologies to Flavia Dzodan & DN Lee, my science communication will be intersectional or it will be bullshit.

I want everyone to have access to scientific ideas, to scientific habits, to the radical notion that we are each entitled to talk science with each other (though we may come from different backgrounds, whether in science or in culture). But we must acknowledge that some people already have this access more than others. Science communication should not only be aimed at those from highly educated backgrounds, those with family wealth, those with internet access and science capital and a sense of entitlement. If we ignore the vast majority of people who can engage with science, who can become scientists, then what are we really even doing?

President Barack Obama hosts the second White House Science Fair celebrating the student winners of a broad range of science, technology, engineering and math (STEM) competitions from across the country. The President views exhibits of student work, ranging from breakthrough research to new inventions, in the Red Room of the White House, Feb. 7, 2012. (Official White House Photo by Lawrence Jackson)

Part of the appeal of science is supposed to be our wonder at the way the world is, a willingness to look below the surface and upend our pre-existing assumptions. So why would we accept a system in which access to science depends (still! after all these years!) on surface traits like skin color, gender, or age? It may be tempting to stick to our ideal vision of science as a meritocracy, where all will be rewarded according to their capability, but decades of research on unconscious bias in hiring, retention, and negotiation across science and engineering fields show that if we passively wait for things to get better, we are really just endorsing the status quo.

I love the community of people I have found in science communication. But I can’t help but notice how much of the administrative and organizational work is done by people who are still being disadvantaged in mainstream science: women, people of color, LGBTQ people, people with disabilities. Even the fact that so much of science communication is done by younger researchers, who face a perceived stigma around this work, rather than older researchers with more career stability, is incredibly unfair. We need to actively include these communities in our engagement programs  without asking them to shoulder an unfair portion of the load. Science communication has never been about clever men showing off, but it sometimes looks that way, which is just as bad for the future of science.

You might ask, why do marginalized communities and young researchers take up the banner of science communication more often? Who stays in science is largely determined by how well they feel they fit in. People who don’t resemble the stereotypical scientist know the importance of representation, the importance of seeing people who look like you in the field you want to pursue. Younger researchers are also closer to career decision points, remember the factors that led them to choose science, and still have the passion to provoke that decision in others. It’s encouraging that so many people are still working hard to communicate science well to all audiences… but we must continue to support that work if we want it to continue! Science is an activity done by people, and it is at its best when it makes use of the full spectrum of human experience and knowledge.

I refuse to let science communication put people in boxes, when science is about thinking outside the box. We can do better.

Two Cultures, or Many?

One of the most pernicious myths in neuroscience is that of the left brain/right brain divide. You have surely heard it before: the idea that half our brain is logical, scientific, and calculating while the other is creative, artistic, and empathic. There is no evidence for such a distinction in the actual brain, but the simplistic categories give people easy ways to identify themselves, and others, as members of tribes with specific values. It’s just as easy to feel good about yourself for supposedly having a brain that’s rational or creative as it is to put someone else down for being too cold or impulsive. This myth isn’t about the brain or neuroscience at all: it’s about putting ourselves, and others, into groups.

false dichotomy

In the pursuit of knowledge, similar false dichotomies can arise. Those seeking to understand science may think of themselves as fundamentally different from those seeking to grasp history or literature, and vice versa. This was most famously written about in C.P. Snow’s Two Cultures essay, where he noted the division and in fact the disdain that had arisen between intellectuals in the humanities and the sciences, and how this prevents people from pooling their knowledge to solve the great problems facing humanity. After all, if you do not respect the source of someone else’s knowledge, why would you bother to listen to them?

We should know better. After all, knowledge isn’t just a dry list of facts, but a set of underlying connections between those facts and ideas, as well as an understanding of context (whether it’s human context or physical context). People who study interdisciplinary fields like nanoscience, my area, know that a chemist can bring a very different approach than a physicist does to the same problem. Often both are useful in gaining understanding. Why not extend this same respect to the social sciences and humanities? And why do we not bat an eyelid when someone says they ‘don’t get’ science, when we’d be appalled if they said they couldn’t read?

In my view, an open mind is critical to any pursuit, whether it’s scientific, literary, or even comedic. Don’t limit yourself by how others think before you; go outside the pre-existing accepted framework to solve problems. Isn’t that true creativity, which is required for any academic pursuit as well as for the simple but rewarding task of making sense of this world? Rather than drawing a line between science and the arts, between types of people, we should share our knowledge and natures, expanding our understanding by sharing our humanity.

world

Hotwiring the Brain

The most complex electrical device we possess isn’t in our pockets, it’s in our heads. Ever since Emil du Bois-Reymond’s discovery of electrical pulses in the brain and Santiago Ramón y Cajal’s realisation that the brain was composed of separate pieces, called neurons, we have known that the brain’s behaviour stems from its independent electrical parts. Many scientists are studying electronic implants that can affect how our brains think and learn. New research on conducting polymers that work well in the body may bring us one step closer to the ability to manually overhaul our own brains.

Ramón y Cajal’s drawings of two types of neurons in 1899.

Ramón y Cajal’s drawings of two types of neurons in 1899.

The immediate brain health implications of plugging electronics into the brain, even with a very basic level of control, would be astounding. The connections in the brain can adapt in response to their environment, forming the basis of learning. This ‘plasticity’ means the brain could adapt in response to implanted electronics, for example by connecting to prosthetic limbs and learning to control them. Implantable electrodes which can excite or inhibit neural signals could also be used for treatments of disorders stemming from bad neural patterns, such as epilepsy and Parkinson’s disease.Since the 1970s, brain-computer interfaces have been studied intensively. Passive electrodes which can record brain waves are already in widespread medical use. Invasive but accurate mapping of brain activity can be done by cutting the skull open, as neurosurgeons do during surgery to avoid tampering with important areas. Less invasive methods like electroencephalography (EEG) are helpful but more sensitive to noise and unable to distinguish different brain regions, not to mention individual neurons. More active interfaces have been built for artificial retinas and cochleas, though the challenge of connecting to the brain consistently and for a long time makes them a very different thing from our natural eyes and ears. But what if we could directly change the way the brain works, with direct electronic stimulation?

However, current neural electrodes made from metal cause problems when left in the brain long term. The body views foreign bodies in the brain as a problem and over time protective cells work to minimize their impact. This immune response not only damages the brain region around the electrode, it actually works to encapsulate the electrode, insulating it electrically from the brain and removing its purpose in being there.

These issues arise because of how hard and unyielding metal is compared to tissue, as well as the defense mechanisms in the body against impurities in metal. Hypoallergenic metals are used to combat this issue in piercings and jewelry, but the brain is yet more sensitive than the skin to invasive metals. A new approach being researched by scientists is the use of conducting polymers to either coat metal electrodes or to even comprise them, removing metal from the picture altogether.

Conducting polymers are plastics, which are more soft and mechanically similar to living tissue than metal. Additionally, they conduct ions (as do neurons in the brain) and are excellent at transducing these to electronic signals, giving high sensitivity to neural activity. Researchers at the École des Mines de Saint-Étienne in France have now demonstrated flexible, implantable electrodes which can be used to directly sense of stimulate brain activity in live rats, without the immune reaction plaguing metal electrodes.

It’s a big step from putting organic electronics in the brain and reading out activity to uploading ourselves to the cloud. But while scientists work on improving resolution in space and time in order to fully map a brain, there is already new hope for those suffering from neurodegenerative diseases, thanks to the plasticity of the brain and the conductivity of plastic.

Science Capital

There are lots of different approaches to understanding who studies science and even who feels entitled to talk about it, but the idea of Science Capital is an especially interesting one.

Science capital comes from not just what you know, but also how you think, what you do, and who you know: the cultural factors that lead someone to feel interested and, perhaps more importantly, accepted in science. Enterprising Science have a nice video about the idea and how they are working to measure it:

For those working in science communication, it’s an important reminder to consider how we can not just pass on knowledge, but help others build up more science capital so that they feel entitled to be part of the conversation.