CoAct Webinar: Co-shaping evaluation in Citizen Science?

CoAct Webinar: Co-shaping evaluation in Citizen Science?

On January 27th, 2021 we will organise the first public CoAct webinar: Co-shaping evaluation in Citizen Science? Towards more participatory approaches in evaluation of Citizen Science in cooperation with ECSA and EU-Citizen.Science.

Speakers:
Anna Cigarini (University of Barcelona, CoAct)
Johannes Jäger (IEA Paris)
Barbara Kieslinger (ZSI, CoAct)
Katja Mayer (ZSI – CoAct, University of Vienna)
Obialunanma Nnaobi (Vilsquare)
Teresa Schäfer (ZSI, CoAct)
Katie Richards- Schuster (University of Michigan)

Date and time: 27 January 2021, 4-6pm CET

Registration: until Jan 18th via coact.webinar@zsi.at Please state your name, your email address and your motivation to participate in the webinar.

Documentation: the webinar will be recorded and shared on the CoAct website several days after.

Introduction

Citizen Science is a means of bridging science and society. In addition to the generation of scientific knowledge, Citizen Science activities are particularly well-equipped to respond to societally relevant questions, contribute to science communication and foster scientific literacy in society. While all these aspects are highly relevant for citizen engagement, empowerment and social innovation, they are rarely evaluated in a coherent way. Current evaluation activities in Citizen Science tend to focus on scientific aims, data reliability, and at most the socio-ecological relevance of the results. In the case of projects with a more accentuated educational goal, these are complemented by an assessment of the learning gains at the level of individual participants. Wider societal and political implications are hardly ever assessed, which is exacerbated by the fact that they are notoriously hard to measure. 

During the discussions at the 2020 ECSA Conference, it became clear that there are already a lot of evaluation instruments available – including digital ones – and that some of them also enable participatory dimensions. However, it was reported that few of these instruments are adopted, if any at all. Is it because they are too little known? Is it because it is so difficult to create content-independent, digital environments that enable participatory evaluation for many domains and research questions? Or is it because evaluation is often tacked on to ensure compliance, instead of being a central part of research design? This webinar is dedicated to discussing strategies, formats and tools for participatory evaluation with a special focus on co-evaluation. 

Co-evaluation is a form of participatory evaluation that initiates the conversation on expectations, objectives and impact already at the start of a project or initiative, either when the program or research design is co-created with different stakeholders or at the latest when the participation of actors is negotiated. The main difference between co-evaluation and conventional types of research evaluation is that participants are involved in the decision on project goals and evaluation instruments.

Objective of the webinar

Participatory evaluation is an approach that aims at giving voice to the stakeholders of an intervention in its evaluation design, process and results. This webinar will shed light on  the specificities of this methodology, as well as challenges and opportunities related to its application in citizen science. The aim of the webinar is to furthermore provide an overview on co-evaluation as a strategy and to discuss which respective approaches and options have been available for a long time in participatory research and citizen science, how they have been received, what opportunities they have opened up, what obstacles have been overcome, but also what we can learn from them for the future.

After an introduction and an overview about the state of the art of evaluation (participatory and non-participatory) in citizen science, core principles of co-evaluation will be presented. Experts will then discuss their experiences on a panel, with a special focus on how to approach participants as evaluators, current challenges in times of crisis and physical distancing, and resulting digital options for more participation in evaluation. 

This webinar is targeted towards researchers, evaluators, project designers, and communicators working in a participatory research and citizen science context. The objective of co-evaluation is not only to promote discussion and learning for the scientific dimension of a project, it should also promote a project’s impact including change in the living environments of project participants. Thus the discussions – for example how to best approach participants as evaluators -are useful to people involved in citizen science, program design, policy, and planning. 

Topics

Participatory evaluation in citizen science, co-evaluation, how to approach participants as evaluators, social impact

Agenda and formats

Min SectionDescription
10Arrival and WelcomeShort introductory round and welcome, incl. a brief summary of the CoAct project
30Co-Evaluation Primer
Barbara Kieslinger, Katja Mayer, Teresa Schäfer
Lecture on evaluation in citizen science, with a special focus on participatory evaluation formats that could be implemented from the start of the project, giving voice and decisive power to the participants in the choice of evaluation criteria and instrument. 
45Conversations on experiences
Katie Richards- Schuster, Obialunanma Nnaobi, Johannes Jäger, Anna Cigarini
Discussants share their experiences with participatory evaluation in brief lightning presentations. Then the discussion will revolve around challenges and benefits of such evaluation (informed consent, ownership, overload, ..) and options and (potential) instruments for co-evaluation
30Discussion/Q&ADiscussion of questions posted to the chat
15Feedback and SendoffFeedback session

Speakers:

Participatory research with young people is important because young people are experts in their lives, and their lived experience can and must shape knowledge developed about them and their communities.

Katie Richards-Schuster, Ph.D., is an Associate Professor and Director of Undergraduate Minor Programs at the University of Michigan School of Social Work in Ann Arbor, MI, USA.  Her research focuses on understanding the strategies and approaches for engaging young people in communities, the contexts and environments that facilitate youth engagement across settings, and the impact of youth participation in creating community change.   She has worked in and with communities to promote youth participation and has led national and global efforts to increase youth voice in research and evaluation.  She is a leading scholar in using participatory research and evaluation approaches with young people and communities and is the former co-chair of the Youth Focused Evaluation TIG within the American Evaluation Association. 
https://ssw.umich.edu/faculty/profiles/tenure-track/kers

I realized early that having (all) stakeholders contribute to designing and implementing programme M&E systems leads to better understanding of the intervention, strengthens ownership, improves accountability and gives voice to the most vulnerable. The stakeholders own the process and are “Champions” in its implementation.

Obialunanma Nnaobi is a development practitioner whose work combines elements of research, strategy and advocacy to support good governance causes, innovative use of technology and the empowerment of women and youth. As Co-founder at Vilsquare, she works with a wide range of partners to deliver on pan-African solutions to the continent’s infrastructural challenges. She has held key positions in multi-stakeholder initiatives in Nigeria like the Open Government Partnership (OGP) where she supports diverse stakeholders to collaboratively achieve shared accountability objectives and development targets. Twitter: @nmannaobi @vilsquare
https://vilsquare.org/makershub/

We must move away from metric madness, from our obsession with outcomes, towards a process-oriented form of evaluation that is tightly integrated with teaching, mentoring, and facilitation.

Johannes Jaeger is an evolutionary systems biologist and philosopher. He is interested in developing a theory of knowledge that is tailored to open science, inspired by his work on organismic agency and innovation in biological evolution. He is the current D’Alembert Research Chair at the Institut d’Études Avancées (IEA) de Paris, and associate faculty at the Complexity Science Hub (CSH) Vienna. Twitter: @yoginho
www.johannesjaeger.eu

Considering evaluation as an integrated research activity and establishing a structured dialogue with participants since the very beginning beyond objective and quantifiable measures is crucial to build trust and mutual understanding, and thus a reflective evaluation capacity.

Anna Cigarini is a PhD candidate in information and knowledge society at Universitat Oberta de Catalunya. She is a member of OpenSystems (Universitat de Barcelona) in the CoAct project, and collaborator at Dimmons (Universitat Oberta de Catalunya). She holds a MsC in sociology and demography, a MsC in population studies and a BsC in statistics. Anna is interested in the intersection of the technical and social aspects of technology. In particular, her research focuses on the governance of citizen sciences’ communities of practice.
Twitter: @anna_cigarini, @OpenSystemsUB, ‎@dimmonsnet

If we want to take co-design seriously we also have to take co-evaluation seriously.

Barbara Kieslinger is a senior researcher and project manager at the Centre for Social Innovation in Vienna, Austria, ZSI. Since 2012 Citizen Science has been a topic of research for her, next to the relation between technological and social innovations. Barbara coordinated large research projects dealing with innovations in workplace learning and was recently involved in projects related to digital social innovation and the maker community. Barbara currently coordinates an EC-funded project on open healthcare, which facilitates co-design of open healthcare for people with physical limitations. Barbara also serves regularly as external expert for the European Commission and reviewer for scientific journals and has recently been elected as part of ECSA’s board of directors.  Twitter: @bkieslinger

In citizen science we need evaluation that matters.

Katja Mayer is a sociologist at the University of Vienna, Austria, who works at the interface of science, technology and society. Her research examines the interactions between social science methods and their public spheres, focusing on the cultural, ethical and socio-technical challenges at the interface of computer science, social sciences and society. In addition, she is Senior Scientist at the Center for Social Innovation in Vienna, ZSI and Associate Researcher at the University of Vienna’s ‘Governance of Digital Practices’ platform. Twitter: @katjamat

Teresa Schäfer studied Economics at the University of Vienna. She is senior researcher at ZSI and focuses her work on participation processes in digital social innovations and the assessment of their impact. Teresa has been leading the consultation process for the development of the Citizen Science Whitepaper for Europe and is work package leader for evaluation and impact assessment in several citizen science projects (e.g. CAPTOR, CoAct, EU-Citizen.Science). Teresa has many years of experience in participatory methods for design, evaluation and impact assessment, involving a broad range of citizens, like retired people, or migrants, in research projects in the 6th/7th FP and H2020.