The Role of Research in Producing Evidence for Impact Generated through SEA Projects
Three key approaches can be used to collect evidence of transformation that can create an impact. Evidence can be ‘hard’ quantitative data or ‘soft’ qualitative data collection approaches, or a blend of these approaches, referred to as a mixed approach to data collection. Most importantly, the evidence or data we collect should be collected through a formal, reliable, robust and strong approach. To select the correct approach, the SEA practitioner needs to understand which approach will be most suited to the project when planning methods for collecting evidence.
Softer and qualitative approaches include the collection of anecdotal evidence from interviews or storytelling that is then transcribed and coded. Hard and quantitative approaches may include surveys, questionnaires, polls, and so on. The most robust evidence is believed to include a blended or mixed approach, in which researchers collect data by selecting methods from both approaches.
Again, SEA practitioners must consider how they will use artistic approaches, if any, in the collection of evidence. Arts-based methods can be very useful in assisting SEA practitioners in collecting creative outcomes that can provide rich visual and textual data that can generate different ways of knowing about the realities that communities face and deal with. For example, collage making can overlap with creative writing or poetry to enable a whole new dynamic through which we can collect evidence of impact and empowerment. Patricia Leavy (2017, p. 191) aptly explains:
…the artistic process itself may be the enquiry. These approaches are most commonly used when the aim is to explore, describe, evoke, provoke or unsettle … it (arts-based research) follows a generative and emergent process, open to the unexpected – to surprises, new insights, and bends in the road. So, even when we have a plan for how a particular enquiry will proceed, in practice, it can and often ought to be a messy process.
Arts-based methods used for data collection can uncover underlying values, social injustices, identity and political complexities (Leavy, 2017).
The following factors need to be considered when selecting suitable approaches for collecting evidence in SEA interventions:
- Number of sources that provide evidence
- Quality of the sources that will provide evidence
- Process for the collection of evidence and how it is integrated into the project design
- Visual and text-based data that will be generated using arts-based methods
- Analytical approaches that will be selected to understand the data
- Arts-based approaches that can be used to analyse data
Andrea Karpati (2020) delivered a collection of methods for the evaluation of artistic projects in the AMASS project’s Deliverable 1.2, Comparative Policy Analysis and Evaluation of Artistic PROJECTS (p. 25-45. The document discusses several methodological approaches for collecting evidence of the impact generated through artistic projects. Students who are interested in knowing more about the application of art-based approaches for collecting evidence of their SEA projects can consult this resource at this link:
Note that this document is not a compulsory reading material for this learning unit.
The following activity will assist you in reflecting on and critically considering how an SEA project has been evaluated: While engaging with the activity, you will benefit from taking personal notes while reading, as it will assist you in reflecting on the questions posed for reflecting on how to gather evidence of impact.
In the previous learning unit, you were introduced to the case study of the SEA project Rakkaustalkoot (Love Talks), which was discussed in the article by Hiltunen et al. (2021). For this unit, we will continue to work with SEA Case Study 1, as you are already somewhat familiar with the content. For this activity, you are required to read Chapter 5, Love Talks: Integrating Foreign Residents into the Rovaniemi Community.
Sarantou, M. (2021). Love Talks: Integrating foreign residents into the Rovaniemi community. In Karpati, A., & Sarantou, M. (Eds.), Arts-based Social Interventions: First Results of the AMASS Testbed. University of Lapland. https://urn.fi/URN:ISBN:978-952-337-270-2
Reflect on the following questions:
- What kind of data collection approach was selected for Love Talks to assess the impact of the project?
- How effective was this research method? Were you able to clearly understand the impact that the project generated? Can you give an example?
- How could the assessment of impact have been improved? What additional evidence could have been collected?
- Can you describe different methods for data collection, and what kind of evidence or knowledge would these methods produce?
- Can you think of any arts-based method that could have been used to gather evidence of the impact?
Hiltunen, M., Koskenniemi, P., & Sarantou, M. (2021). Love Talks and Neighbourhood: Promoting encounters, tolerance and social inclusion by means of art in daily life and the living environment in Finnish Lapland. Malta Review of Educational Research, 15 (Supplement issue on Socially Engaged Art and Global Challenges), 97–117. http://www.mreronline.org/issues/supplement-issue-on-socially-engaged-art-and-global-challenges-december-2021/
Leavy, P. (2017). Research design: Quantitative, qualitative, mixed methods, arts-based, and community-based participatory research approaches. Guilford Publications.
Sarantou, M. (2021). Love Talks: Integrating foreign residents into the Rovaniemi community. In A. & Sarantou, M. (Eds.), Arts-based Social Interventions: First Results of the AMASS Testbed. University of Lapland. https://urn.fi/URN:ISBN:978-952-337-270-2
Vella, R. & Sarantou, M. (eds.) (2021). Documents of Socially Engaged Arts. InSEA. https://doi.org/10.24981/2021-DSEA