Arts-based Evaluation

Art is a window into other worlds. Engaging with a work of art can enable us to understand, empathize, and feel things that we ourselves may not have experienced. The act of creating art allows us to reflect upon our own experiences, sometimes uncovering hidden truths in the process. The power of art is both in the act of creation and in the interaction between the viewer and the artwork, which can transform the everyday into the remarkable, altering the ways in which we view social issues and the world around us.

 Arts-based evaluation for strategies addressing sexual-violence

Arts-based evaluation (ABE) is an alternative and accessible approach for evaluating strategies to address sexual violence and rape culture on campus. ABE provides alternative, participatory approaches that offer other possibilities, working alone or in conjunction with conventional evaluation approaches. ABE processes offer ways to engage program participants using art (e.g., collage, photography, theatre, dance, writing, poetry, digital media, paint, etc.) as a reflection and an expression of the impact and outcomes of a program strategy. It enables evaluators to investigate and translate complex responses to strategies contextualizing participants’ experiences and articulating new ideas about potential program outcomes.

Arts-based approaches to evaluation enable the evaluator to combine the conventions of traditional evaluation with those of qualitative arts-based methodologies in order to enable deeper evaluation insights, meaning making, alternative ways of understanding, and challenging the power dynamics often inherent in more traditional methods of evaluation.

An Important Distinction

Arts-based strategies to address gender-based sexual violence
versus
Arts-based evaluation to assess strategies that address gender-based sexual violence

It’s important to clearly identify how you are using ABE for evaluation. While ABE can provide a very powerful evaluation tool, sometimes the boundaries between the strategy and the evaluation can become blurred. This is why the sections on identifying goals, objectives, and outcomes.

ABE can provide a rigorous and detailed assessment and elicit deeply revealing data when used well. Arts-based evaluation provides an opportunity to seamlessly embed the evaluation into the curriculum for optimal alignment. Oftentimes, when using arts-based strategies, the curriculum, research questions (if the project requires research), and evaluation overlap and interrelate. This is one of the greatest strengths of using art to address social issues; the strategy, research, and evaluation can build upon each other and unfold in harmonious alignment.

ABE is well-suited and especially aligned to evaluate strategies that address gender-based sexual violence on campus because it:

  • Can be mobilized in the interests of the marginalized who may otherwise be excluded by traditional evaluation frameworks.
  • Can initiate provocative conversations and make challenging ideas accessible and inclusive. Complicated academic and policy language in questionnaires may potentially alienate survivors and traditionally marginalized populations and as a result exclude those perspectives that we need in order to end up with meaningful evaluation.
  • Has a proven track record in contributing to projects focused on social change.
  • However, ABE approaches can also be effectively used to evaluate strategies or projects that are not social change oriented. When ABE is used in a strategy addressing social change it can be useful to integrate the evaluation into the strategy design for a seamlessly aligned projects.
  • Can help us see a situation through someone else’s eyes and share an experience empathetically in ways that a survey or questionnaire may fail to do. This is particularly crucial when developing and implementing policies that respond to experiences one might never have or expect to have.
  • Can provide opportunities for participants to speak about the unspeakable, to make the invisible visible when exploring difficult subjects. Art can illustrate complex ideas in profound ways which is especially relevant when evaluating strategies addressing gender-based sexual violence and trauma.
  • Can be used to bridge institutional divides and provide a more inclusive, as well as creative ways to share information about topics that may be emotionally fraught, trauma laden or alienating.
  • Can be used to facilitate evaluation with participants who may struggle with communication or literacy but have alternative ways of expressing themselves and have important insights to share.
  • Can be more powerful and evocative than traditional methodologies (such as, questionnaires, interviews, etc.) in providing information and insights about an issue.
  • Enables evaluation design that is pluralistic and culturally relevant or inclusive in regard to visual language, symbols, imagery, and representation. ABE can employ culturally appropriate and empowering imagery while carefully avoiding cultural appropriation. ABE enables evaluators to incorporate art forms that are already a part of participants cultural repertoire and that are meaningful to participants and their community.
  • Can be done in a healing-centered way with a survivor-centered and trauma-informed approach with attention to not re-triggering participants.

With great power comes great responsibility

Voltaire said it first, “With great power comes great responsibility”. Some use a more current attribution, citing Peter Parker’s Uncle Ben in Spiderman.

While it is a powerful tool, ABE is not always an easy choice. Following are some considerations to be aware of when thinking about best practices.

Questions to consider when employing ABE

  • How can creative, arts-based evaluation methods contribute to the process of evaluating strategies (aimed at addressing and ending sexual violence and rape culture on campus)?
  • How can you engage participants in collaboratively creating emergent arts-based evaluation tools for carrying out program evaluations?
  • Does this context lend itself to using arts-based evaluation approaches? What can this approach bring to your evaluation that other methods may not be able to?
  • How will ABE impact participants? (e.g., increased buy-in, specifically suited to community needs, trauma-informed, etc.)

Promising practices to consider when employing ABE

‘Buy-in’

Not every funder, administration or audience is going to buy into ABE immediately and it may take more time to explain why you have chosen this approach to evaluation. Stakeholders may feel more comfortable with traditional methods that they are familiar with. Suggesting a mixed evaluation approach may help ‘ease’ stakeholders into using arts-based evaluation; for instance, pairing a photovoice activity with a few basic evaluation questions. Often the ABE data adds a deeper and more comprehensive perspective to the overall evaluation, which demonstrates the strength and advantages of ABE.

Powerful Emotions

 

Arts-based activities may elicit strong emotions which could potentially be triggering for participants, facilitators, and evaluators. A trauma-informed approach can help evaluators understand and respond to the powerful emotions ABE may elicit. It’s important to ensure that your evaluation is aligned to your objectives and that you carefully consider the context and fieldsite.

Comfort Level & Trust

 

Not all participants will want to take part in ABE. This type of evaluation is well-suited to situations where you have already built trust between participants or when you have the time to build trust because using non-traditional ways of communicating can make participants feel vulnerable. Being vulnerable in the context of this work is not necessarily a drawback, but it’s a consideration that requires ethical consideration and support for facilitators and participants. That said, there are many ways that arts-based evaluation is easy for participants to engage with because it’s often accessible and enjoyable.

Inclusivity

 

 

While it can be one of the most inclusive strategies, it can in some cases create alienation if too much is asked of the participants in too short a time. When using ABE be mindful that you allow sufficient time for participants to fully participate in the evaluation. Following this, creating a climate of consent is always important but can take more time when using involved ABE strategies.

Resources

 

 

 

Arts-based evaluation, as with most evaluation, works best when well planned and executed. This can require significant resources in terms of time, facilitators, and materials if your evaluation is extensive. However, many ABE projects can be scaled to accommodate resources.

Data Analysis

 

 

 

 

ABE evaluation has the potential to produce very clear data, but also has the potential to create complexities if the prompts that accompany the evaluation activity are not clearly aligned. Like other evaluation approaches it will be important to consider how you work with the evaluation data that you receive. Because there can be multiple interpretations of data it’s especially important to have a clear, well-planned evaluation.

Generalizability

 

 

 

 

 

Due to the nature of ABE, the number of participants is usually limited and therefore while the data can be extremely nuanced and revealing the size of datasets often does not allow for generalizability.

Implementation

As discussed throughout the toolkit, evaluation is most rigorous, comprehensive and usually most effective when you evaluate throughout the strategy or project. Ongoing evaluation, can help create coherence and alignment. Illustrated below is how ABE may be implemented, what it can do and the benefits of using arts-based evaluation at every stage.

At beginning of the project ABE can:

  • Provide insights to help gather a deeper understanding of climate and culture on campus.
  • Identify key issues and needs within a specific community or the broader campus community.
  • Reveal baseline knowledge about the issues the strategy is designed to address.
  • Gathering data regarding awareness, attitudes of participants, and of resources available on campus for addressing and responding to gender-based and sexual violence.
  • Illuminate gaps, unaddressed concerns, underrepresented communities, and under-served groups.
  • Generate the evaluation questions you might have never thought to ask about issues you didn’t know existed.

Throughout a project ABE can:

  • Provide data to evaluate the effectiveness of the strategy at particular points in time.
  • Provide an opportunity for participants to provide feedback that can be used to improve the effectiveness of the project by sharing how the project is impacting their learning, understanding, attitudes, awareness, and behavior.
  • Provide feedback regarding how participants might be feeling or experiencing the strategy.
  • Gather, document, and present evidence.

At the end of the project ABE can be used to:

  • Evaluate effectiveness.
  • Evaluate the impact.
  • Share best practices.
  • Generate potential solutions to issues from within a community.
  • Understand what the strategy meant in the lives of participants.
  • Understand which strategies to develop next.
  • Collect and archive data to be shared with the community.

Case Study: Multimedia Journal

Asking participants to keep a multimedia journal responding to a workshop curriculum on rape culture on campus can provide an evaluation that is embedded in the curriculum. Participants respond to prompts after each workshop session so that facilitators can evaluate whether the workshop is meeting objectives.
This is an example of how ABE can be used throughout a project to provide data to evaluate the effectiveness of the strategy at particular points in time by:

  • enabling participants to provide feedback that can be used to improve the effectiveness of the project by sharing how the project is impacting their learning, understanding, attitudes, awareness, and behavior,
  • providing feedback regarding how participants might be feeling or experiencing the strategy,
  • gathering, documenting and presenting evidence that can be used for overall evaluation of whether the strategy has met the objectives.
  • demonstrating ways in which evaluation can be embedded in the
    curriculum or strategy.

Case Study: Forum Theatre

Forum theatre was created with the objective of empowering oppressed populations to change their world. Dramatized scenes of rape culture on campus provide opportunities for participants to discuss their own experiences and provide an analysis of potential strategies to respond to specific incidences.

Evaluation is conducted collaboratively as participants analyze narratives and responses. The evaluation at the end of the forum theatre can provide opportunities to evaluate effectiveness, evaluate impact, employ the performance to generate potential solutions to issues from within a community, and understand which strategies to develop next. Performative talk backs can be used to understand what the strategy meant to participants. Embedded in the structure of forum theatre is a facilitator who acts as the connection between the performance piece and the audience. This person can ask key evaluative questions that are helpful in evaluating larger campus wide issues. “Were these scenes and situations common?”, “Do you think that your campus has addressed these issues?”, or “If something like this happens, do you know where to go to receive support or to make a complaint?”