Planning your evaluation
There are different types of evaluation depending on who you want to be involved and what questions you're trying to answer.
Decide who'll conduct the evaluation
- can lend credibility to your findings, as they're often perceived as more objective than an internal evaluator
- may have expertise you lack internally, eg data analysis.
- your evaluation is completed quickly and inexpensively
- those involved in the activity have their voices heard
- that evaluations are relevant and present feasible recommendations, since the evaluator is familiar with activity staff, communities, and key issues.
Identify your goals
Decide who needs to be involved
- activity participants
- front-line staff
- organisation leaders
- community members.
- identify questions that meet the needs of both practitioners and participants
- apply results to improve programme performance
- empower participants —the people involved in the activity control the evaluation process and any decisions made as a result
- strengthen evaluation skills for everyone involved
- sustain organisational learning and growth.
Work out how to deal with ethical issues
- get approval from any ethics boards or committees
- get informed and voluntary consent from all your participants
- provide information sheets about the research
- have policies in place in case of any disclosure
- are clear about how the research findings will be used and how the researcher will ensure confidentiality.
- Health and Disability Ethics Committees (HDEC) — for research involving health services
- Institutional Ethics Committees (IEC) — for research originating at a tertiary education institution
- internal ethics committees at specific organisations (eg Plunket, MSD (including CYF) or ACC, or other organisations like schools)
- New Zealand Ethics Committee — any New Zealand research not eligible for ethics review from other institutions can apply for a free, independent ethics review.
You can conduct an evaluation yourself (internal evaluation) or hire someone to conduct it for you (external evaluation).
An independent external evaluator:
However, external evaluation can be costly and may take a long time to complete.
Using a member of your organisation to conduct the evaluation internally can help to make sure:
However, internal evaluations may put additional pressures on scarce organisational resources and staff, and the results may be perceived as biased.
You could also consider hiring an external consultant to support and guide internal evaluators.
Identify which activity elements you're interested in monitoring, and the specific big-picture questions you aim to answer, eg “How have participants' attitudes towards sexual violence changed since starting the programme?” or “Did we reach as many individuals/families as we had hoped, and if not, why was this the case?”
You'll use these goals to guide your evaluation questions.
Theories to help you identify your goals
Sexual violence primary prevention activity designers, developers and practitioners should have an explicit theory of prevention which describes how their activity will produce positive change. Your theory determines what you'll need to evaluate and how.
Best practice review
If you're not clear on what you want to evaluate, consider doing a “best practice” review before proceeding with your evaluation. This review is likely to identify your activity's strengths and weaknesses, giving you important insight into what to focus your evaluation on.
You can use the what works section of this site to identify New Zealand best practice.
Think about who you'll need to include in your evaluation in order to develop and answer your questions. You may need to consult:
Think about whether it's possible and appropriate to collect data from all relevant people (eg, all students in a school-based programme), or whether you'll need to collect a sample.
Theories to help you decide who'll be involved
A participatory evaluation approach involves stakeholders (funders, activity designers and practitioners, and programme participants) in designing and carrying out the evaluation. Collaboration and inclusiveness are key principles of a participatory approach.
A participatory approach helps:
Empowerment evaluation is a is a form of participatory approach that specifically focuses on building an organisation's evaluation capacity.
The key goal of empowerment evaluation is to enable organisations to incorporate evaluation into their everyday activities — it's more focused on strengthening evaluation processes than on obtaining clear evaluation results. Empowerment approaches involve bringing in an external empowerment evaluator to coach key members of an organisation through the evaluation process.
You'll need a policy to ensure the privacy of those involved and to deal with any disclosures that happen during the evaluation process. You'll also need to make sure you:
As a general rule, research and evaluation focused on informing service delivery or improvements does not require ethical approval — but research on people's experience of sexual violence always does.
Ethics approval is particularly important when dealing with vulnerable populations like children or people with disabilities.
Research involving human participants in New Zealand can be approved by:
It's important to seek ongoing funding for evaluation to ensure that broader and longer-term impacts on individuals, families and communities are captured.
Think about setting aside some of your funding to share what you learn — what worked well, what didn’t work, and why.
If you're still developing your activity, consider allocating funding for an evaluator now. Engaging an evaluator at the beginning of activity development can help you develop your evidence base, articulate your theoretical approach and develop a clear theory of change or logic model.
Ethics and research
- Researching Violence Against Women - A Practice Guide for Researchers and Activists — Ellsberg, M. and Heise, L., (2005) (PDF 814KB)
- Ethics — Sexual Violence Research Initiative
- United Nations Convention on the Right of the Child (UNROC)
- International Ethical Research Involving Children (ERIC)
- Engaging children in decision-making — Ministry of Social Development (2003) (PDF 961KB)
- Involving children and young people as participants in research and evaluation projects — Ministry of Education (2010)