The Values Versus Violence organisation has grown out of what was known as the Kids Taskforce. The programme is endorsed by Sir Hugh Orde, the President of ACPO. Sharon Doughty, the executive producer of all of the Watch over Me series, explains how the programme started and her passion for doing this.
Investigating the effectiveness of the programme
VVV evaluation West Midlands Police:
VVV has developed and delivers two programmes, one aimed at primary school pupils and the other aimed at secondary school pupils. Central to VVV programmes is the framework they provide for helping children to recognise and assess the risks they face.
This report builds on the early findings of evaluation commissioned to assess the use and effectiveness of the Values Versus Violence (formerly Kids Taskforce) resources under the Home Office contract. As a free resource supported by HO grant funding, the evaluation focused on use and effectiveness rather than value for money.
Further evaluation is being carried out with 80 primary schools for a Year5 resource in Birmingham, supported by the West Midlands Police.
The VVV resources aim to empower young people by helping them to develop positive behaviours and learn how to keep themselves and their friends safe. They provide opportunities to discuss criminal behaviour and victimisation in a safe environment and, as a result, they lead to changes in behaviour. The principal focus of the Home Office evaluation study is to support the Kinsella Report related to gang and knife related violence; in particular, to avoid knife crime and getting drawn into gang membership. This provides the basis also for the evaluation of Birmingham primary schools, currently being undertaken for the West Midlands Police.
There are, therefore, three main outcomes against which the education resources can be evaluated, for:
- Young people to have effective strategies for dealing with peer pressure and recognition of risky situations
- Young people to know who in the community can help them
- The programme has a positive impact young people’s awareness, understanding and behaviour on knife crime and gang membership
The evaluation framework included:
- Measures: standardised instruments (not gearing questions)
- Replicable: ‘targeted’ evaluation with schools (i.e. not ‘open’ for interpretation)
- Teacher survey: closed questions (90%) with some open-ended text responses (10%)
- A maximum of 10 to 15 minutes to be completed (HO: online; West Mids: paper copies)
- Student response; Teacher feedback (Pre/use; Post/use)
- Interviews: semi-structured, to last no longer than 30 minutes
- By telephone
The restricted period of evaluation for the purposes of the Home Office study has meant that its impact has been largely judged by the accounts and feedback from a teaching staff who have used the resources. The early findings from the West Midlands evaluation are based on pre-use survey returns.
- A questionnaire (paper/electronic copy) was sent with the primary school resource to 80 Birmingham schools at the end of October 2012. The pre-use survey returns were received from 18 teachers (17 schools) by mid-January with a target of 20% return i.e. 20 schools.
- Under the Home Office contract teachers who had been sent resources were emailed a link from March 2012 to submit data regarding why and how they planned to use the resource; subsequently returning by completed questionnaires on the use and impact of the resources, by the end of the summer term (July 2012). This was supplemented by telephone interviews with teachers from schools who had not returned an online survey, in the summer term.
- West Midlands
Pre-use of resources:
- In total 18 teachers completed a survey on how they planned to use the resources (from 17 schools). Those responding to the survey expected to use the primary resource with 800 pupils. The resources were to be used with school children in Year 5 (only one school planned to use across Years 3-6).
- The results show that the subject matter and topics covered by the resources were the most important factor in schools choosing them. This was closely followed by the fact the resources are offered in partnership with police and schools. That the resource was ‘something new’ and its ease of use were equally ranked together but less important than the topic and partnership between the police and fire service and schools.
- All schools thought that the resource would improved awareness of personal and safety issues (100%) and that it would also result in better engagement (all agreed, except 2 schools that were “don’t know”). Around half thought it would improve knowledge retention (53%) with a quarter believing it would not improve retention (24%), the rest “don’t know”).
- Schools were also asked whether students at the school know about or have experience of knife crime and gangs. Around half of schools (56%) thought that their pupils/students ‘had little or no knowledge of issues around knife crime and gangs’. Just under half (44%) of schools that that some students knew about or had experience of knife crime and gangs, and only one school responding that a large number of students have extensive knowledge about knife crime and gangs.
- Finally, all schools were planning to use the resource in the two terms that followed distribution, with one planning to use the resource across the school year.
Of those teachers above who completed the pre-use questionnaire, they also surveyed and collated the responses of pupils on their attitudes and experiences.
- Being safe:
- Do you feel safe around the local area?
- All the time (24%); Sometimes (53%); Never (13%)
- Do you feel safe online?
- All the time (50%); Sometimes (41%); Never (9%)
- Who is the first person you would talk to or ask for help if you are worried about something?
(NB one school did not complete this section accurately, therefore total 16 school returns)
- Friends (11%)
- Brother/sister (9%)
- Parents (45%)
- Teacher (8%)
- Local police officer (3%)
- Myself (22%)
- Do you think it is okay to carry a knife if you are being bullied/picked on?
- Yes (6%); No (82%); Don’t Know (12%)
- NB (2.5% Yes) are from a single school; average of other schools is (4% Yes)
- Do you think it is okay to carry a knife if someone you know asks you to?
- Yes (7%); No (77%); Don’t Know (16%)
- Do you think it is okay to carry a knife to impress your friends?
- Yes (5%); No (86%); Don’t Know (9%)
- NB (4% Yes) are from same single school as above; average of other schools (1% Yes)
- Do you think being in a gang is a good way to stay safe?
- Yes (30%); No (43%); Don’t Know (27%)
- Do you think some people have no choice but to join a gang?
- Yes (40%); No (33%); Don’t Know (27%)
- Home Office (2012)
Pre-use of resources:
- In total 28 teachers completed a short online survey on how they planned to use the resources. Those responding to the survey expected to use the MissDorothy.com resource with 4,215 pupils and the Watch Over Me resource with 680 pupils/students. The resources were mainly to be used with school children in years 3 to 6, aged 7 years to 10 years.
- The results show that the subject matter and topics covered by the resources were the most important factor in schools choosing them. This was closely followed by their ease of use, whilst using a new resource/something different was much less important.
- All schools thought that the resource would improved awareness of personal and safety issues (100%) and improved knowledge retention (100%) whilst nearly all thought that it would also result in better engagement (92%).
- Schools were also asked whether students at the school know about or have experience of knife crime and gangs. Around two-fifths of schools (41%) thought that their pupils/students ‘had little or no knowledge of issues around knife crime and gangs’. Just over half (51%) of schools that that some students knew about or had experience of knife crime and gangs, with around one-in-six schools (17%) responding that a large number of students have extensive knowledge about knife crime and gangs.
- Finally, schools were asked when they intended to use the resources. Around two-fifths planned to use the resource that term (43%) whilst a slightly higher proportion planned to use it ‘sometime during the next term’. Around one-in-ten schools (10%) did not know when they would use the resources.
|Home Office evaluation||%||West Mids %|
(comparison)Do you feel safe around the local area?All the time 51 24Sometimes 39 53Never 9 13 Do you feel safe online?All the time 59 50Sometimes 34 41Never 7 9 When you are worried about something will you talk to someone or ask for help or will you deal with it yourself?
(choose one option only)Parents 36 45Teacher 19 8Friends 16 11Brother/sister 16 9I will deal with it myself 9 22Your local police officer 4 3 Do you think it’s ok to carry a knife if you are being bullied/picked on?Yes 4 6 No 91 82Don’t know 6 12 Do you think it is OK to carry a knife if someone you know asks you toYes 3 7No 93 77Don’t know 4 16 Do you think it’s OK to carry a knife to impress your friendsYes 3 5No 94 86Don’t know 3 9
Post-use of resources
- In total 25 teachers completed a short online survey by July 2012 on their opinions about the resource which they used with, in total, around 2,700 children. This reflects the number who committed to using the resources by the summer term. All of the respondents used the resource ‘with the whole class as part of an on-going theme’ with two teachers (8%) stating that they also used it ‘with a targeted group of pupils deemed most at risk/vulnerable’.
- All teachers thought that the resource had a positive impact on the class. The biggest impact was improved awareness of personal and safety issues (92% of teachers thought this were definitely the case) followed by better engagement (80% ‘definitely’) and improved knowledge retention (64% ‘definitely’).
- All teachers (100%) stated that the resource met their expectations; all but one teacher (“don’t know”) said that they would use the resource again the future and all (100%) would recommend it to other teachers.
- In the open-ended questions teachers responded that what worked best about the resource was the quality of the work books, their ease of use and the fact that they appealed to children.
Telephone semi-structured interviews were carried out with teachers regarding their use of the resource, what they thought of it and feedback from pupils. In total 13 teachers were interviewed at the end of the summer term, June/July 2012 (9 primary; 4 secondary). In addition 2 school visits included teacher interviews, focus groups with pupils and classroom observation.
- All the teachers used the resource ‘with the whole class’ regularly, within PHSEE/citizenships lessons; some also using it as a complementary resource to refer to as related issues arose in other lessons and by pupils. One teacher uses the resource with excluded pupils.
- Summary feedback includes:
- What worked best about the resource? – it caters for all abilities and there is no discrimination; the lesson plans are straight forward and topics relevant; children can use the resource to work independently; for teachers it has the flexibility to teach by topic or to draw down and split a topic over week(s)
- What were the unexpected achievements? – it encourages independent learning from the children, as they dip in and out of the book when they want to; it provided the teacher with insight into the lives of the children outside of school
- Crime/gang related comments: (primary) – it linked to topical local issues and incidents, with children nervous/under confident; crime safety prompts good conversations, with Yr6 pupils more ‘street wise’ about knives and gangs; improved awareness and knowledge, how to get help, who to speak to and who to avoid to stay safe; (secondary) – used effectively to raise issues and discussions on staying safe, carrying weapons and drugs
- Comments from focus group: (primary) – “it’s really a diary where you can put your thoughts and feelings”; “it gives good advice about what you should do”; “you can write down your problems and (teachers) can help you”
- It is the subject matter and the quality of the educational resource that is most valued by teachers.
- It is an accessible resource, used in a ‘whole class’ group that caters for all abilities and encourages independent learning, and teacher insight into students’ thoughts and feelings
- Mainly used in PHSEE it is complementary to other lessons and as topics arise linked to local issues, prompting conversations and discussion
- Although a relatively small sample, feedback from the Home Office evaluation is of increased awareness and understanding
- Its impact on knowledge retention and effective strategies in recognition of risky situations will need to be reviewed over time; however, that teachers would re-use and recommend use is a strong indicator of its educational value
- The restricted period of the Home Office evaluation limited the number of returns to the survey response, with different time of distribution and use of resources by schools. Where it was a free resource to the school, there was little or no incentive for survey returns in particular ‘post-use’ when the resource was used at a later date in PHSEE lessons and/or complementary use over time.
- However, for the Birmingham schools’ evaluation the resources were targeted at 80 primary schools for Year 5 pupils, under a partnership arrangement with West Midlands Police and Fire Service. That partnership between Police, Fire Service and Schools was seen as an important factor by teachers in choosing to use the resource. .
- The telephone interviews introduced at the end of the summer term for the Home Office evaluation provided an efficient and effective mechanism with teachers’ responses to semi-structured interviews at times arranged when the resources had been used, with dialogue encouraging links to those schools for possible review/interviews at a later date to assess impact over time. This might be considered for follow up with Birmingham schools under the West Midlands evaluation (school visits are time intensive and perhaps best used to develop a small number of in-depth case studies or in building relationships with schools over time where there is greatest need or impact).
- Next Steps: West Midlands
- Reminder notices will be sent to the Birmingham schools for any remaining pre-use questionnaire returns (target: 20 schools; 20% return of total).
- There will be a post-use questionnaire sent to those schools for completion in the summer term, after Easter; included in notices to be sent in March.
- Consideration will be given to telephone interviews in the summer term, where this is used to ensure teacher feedback and broader narrative on the resource’s effectiveness and future use.
As at 28th February 2013
Citadel Policy & Communications Ltd, on behalf of Values Versus Violence
Jonathan Hopkins, Director
Cabinet Office (2004). Evaluation of the missdorothy.com and Watch Over Me programmes.
National Children’s Bureau (2005). An evaluation of the missdorothy.com programme in primary schools.
Office of Fair Trading (2006). Evaluation on the delivery of the missdorothy.com and Watch Over Me programmes
Lancaster Healthy Schools Advisor (2007). Evaluation report of the missdorothy.com learning programme.
Craigatin Psychological Services (2008). The missdorothy.com programme in Harrow: Comparing empathy, self-esteem, and safety perceptions in two London Schools.
Craigatin Psychological Services (2009a). Building safety awareness: Do the missdorothy.com and Watch Over Me programmes work?
Craigatin Psychological Services (2009b). Promoting empathy and self-reflection: Why do the missdorothy.com and Watch Over Me programmes work?