Analysis of the use of ExamReaders at The Manchester College in the GCSE English exam 2019

Nine of our students at The Manchester College who use a scanning pen as a normal way of working (NWW) used a ExamReader in the GCSE English exam.  The cohort comprised of 3 male and 6 female students studying on main curriculum areas ranging from Health and Social Care, Animal Care, Child Care, Business Studies and Beauty.  Interestingly, 7 of the students who volunteered to trial the pen were from the Care subjects, where there is a focus on the availability and take up of support.

The students knew that the ExamReader was currently the only support available, given that the JCQ regulations specify a human reader cannot be used in an exam testing reading.

8/9 of the students responded to a questionnaire.

All students found the pen either easy or very easy to use.

When commenting on how good they thought their own reading was, without using a pen, 2 said it was good, 3 said not bad and three described it as poor.

In terms of how much they used it in the English exam to read out the questions and extracts, 2 said they used it to read most of the exam.

When asked how much better they thought the pen had allowed them to perform, one said a lot better, four said quite a bit better and one said a little better.

6/9 of the students who said they did not use it much still maintained it helped them, indicating that the ability to access just short extracts or a few key words was sufficient to generate a sense of improved performance.

Four of the students used the pen to read single words only, one student read whole sentences / lines only and three used the pen for both methods. 

7/8 students said they would recommend the pen and one said they might.

General positive comments about the pen’s performance are as follows:

“It’s good to use on set words because when you look at it and don’t know what it is and then the pen tells you what the word is, you can understand the text more.”

“It’s been really helpful to be able to know which words I don’t understand.”

“It made me feel more relaxed in lesson.”

 “I liked the fact it read out loud words that I couldn’t recognise.”

 “It helps me understand what is going on and gives me quite a good understanding”

“I didn’t have to ask for help”.

In terms of organisation, I charged the pens up prior to the exams and topped them up after each.

To conclude, for all students, the pen was viewed positively and is a particularly useful device for the students who are actively working towards greater independence.  Students need to ensure they have practised well, appreciate the dexterity required and know how to change the voice speed.

It would seem that as long as it is correctly scrolled along the line, is correctly set up for a right-handed or left-handed person, is delivered on the day fully charged and the student has had sufficient practice, not much can go wrong.

Julie Davis, Assistive Technology Assistant, June 2019

 

The Impact of ExamReaders in Exams for Students with EAL, SEN or Low Reading Ages

Introduction

The aim of this project was to identify to what extent the ExamReader Pens are able to support EAL, and SEND students with low reading ages in their GCSE exams. EAL students with identified SEND are at the greatest risk in regards to low attainment; when compared to EAL student with no recorded SEND, those with School Action, School Action Plus and statements are 16, 24 and 40 NC months behind their counterparts (Strand, et al. 2015). Students with both EAL and SEND are denied access arrangements (for example readers and extra time) during public exams due to their EAL status. However, they are able to make use of ExamReader Pens (for example C-Pen ExamReader) which have been approved by the Joint Council for Qualifications (JCQ) for use in exams without any special access arrangements.

Design

The study took the form of a quantitative experimental methodology which compared the attainment of a control and test group; followed by a small qualitative questionnaire which focused on student perceptions of the support provided by the ExamReaders. There was an initial small scale pilot study (16 students – year 9) followed by the larger study (46 students – year 11). In the pilot study the groups were drawn from a population consisting of year 9 (KS4) computer science students who meet either the EAL, SEND or low reading age criteria. The identified 16 students were paired based on either their SEND status (K or statement), EAL status (fluent or emerging) or reading age (12 or below). From this paired population the students were randomly divided into two groups of 8; test group and control group.

In the larger study a test group of 23 students was drawn from the population of year 11 science students (all of the students in year 11); every student in this population who met either the EAL, SEND or low reading age criteria was included in the test group. This was done to ensure that students (known to have EAL, SEND or low reading ages) taking their GCSE exams would not be unfairly disadvantaged by the study (i.e. denied access to a digital technology which it was believed would improve their assessment grades). Instead the 23 test students were randomly matched with 23 students with a similar prior attainment and gender (without EAL, SEND or low reading ages) to make up the control group.

Intervention

In the pilot study both the test and control groups underwent a series of three short (including a baseline) computer science assessments. The year 9 students in the test group had access to an ExamReader pen, the students in the control group did not. Neither group had access to an exam reader during the baseline assessment.

In the larger study both groups again underwent a series of three science assessments (mock, mock 2 and actual exam). The students in the test group had access to an ExamReader pen, the students in the control group did not. Again neither group had access to an ExamReader pen during the baseline test (mock 1). A qualitative questionnaire made up of Likert scale and open questions was used to collect the year 11 test group students’ perceptions about the ExamReader pens following their actual GCSE exam.

Outcome Measures

In the pilot study the students’ scores from the two assessments were compared to their baseline scores (Table 1). The assessment data suggested that the difference between the test and control groups had a positive effect size of 0.17 (first assessment – low effect size – approximately 2 months’ progress) and 0.38 (second assessment – moderate effect size – approximately 5 months’ progress).

Computer Science

Test Test 2 Baseline Diff 1 Diff 2

Test 3.0 4.9 2.7 0.4 2.7

Control 5.0 6.6 4.6 0.0 2.0

Diff -2.0 -1.7 -1.9 0.4 0.7

SD 3.46 2.17 3.34 2.51 1.73

Effect Size -0.58 -0.79 -0.57 0.17 0.38

Table 1: Year 9 Assessment Data Analysis

In the larger study students’ scores from the two assessments (mock 2 and actual exam) and the baseline assessment (mock 1) where compared for the control and test groups students’ scores (Table 2). The assessment data again suggests that the difference between the test and control group had a positive effect size of 1.03 (mock 2 – very high effect size – approximately 12 months’ progress), 0.12 (actual exam – low effect size – approximately 2 months’ progress) and 0 (mock baseline – no effect – approximately 0 months’ progress).

Science

Baseline Mock2 Actual

Test 1.26 1.74 2.74

Control 1.26 0.85 2.59

Diff 0.00 0.89 0.15

SDev 1.08 0.87 1.29

Effect size 0.00 1.03 0.12

Table 2: Year 11 Assessment Data Analysis

Table 3 shows the questionnaire results where 41% of the 23 (test group) year 11 students used the ExamReader pens in all or most of their exams; 36% used the ExamReader pens in some of their exams. 71% of the students who made use of the ExamReader pens during their exams agreed or strongly agreed that they helped them to understand more questions. 59% of these students agreed or strongly agreed that they helped them to attempt more questions. The remaining students in each instance neither agreed nor disagreed.

All Most Some None

ExamReader use in Exam 2 7 8 5

9% 32% 36% 23%

Strongly Agree Agree Neutral Disagree Strongly Disagree

Understand 1 11 5

more questions 6% 65% 29%

Attempt 3 7 7

more questions 18% 41% 41%

Table 3: Year 11 Questionnaire Data Analysis

The students were very positive about the ExamReader pens in the open question section of the questionnaire describing them as: "A good piece of technology that can help a lot of people”; “They are brilliant for words you don't understand"; "It was good, the school should use them again for the next year coming up"; "Alright I believe that the ExamReader pen helped a lot in the exam and I am happy I get to use it in my exam"; "It helped me read the words I didn’t understand"; and "It helps to understand words your (sic) not familiar with".

Results

In the pilot study the students in the test group (using ExamReader pens) made more progress from the baseline test to the first test (0.4 marks) than the control group (0 marks); and from the baseline test to the second test (2.7 marks) than the control group (2.0 marks). These results and the data outlined in the previous section seem to suggest that the ExamReader pens were able to support and have a positive effect on attainment for EAL, SEND, and students with low reading ages.

In the larger study the students in the test group (using the ExamReader pens) made more progress in their second mock exam (0.48 grades on average) than in their first (baseline) mock exam; compared to the control group who made less progress (-0.41 grades on average). There was a difference of nearly 1 grade (0.89 on average), with the test group outperforming the control group. The test group made more progress in their actual GCSE exam (1.48 grades on average) than in their first mock exam; compared to the control group who made slightly less progress (1.33 grades on average). There was a slight difference of 0.15 grades on average. Although, this difference seems slight, it is important to note that none of the students in the control group have EAL, SEND or low reading ages. Therefore, any progress made by the test group towards matching the grades of the control group is very positive. The ExamReader pens appear to have ensured the academic progress of vulnerable (EAL, SEND and low reading ages) students.

Percentage progress from Mock to Actual - Control

1 2 3 4 5

1 7 7 5 1

4% 30% 30% 22% 4%

Percentage progress from Mock to Actual - Test

1 2 3 4 5 6 7 8

1 3 5 9 2 0 0 1

4% 13% 22% 39% 9% 0% 0% 4%

Table 4: Year 11 Percentage Progress

Table 4 notes that in the control group 30% improved by 3 grades; 22% improved by 4 grades; and 4% improved by 5 grades between their mock and actual exam. In the test group 22% of the students improved by 3 grades; 39% improved by 4 grades; and 9% by 5 grades between their mock and actual exam. In general, the students in the test group with access to the ExamReader pens made higher levels of progress.

Conclusions

The results of this study seem to support similar research from Garner Education Services Ltd (2016) which concluded that ExamReader pens can support dyslexic students in achieving an improved grade in their GCSE English Reading paper; (5 out of the 6 students, who undertook the paper with the assistance of the ExamReader pens, achieved a real increase in their results) whilst also improving emotional well-being, improved confidence and attitude to learning. The results of the Higgins and Raskind (2005) study indicated significant and moderate gains in performance when students were able to use the ExamReader pen.

It has been suggested that characteristics associated with SEND may block a student’s access to the content of the test (Thurlow, et al., 2009), with the result being that students are not able to show their knowledge and skills simply because the assessment itself has created a barrier to doing so (Thurlow, et al., 2010). This study indicates that ExamReader pens can benefit students with EAL, SEND and low reading ages by providing them with a means to independently undertake exams. The students can take an exam in the same room as their peers by plugging in the headphones. Furthermore, the ExamReader pen can be used during public exams (GCSE’s) without any special access arrangements and can provide support to all EAL/SEND students. It is important that students are given time to familiarise themselves with the reader pen technology (Thurlow, et al., 2010).

References

Strand, S., Malmberg, L. & Hall, J. (2015) English as an Additional Language (EAL) and Educational Achievement in England: An Analysis of the National Pupil Database.

Garner Education Services Ltd. (2016) The C Pen Exam Reader – A Quantitative and Qualitative Study.

Higgins, E. & Raskind, M. (2005). The Compensatory Effectiveness of the Quictionary Reading Pen II on the Reading Comprehension of Students with Learning Disabilities. Journal of Special Education Technology, 20(1), p.31-40.

Thurlow, M., Moen, R., Lekwa, A. & Scullin, S. (2010) Examination of a Reading Pen as a Partial Auditory Accommodation for Reading Assessment.

Thurlow, M., Laitusis, C., Dillon, D., Cook, L., Moen, R., Abedi, J., & O’Brien, D. (2009) Accessibility principles for reading assessments. Minneapolis, MN: National Accessible Reading Assessment Projects.

Royal Air Force Cosford

Your name:

 Stephen Hern

Your role: 

Specific Learning Difficulty (SpLD) Advisor

 Your company name:

 Royal Air Force Cosford

 Describe your company:

Military trade training environment. A range of trades including, aircraft mechanic/technician, communications technicians, physical training instructors, officer cadets and photographers. Age ranges from 17 – 55.

Did you trial the ExamReader initially, if so, what “review process” did you follow and what conclusions did you draw?

Yes. We used 2 Students who have previously had ‘readers’ within their previous schools/college. We then got them to trial them for 3 weeks and then give their feedback on their experiences.

What do you most like about the ExamReader?

The feedback we got was very positive. The best thing being that they could replay the spoken text as many times as they wanted without the fear of embarrassment they would have got with a real person doing the reading. The charging time and length of use once charged was also positively commented on.

Please describe how you are using the ExamReader?

Students can use it for their everyday studies and then within any examination.

Please describe how the use of the ExamReader has affected your students (performance, confidence, independence etc)?

Students who have previously failed exams not using the pen have subsequently passed the resits. Confidence is the biggest change and the feeling of not worrying about lengthy exam questions.

The biggest change is not having to get another member of Staff to assist within exams and subsequently no need for an extra room being allocated.

 How likely are you to recommend the ExamReader to others?

Very likely.

Any other comments you would like to add ?

I have passed all the information to other RAF Units who may feel they can benefit from them.

 

 

 

Ayrshire College

IAIN STRACHAN

Inclusive Learning Officer

Ayrshire College (Kilmarnock, Ayr & Kilwinning)

Ayrshire College has three campuses across the councils that make up Ayrshire; East Ayrshire, Kilmarnock campus, South Ayrshire, Ayr campus and North Ayrshire, Kilwinning campus.  This college has students ranging from 16 to mature students of any age.  We also have school students who attend twice a week.

Across the 3 campuses we cater for circa, 11,000 students. Here at the Kilmarnock Campus around 20 students have access arrangements which involve the ExamReader Pen.

As an Inclusive Learning Officer at Ayrshire College I am always trying to keep abreast of any assistive technologies that can help students with various learning difficulties.  I came across the ReaderPen, specifically the ExamReader at an Assistive Technology event at the Golden Jubilee Hotel in Glasgow on the 15th June 2017 and was immediately excited about how it could be used to assist students. Many students with difficulties with processing information and students with dyslexia can sometimes misread questions during exams.  This can lead them to write a great answer to the wrong question.

On returning back to Ayrshire campus I brought this technology to the attention of my team leader who spoke to the Head of Inclusive Learning.  They both agreed that this technology was something that the college should invest in.  Now, at the end of 2018, the three campuses that make up Ayrshire College have at least half a dozen of these devices, both ExamReader and the ReaderPen each.  These devices are very popular with the students.

I am now presenting AT workshops across the campuses to different classes to inform all students, not just those with needs assessments, of the assistive technologies that are available.  One of the most popular items during these presentations is the reader pen.  Students like the compact size and how it can be set up for left and right-handed users.  Additionally, the fact that it is quite discreet and can be used with headphones was something the students commented on.

Student A, studying IMI Motor Vehicle at the Kilmarnock campus, uses the ExamReader. This student said this technology was a ’miracle’. They added it also boosted their confidence because they knew they would not get the words mixed up, so knew they were answering the question asked.

Student B, studying HNC Coaching & Developing Sport at the Kilmarnock campus, thought it ‘really helped’.  They said it made answering set questions clearer as they knew they would not misinterpret the information.  This student also liked the fact they could sit in the class with their peers and do the assessment without being taken out due to the headphones.

This equipment could be adapted in various ways but one that students have brought up is they would like to be able to use it on their laptop or desk top screens.  I don’t know how viable this would be.

I find this assistive technology a great learning tool and would definitely recommend it to other schools and colleges.   We are always looking at ways to make students more independent learners and this technology definitely helps them to achieve this.

 

 

Boldon School

Your name

Nicky Korn

Your role

Examinations Officer

Your school name

Boldon School

Describe your school (type, age, location, demographic etc)

Secondary Mainstream. Top 5% nationally for deprivation.

Did you trial the ExamReader initially, if so, what “review process” did you follow and what conclusions did you draw?
Following a demonstration of the ExamReader Pens at an Exam Course in Leeds in October 2017, the school signed up to trial the Exam Reader Pens and as the students had success in using them during their Year 11 mock examinations, we decided to make a larger purchase. It enabled students who find asking for help quite difficult to be much more confident during examinations.

What do you most like about the ExamReader?

That Learning Support Assistants (LSA’s) as a resource do not need to be taken ‘off timetable’ and away from students they usually to support for the assessments, students can use the ExamReader pens instead.
Access Arrangements do not need to be applied.

The ExamReader Pens give students independence and ownership of their own learning. The confidence that the ExamReader Pens give the students.

Please describe how you are using the ExamReader?

Students use the ExamReader Pens for assessments and in some lessons, during the school year and before assessments to get them used to them.

Please describe how the use of the ExamReader has affected your students (performance, confidence, independence etc)?
Students are more confident. We are making students less reliant on a member of staff and therefore promoting independence and giving them ownership of their own learning.

Please describe any other areas in school you feel the ExamReader (staff time savings, school results, cost savings etc)
As previously stated, by using the Exam Reader Pens this cuts down on the number of LSA’s we have to allocate to exam reading, therefore ensuring students who are not sitting assessments across the rest of the school have their entitled support.

How likely are you to recommend the ExamReader to others?

I would whole-heartily recommend the ExamReader Pens to other schools.

Any other comments you would like to add?

As well as promoting independence the ExamReader Pens have empowered the students who struggle to read and have given those students the confidence to achieve positive outcomes.

Priory Community School

Your name:

Amanda Sheppard

Your role:

Exams Officer

Your school name:

Priory Community School

Describe your school

11-16 age range. @ 50 years old. located just outside Weston-Super-Mare, Somerset.

Did you trial the ExamReader / Reader initially, if so, what “review process” did you follow and what conclusions did you draw?

We did a trial using some SEND y11s. they were allowed to take them home to try to get used to them more quickly. The SEND dept. reviewed each student, it didn’t suit everyone, some students just didn’t get used to them.

What do you most like about the ExamReader?

That students can sit in a class with others and that they are able to use it in the parts of the English exams where readers are not permitted.

Please describe how you are using the ExamReader?

They are used as part of a student’s normal way of working. I am told which students are using them and when it comes to the exams and mock exams, I make sure that they are laid out on the exam desk ready. The student brings in their ear plugs.

Please describe how the use of the ExamReader has affected your students (performance, confidence, independence etc)?

The pen has made them more confident. They are able to sit in the same exam room as their peers and not be singled out as being different.

Please describe any other areas in school you feel the ExamReader has helped (staff time savings, school results, cost savings etc)

The exam reader certainly saves staff time and money, we can use less invigilators for exams as we use less rooms. Turing class time the SEND teaching assistants can be deployed around the entire room instead of having to stay with mainly one or two students.

How likely are you to recommend the ExamReader to others?

I have already recommended them to my local exam officer network group.

Wiltshire College

At the beginning on the year Wiltshire College was kindly trialled a ReaderPen and an ExamReader pen at each campus. We trialled using them by identifying students who accessed our Learning Skills and Development Service (LSDS) whom we thought may benefit from their use. We decided to trial them with a few students we knew who struggled with their reading, but who did not necessarily qualify for a reader in their exams, or for those who disliked using a human reader. They were trialled on a variety of ages and abilities across many different courses and across our four main campuses.

Largely the feedback was positive and students who had used them previously commented on how improved they were from the pens of old.

The main positives were; the ease of their use, how they looked fairly inconspicuous i.e. they did not stand out like a human reader would, and that the student felt so much more confident in being able to work independently rather than relying on another human to read for them (which, at times, made students feel self-conscious).

One of the major successes came from our Trowbridge campus whereby three mature students studying hairdressing used them for revision and exams. Having been out of education for some time, Students A, B and C had returned to study a Level 3 Qualification. All three students had struggled previously at school and, although all three were technically and practically very able, they struggled to access the carrier language of the technical exams. They approached LSDS to discuss their needs and the LSDS Assessor advised that the exam reader pen may be a suitable option for them. The students took it in turns to trial both the ReaderPen and the ExamReader pen for accessing class content, revision notes, reading through text books and for practice exams. As the pen became their normal way of working throughout the months leading up to the exam, it was agreed that the students could use the ExamReader in their next exam.

As a result, all students then passed the exam which they had previously failed. Students A, B and C were elated, and their confidence improved tenfold as they were able to demonstrate their true knowledge and subsequently they grew in confidence.

Student A reported –

“The ReaderPen is amazing. I have difficulty keeping information in my brain when I read books and reading with answering questions. This amazing pen helped me free my mind, so I could find the answer that I learned through the year to write the answer down. I would love to invest in this pen to help me read books at home coz I can read a few pages but then my mind stops remembering what was on the previous page. This is a great invention. How would I go about buying or investing in this pen for myself at home to help me for the rest of my future? I am happy anytime to give feedback and reviews of this pen. It is a great product to help dyslexic people. I am very grateful that I have had the opportunity to use the pen and use it in my hairdressing theory exam coz when I was at school I didn't not do very well with exams and this pen helped me open my mind and pass my exam. I know this reader pen with help other people in colleges and schools and great if it becomes a product to buy for home life too. Thank you very much for letting me test this wonderful product out. It has made me feel I can learn anything when I was using this pen coz I know it would free my mind to pass exams and make information stay in my brain. Thank you again for letting me try this product.”

To summarise, in my opinion, championing assistive technology such as the ReaderPen has to be the way forward if we want to remove barriers and promote confident, independent students who are equipped with the necessary strategies and tools to ultimately be happy and successful adults within our community.

- Alex Tuong, Specialist Co-ordinator for Exam Access Arrangements, Wiltshire College

Taverham High School

Your name

Pauline Phillipson

Your role

Assistant SENCO

Your school name

Taverham High School

Describe your school (type, age, location, demographic etc)

High School, Age 11-19. Norwich

Did you trial the Exam Reader / Reader initially, if so, what “review process” did you follow and what conclusions did you draw?
We gave the ExamReader to several students and asked them to do past Exam papers. We compared their results with and without use of the pen, and we asked them for their opinion on, ease of use, effectiveness and did they feel that the pen gave them more confidence in the exam. We decided that having the pens would be very useful to our school.

What do you most like about the ExamReader?

Having the pens gives the students the confidence to be in control of their exams. They use them in lessons too which enables them to be far more independent. They cut the cost of invigilators/amanuensis in external exams.

Please describe how you are using the ExamReader?

In internal and external exams, in lessons with students that have EAA e.g. English, History, Classics lessons

Please describe how the use of the ExamReader has affected your students (performance, confidence, independence etc)?
Confidence and independence improved, work productivity increased

Please describe any other areas in school you feel the ExamReader has helped (staff time savings, school results, cost savings etc)
In the class room where several students need assistance with reading and the Teacher and LSA would struggle to get to them all, also a reduced cost of amanuensis for GCSE exams

How likely are you to recommend the ExamReader to others?

Highly likely