Educational Technology Blog
A quick post about how the UK government have used and abused the software methodology of Agile. Something similar has happened in education…
Today the BBC fell into the common trap of unnecessary hyperbole and concern around AI.
Starting with confusing robotics and AI (two vastly different subsets of computing), the interview ended in an interviewee asserting that Israel have some form of killing machines that can identify enemy targets without human intervention.
For a start, there was no mention of war conventions (human intervention is required in war!), and we should remember that machines that kills without human intervention have been around for years. We might call such machines “bombs” and “mines”.
When the interviewee spoke about how the killing machines could be “scaled up like Google”, we are talking less about robot AI from the Terminator movie and more about mines across warzones in the last hundred years.
The reason this is so important is that people develop a fear of AI, what is possible and what is going to change.
We don’t expect BBC reporters to know everything about each topic covered on the Today programme, but surely they can do better than this?
An academic research paper (Palmer and Devitt, 2007) discussed multiple choice questions in a very positive light:
“Well-constructed peer reviewed multiple choice questions meet may of the educational requirements and advocate that this format be considered seriously when assessing students ”
Some may doubt the validity of this paper, as it is over a decade old. However newer studies agree with that well-crafted multiple-choice tests and quizzes are of benefit to staff and students alike (Abdulghani et al., 2017)
Indeed the ‘wrong’ answers in a multiple choice test have their own jargon (“distractors”), even questions have their own name (“stem”). However, what is absolutely clear is that we need to develop our questions to test higher levels of cognitive reasoning to discriminate between high and low achieving students.
Xu, Kauerand Tupy (Xu, Kauer and Tupy, 2016)feel there are ways to optimise the development of multiple-choice questions. These ideas are summarised below:
✓When formulating your tests you should consider the following improvements:
✓Develop questions that test higher order thinking
✓Discourage guessing from students, and tell students before hand what is being tested
✓Learn from your exams by analysing question results
✓Use clearly written questions that cover a range of topics
✓Use questions that test the taught content
✓Plan feedback to be immediate, or delayed depending on the context. Consider instant✓digital feedback using computing resources
✓Give students the chance to self-correct
✓Utilise students opinions on the tests you set
✓Avoid negative questions (pick the option that is NOT…) and all /none of the above questions
✓Use 3-choice items
✓Avoid difficult to understand answers like A and B, not C
✓Choices should be similar in length
✓Questions should be as short as possible
✓Randomize answer positions
✓Help students to understand why cheating does not help them
✓Randomize question order and answer positions to minimize cheating, and change the questions each year
So using multiple choice questions is a good idea, but only if we do them well. We need to think about how we will develop them to be beneficial for both students and teachers.
Abdulghani, H., Irshad, M., Haque, S., Ahmad, T., Sattar, K. and Khalil, M. (2017). Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study. PLOS ONE, 12(10), p.e0185895.
Palmer, E. and Devitt, P. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Medical Education, 7(1).
Xu, X., Kauer, S. and Tupy, S. (2016). Multiple-choice questions: Tips for optimizing assessment in-seat and online. Scholarship of Teaching and Learning in Psychology, 2(2), pp.147-158.
Last year Prince Harry said that teachers go beyond text books and this year public attitudes.
This year the Global Teacher Status Index for 2018 put the status of UK teachers below those of Malaysia, China and Greece.
Could the two be linked? When celebrities or those without domain knowledge publicise their interpretation of what teaching should be they degrade the profession. When added to the pressure the government puts on teachers to expand their role into mental health, and do more than less it seems the outcome is self-evident.
Originally published by Mark Jones on Flickr
Bill Gates, are you serious?
Bill Gates has spoken about education. Speaking principally about the US education he speaks of his dream about good teaching:
“you can take that top 10% and have them help the others to get best practices, the best teaching ideas to spread over the country”.
It sounds great! Spreading best practice works in business, and surely this is a great idea that should be adopted.
The problem is teaching is not the same as a business process.
- Best practice is not always transferable
A teacher working in a deprived area has a very different job to a teacher working in a Grammar school in a leafy suburb in the UK. The job is very different, in terms of the information presented, format but also the expected achievement of the students. Much of teaching is getting students to follow you and understand the content on a personal level. How can you spread this as best practice? There are elements of acting in teaching so should we get drama students to watch Nicolas cage (who is in the top 10% of actors) to learn how to emote? That is, clearly not going to work as although Nicolas Cage is a famous actor, he cannot act.
- The assumption that the 90% need to improve
An outstanding inspirational teacher can help a student in areas outside their subject, but may not deliver the best results within their subject. How do we measure the extra that the best teachers give? The 90% set the environment in the school for the 10% – a teacher great at discipline gives the next teacher in the day a settled class ready to learn. They may not be measured as the top 10%, but are essential for the running and discipline of the school.
- The measurement problem – we don’t know that the 10% is better than the 90% in We don’t know what a good teacher looks like in terms of exam grades.
Students are often measured by their ability by a previous teacher, or school. Secondary schools rely on data given to them about students when they are 11, and measure their exam performance when they are 18 against this data. This data is frequently inaccurate when the student is 11, but also this data can go out of date over time.
So: Can you come up with a better strategy?
Let us have a look at some of the best educational systems in the world.
Finland routinely tops global rankings, and there a number of ways that they achieve this
Teachers in Finland are well respected and paid. The best graduates consider being teachers. This is actually more like successful businesses, where the best companies consider recruitment and retention to be essential to their organisation. In both the UK and the US it can be argued that teaching is a second-class graduate destination, and certainly the salary does not support the best graduates joining the profession. Teach First attempted to attract the best talent, but the clue is in the name – Teach First means that the graduates are expected to teach as a young graduate before they embark on another career, taking any expertise with them. Teach First graduates have higher recruitment costs, inevitably lost long-term as those teachers leave.
As an alternative could look (as is commonly done) in Shanghai
Shanghai is effectively a city-state run as an SAR by the Chinese government, but it should be clear that Shanghai is clearly part of mainland China. The difference between the west and China is that students respect teachers, and are taught to from a very young age. A poor test score commonly means a student is physically assaulted by their parents. Students are taught to never question a teachers knowledge, even if a teacher makes a mistake.
To copy this strategy would require a cultural change in the society in the US or UK. In another Chinese-influenced state Singaporean students are put under incredible stress to achieve high grades (although since suicide is illegal it is difficult to measure the most drastic solution students may take)
We have to change something, and although money is tight underpaying teachers means that we are not getting the very best to be teachers. Certainly easier than changing societal norms..
While it is good to see this increase in prestige and focus for both Computer Science and Maths (which has been promised investment in teaching and delivery) an obvious question is to be asked: where will extra the extra 12,000 teachers come from?
The extra 8,000 teachers at a cost of £84m works out as £10,500 each. Clearly this does not match with the current £26,000 bursary. If this money is intended to go into training teachers we may look at the Computing at School Master teacher initiative that has struggled to recruit enough computer science experts.
Perhaps the money is to go to welcoming teachers who have since left the profession? The numbers of computer science teacher returning would not fill a classroom, meaning that those who leave the profession are not rushing to come back.
It would appear that the money is intended to be used to upskill existing teachers. This may be money well spent as qualified, competent teachers delivering the ICT syllabus have struggled to switch to computer science. In past years teachers asked to do so have invested vast amounts of their own time to the switch, or worse left the subject or the profession. In any case, students have suffered.
It should be noted that money used in schools to take teachers out of school the fee for cover can be as high as £250 – before fees for the actual training (some schools internally charge for staff whether or not external cover is needed). So how many days training would this new money support?
Worse this might be seen as too little too late. Many schools have struggled to recruit computer science teachers, and some are no longer delivering the subject above KS3. Who wants to retrain for a subject with no future, and that involves a large commitment in time and schools do not support?
Without more detail, it is hard to fathom the Government’s strategy and we will have to wait for more information. We hope it comes soon.