Educational Technology Blog
An academic research paper (Palmer and Devitt, 2007) discussed multiple choice questions in a very positive light:
“Well-constructed peer reviewed multiple choice questions meet may of the educational requirements and advocate that this format be considered seriously when assessing students ”
Some may doubt the validity of this paper, as it is over a decade old. However newer studies agree with that well-crafted multiple-choice tests and quizzes are of benefit to staff and students alike (Abdulghani et al., 2017)
Indeed the ‘wrong’ answers in a multiple choice test have their own jargon (“distractors”), even questions have their own name (“stem”). However, what is absolutely clear is that we need to develop our questions to test higher levels of cognitive reasoning to discriminate between high and low achieving students.
Xu, Kauerand Tupy (Xu, Kauer and Tupy, 2016)feel there are ways to optimise the development of multiple-choice questions. These ideas are summarised below:
✓When formulating your tests you should consider the following improvements:
✓Develop questions that test higher order thinking
✓Discourage guessing from students, and tell students before hand what is being tested
✓Learn from your exams by analysing question results
✓Use clearly written questions that cover a range of topics
✓Use questions that test the taught content
✓Plan feedback to be immediate, or delayed depending on the context. Consider instant✓digital feedback using computing resources
✓Give students the chance to self-correct
✓Utilise students opinions on the tests you set
✓Avoid negative questions (pick the option that is NOT…) and all /none of the above questions
✓Use 3-choice items
✓Avoid difficult to understand answers like A and B, not C
✓Choices should be similar in length
✓Questions should be as short as possible
✓Randomize answer positions
✓Help students to understand why cheating does not help them
✓Randomize question order and answer positions to minimize cheating, and change the questions each year
So using multiple choice questions is a good idea, but only if we do them well. We need to think about how we will develop them to be beneficial for both students and teachers.
Abdulghani, H., Irshad, M., Haque, S., Ahmad, T., Sattar, K. and Khalil, M. (2017). Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study. PLOS ONE, 12(10), p.e0185895.
Palmer, E. and Devitt, P. (2007). Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Medical Education, 7(1).
Xu, X., Kauer, S. and Tupy, S. (2016). Multiple-choice questions: Tips for optimizing assessment in-seat and online. Scholarship of Teaching and Learning in Psychology, 2(2), pp.147-158.
Last year Prince Harry said that teachers go beyond text books and this year public attitudes.
This year the Global Teacher Status Index for 2018 put the status of UK teachers below those of Malaysia, China and Greece.
Could the two be linked? When celebrities or those without domain knowledge publicise their interpretation of what teaching should be they degrade the profession. When added to the pressure the government puts on teachers to expand their role into mental health, and do more than less it seems the outcome is self-evident.
Originally published by Mark Jones on Flickr
Bill Gates, are you serious?
Bill Gates has spoken about education. Speaking principally about the US education he speaks of his dream about good teaching:
“you can take that top 10% and have them help the others to get best practices, the best teaching ideas to spread over the country”.
It sounds great! Spreading best practice works in business, and surely this is a great idea that should be adopted.
The problem is teaching is not the same as a business process.
- Best practice is not always transferable
A teacher working in a deprived area has a very different job to a teacher working in a Grammar school in a leafy suburb in the UK. The job is very different, in terms of the information presented, format but also the expected achievement of the students. Much of teaching is getting students to follow you and understand the content on a personal level. How can you spread this as best practice? There are elements of acting in teaching so should we get drama students to watch Nicolas cage (who is in the top 10% of actors) to learn how to emote? That is, clearly not going to work as although Nicolas Cage is a famous actor, he cannot act.
- The assumption that the 90% need to improve
An outstanding inspirational teacher can help a student in areas outside their subject, but may not deliver the best results within their subject. How do we measure the extra that the best teachers give? The 90% set the environment in the school for the 10% – a teacher great at discipline gives the next teacher in the day a settled class ready to learn. They may not be measured as the top 10%, but are essential for the running and discipline of the school.
- The measurement problem – we don’t know that the 10% is better than the 90% in We don’t know what a good teacher looks like in terms of exam grades.
Students are often measured by their ability by a previous teacher, or school. Secondary schools rely on data given to them about students when they are 11, and measure their exam performance when they are 18 against this data. This data is frequently inaccurate when the student is 11, but also this data can go out of date over time.
So: Can you come up with a better strategy?
Let us have a look at some of the best educational systems in the world.
Finland routinely tops global rankings, and there a number of ways that they achieve this
Teachers in Finland are well respected and paid. The best graduates consider being teachers. This is actually more like successful businesses, where the best companies consider recruitment and retention to be essential to their organisation. In both the UK and the US it can be argued that teaching is a second-class graduate destination, and certainly the salary does not support the best graduates joining the profession. Teach First attempted to attract the best talent, but the clue is in the name – Teach First means that the graduates are expected to teach as a young graduate before they embark on another career, taking any expertise with them. Teach First graduates have higher recruitment costs, inevitably lost long-term as those teachers leave.
As an alternative could look (as is commonly done) in Shanghai
Shanghai is effectively a city-state run as an SAR by the Chinese government, but it should be clear that Shanghai is clearly part of mainland China. The difference between the west and China is that students respect teachers, and are taught to from a very young age. A poor test score commonly means a student is physically assaulted by their parents. Students are taught to never question a teachers knowledge, even if a teacher makes a mistake.
To copy this strategy would require a cultural change in the society in the US or UK. In another Chinese-influenced state Singaporean students are put under incredible stress to achieve high grades (although since suicide is illegal it is difficult to measure the most drastic solution students may take)
We have to change something, and although money is tight underpaying teachers means that we are not getting the very best to be teachers. Certainly easier than changing societal norms..
While it is good to see this increase in prestige and focus for both Computer Science and Maths (which has been promised investment in teaching and delivery) an obvious question is to be asked: where will extra the extra 12,000 teachers come from?
The extra 8,000 teachers at a cost of £84m works out as £10,500 each. Clearly this does not match with the current £26,000 bursary. If this money is intended to go into training teachers we may look at the Computing at School Master teacher initiative that has struggled to recruit enough computer science experts.
Perhaps the money is to go to welcoming teachers who have since left the profession? The numbers of computer science teacher returning would not fill a classroom, meaning that those who leave the profession are not rushing to come back.
It would appear that the money is intended to be used to upskill existing teachers. This may be money well spent as qualified, competent teachers delivering the ICT syllabus have struggled to switch to computer science. In past years teachers asked to do so have invested vast amounts of their own time to the switch, or worse left the subject or the profession. In any case, students have suffered.
It should be noted that money used in schools to take teachers out of school the fee for cover can be as high as £250 – before fees for the actual training (some schools internally charge for staff whether or not external cover is needed). So how many days training would this new money support?
Worse this might be seen as too little too late. Many schools have struggled to recruit computer science teachers, and some are no longer delivering the subject above KS3. Who wants to retrain for a subject with no future, and that involves a large commitment in time and schools do not support?
Without more detail, it is hard to fathom the Government’s strategy and we will have to wait for more information. We hope it comes soon.
The iPhone X launched last Friday for the incredible price of £999 in the UK, and that is before you take into account a case, calls or Apple’s AppleCare + insurance. Apple are an amazing company and make large margins with their iPhone. Can they make such an expensive phone so expensive?
The thinking behind the X’s (pronounced ten) is nothing short of genius. Announced at Apple’s September 12th Event, the new generation iPhone has no home button and a high-resolution edge-to-edge display but was not available until Friday, November 3rd. Consumers could then choose between a more regularly priced iPhone 8 at £699, a price consumers are used to paying for an iPhone.
The advertising has been nothing short of impressive, telling of the new phone’s exclusive features and encouraging demand. These elements were said to be usual for a new iPhone launch, but many people said that they would not be sufficient for the total cost of the phone.
The difference with this current iPhone was not the early marketing, but the high price. How did Apple adjust the market towards a higher price bracket than the market was used to?
One part of the strategy is for Apple to benefit from their fans who would pay almost any price for a new phone before anyone else. Previously these buyers would go to eBay and pay above the standard price for a new phone. Now Apple have a product that can “price gouge” these fanatical consumers with a product aimed at them – with the money going to Apple rather than a reseller.
Apple have been smart in how they have sold the iPhone. As yet we have no information about how many have been sold, other than that the sales have been “off the charts” as announced by CEO Tim Cook. On the first day that people could reserve and pre-order the phone on the Internet, however if you did not get an order for your phone logged in less than an hour you would have a delivery date 4 or 5 weeks in the future.
We then saw long lines of people waiting through the night before launch to get their hands on the new phone and supplies sold out of all store around the world.
Apple are great at under promising. This means when they say you are due a delivery in 1-2 weeks, it usually comes after a week. Deliveries are rarely late, and have happy customers (Amazon often use the same trick). When some people began to have their order date pushed forwards it was thought to be just a few people were getting their phones early due to Apple making sure they meet their promise.
After release, Apple make some iPhone X units available at Apple Stores with same day pickup. The suspicious part is that they managed to do this BEFORE shipping orders to their customers who had already ordered from the Internet.
Slight of hand
Apple were shipping to new customers before fulfilling their existing pre-orders. How have they managed to do this without upsetting the customer that they, are after all, letting people skip the queue ahead of some of their most loyal and obsessed customers. The key is with their under promise – they are still going to deliver to those customers and before the date they promised. The key part to this is that the customers are complicit in this, they agree to Apple’s actions.
Apple have done that amazing thing. They have turned their customers into fanatics – fans – who will support Apple through thick and thin. They listen to their customers but they also share their values and thoughts with their customers and this has brought unparalleled success to the company.
But how many of their presales (when they release the numbers) will be people who have ordered from the Internet and then picked up another order?
Resources by StudeApps
iPhone X Marketing in Business Studies (1BS0) theme 1
iPhone X Higher Lower game
Computing, Citizenship and tutor time:
iPhone presentation and lesson resources
A minister is apparently asking teachers at Universities for lists of lecturers who are teaching about Brexit (source: here). Perhaps the comparisons to 1984 and McCarthyism are overblown (although the principle that this Government may be benign, but the idea that their successors are as yet unknown still stands).
Rather the disturbing principle is rather part of a wider encroachment into academia and education in the vein of the Prevent duty. With the latter law, teachers are expected to look for activities that do not align with British values and report potential terrorist activity. The training for such a duty has been as ill-thought out as it is dire but that is not the most worrying part.
The idea that teachers are able to identify people who are enemies of the state does sound dystopian. However, assuming the requests are benign how does are they expected to identify such people? Under what criteria?
Imaging a stereotyped science teacher who is entrusted with a mixed class. After studying for his PhD and working in industry, he moves into teaching and works as unqualified teacher in an academy.
What experience is he using to decide which students are displaying attitudes not compatible with British values? What if he is bringing prejudice and stereotypical views into his analysis of a situation, maligning disadvantaged students in his care?
If we ask teachers to be the first line of defence we run a very thin line of acceptability. Rather like asking landlords to check passports when they rent out a spare room in their house. Would you be able to identify a fake New Zealand passport; you are now expected to make a decision based on your experience.
You may not be able to sleep well at night if you think about this too deeply. Those that want to be endowed with this type of power are exactly those who should not be entrusted with it. Those that are watching and checking up on us are suddenly given legitimacy and power.