Multiple Choice

Multiple Choice ( MC / mʌltɪpl̩.tʃɔɪs /) or German multiple selection, and response - choice procedure is a question technique used in examinations, tests, exams and surveys, in which there are several pre-defined answers to choose from on any question.

It will consequently be closed-ended questions as opposed to open-ended questions where the respondent must enter a clear answer; However, there are also polls a combined form in which additionally or alternatively to the specifications of its own response may be formulated.

• 2.1 SC Review
• 2.2 MC Rating

• 4.1 Multiple Choice in international comparisons
• 4.2 Other problems
• 4.3 Measures against rates

Different formats and terminology

In some disciplines next to " Multiple Choice ", the term " single choice " (SC) will be used. Single choice or single choice is then available for questions where exactly one answer to be selected, while in "Multiple Choice " more answers may be correct. In many contexts, the appointment is that at Ankreuzaufgaben only one answer can always be right, as so self-evident that it is not explicitly stated - for example, in tests at schools and universities in the U.S. or Australia.

There are basically the following selection formats for test items with answer options n and k ≤ n correct answers, ie with n- k is not relevant distractors:

Molding

In electronic forms and GUIs, it is common with round single choice and multiple choice display with square boxes. Instead of a cross, a hook or the like can be set. Alternatively, just on the touch screen or on television (see quiz shows like Millionaire? ) Are selected and shown in the evaluation of correct and incorrect responses by text and background colors or frames and other styles.

In order to distinguish in the evaluation between unselected and unedited answers, two boxes per question are sometimes " true " / " yes" and "not true " / "no" used. This is a group of ruling with the same question.

On paper forms can fully completed boxes are thus counted as a non- ticked as a correction and. Some automatic evaluation methods, however, expect filled boxes instead of crosses marking the answer.

The given answers can cover all possible answers ever completely or only offer one choice. Sometimes a complete coverage is achieved indirectly by that answer is " none of the other answers get it ".

Scales and matrices

If the answer variants represent different degrees of assessment (eg "very satisfied " to "very dissatisfied" ), one of which must be chosen carefully, one speaks in social research is not multiple choice, but by a scaled question method.

Since is in multiple-choice questions in social science researches opinion and not assessed knowledge, there is also often the last answer option "do not know " or " no answer", since subjects often feel obliged to put some cross.

In special applications, marks must be placed in a matrix. So you can realize more possible combinations.

Two-stage testing tasks

In medical school a few years ago a multiple- choice format was common in the first several statements are proposed, of which any number may apply. Then comes the real question, in which only one answer is correct.

With five answer choices in the example, the complexity is slightly higher than in the case of a single right among the four statements, but significantly lower than that of free combinability including the edge cases that none or all of the statements is true, because then there would be 16 possible response patterns. Even with the restriction that exactly one or two statements may be true, there were already 10 patterns. The reduction of complexity thus facilitates in particular the correction and evaluation. In the example, the answer choices are sorted in ascending order according to the number of true statements, but this need not be the case.

Evaluation of test performance

The fair assessment of MC- tasks is not trivial and easily leads to unjust judgments.

This is most evident in a test with only two possible answers per question ( "agree " / " strongly disagree "). Here is a properly set cross evaluated with a point deducted, however, no point for an incorrectly set, achieved a subject without any knowledge by ticking the respective first box on average 50 % of the possible points, and thus according to current analysis, a "sufficient " or "passed" attests. Subjects who are tested with similar questions in an exam without MC are significantly disadvantaged.

Nevertheless in practice MC tests are evaluated in part, in this way ( in error ). The test results thus obtained are then one to two grade levels above conventionally achieved results (so scored "four" corresponds, for example, a "six", ie no demonstrable knowledge ).

In part, the pass mark at a flat rate regardless of the number of possible answers to 60 % also in the awareness of the problem but in ignorance of mathematical relationships. This approach is, however, except for exactly 5 answer box for each question, also faulty ( see below).

SC Review

If exactly one of the alternatives offered right, all the other wrong, the easiest way to a just assessment of false crosses point deduction (penalty ) is to be added: For two offered alternative answers for each question one point each, with three alternatives half a point at four alternatives third point, etc. Unanswered questions and those in which more than a cross was set, remain without judgment, there's been no point and no deducted. In order to always give the volunteers the opportunity to avoid penalty for unanswered questions, are always at least two alternatives ( "agree " / " strongly disagree " ) are offered. Orders such as " Mark the correct statements to " should generally be avoided.

Since the public consideration of the statistical effect is felt by the deduction process again and again as unjustified or even doubted court, a proper assessment can alternatively be achieved by an adapted key points is applied. For the ( common) case that the volunteer for a pass has to prove a knowledge of 50 % of the substance, then results in the following key points corrected:

In consequence of a court judgment but, for example, now used in the North Rhine- Westphalian universities a fixed key, which takes into account neither the number of alternatives yet of correct answers per question and assumes that each task or correct answer regardless of difficulty and complexity with the same many points (namely, a ) is evaluated. The pass mark is usually at 60 % of the total score, but will be forcibly lifted when otherwise the failure rate of first-time participants in an examination would be too high, because this is considered an indicator of an unreasonably difficult exam. To cover both cases, the clef is fix defined by the proportion of correct responses above the flexible pass mark. Fine graded notes such as 1.3 and 2.7 are not fixed, but are usually inserted linearly into the grid, in which case the question is whether the limit for x is x, 0 or x, 3 applies.

MC Rating

In a task are multiple correct answers, as with a number of individual questions, each with two alternatives ( " hits [ not ] to" ) to proceed and to give a penalty of one point for wrong crosses. Non- or double- crossed responses remain without consequence.

There are therefore always provide two boxes with each response alternative. Subsequently, the individual points are added, negative sums are evaluated here as 0.

To determine the weight of the task within the overall examination, the score achieved to the desired score for the task can possibly be converted. If, for example as five response options need to be evaluated in the example shown, the overall task 2 points from 4 points of division ( ie up a false cross. ), 1 point for part 2-3 points and otherwise no point could bring.

Benefits

With these tests, many learning objectives can be ( with the exception of creative services ) requested. In addition, they can be analyzed automatically in a rule. They are therefore used very frequently, such as the IQ test, driving test and various qualification tests. Also tests in schools and universities are sometimes held in this way. In the selection of companies, this test is also popular because it only requires a solution template.

Disadvantages

The ability to tap incomplete knowledge on purely formal notices the right solution, or at least to eliminate individual distractors, is discussed in the U.S. under the term testwiseness ( " testability " ) ( Millman et al., 1965). In poorly constructed tests earlier helped the rule of thumb to cross the longest answer in doubt. A parodic test that can only be solved by purely formal closing, the New York School Board has published: (PDF, 52 kB).

Kubinger (2005) writes about the often underestimated impact of rate effect on the diagnostic validity of MC tests:

The likelihood that an item of test [ test question in MC; Author's note ] just randomly answered correctly and thus " released" is, is obviously greater, the less possible answers are offered. In the currently available test inventory of psychological diagnostics are usually five, namely the solution including four " distractors ". For such tests, the a priori rate probability 1 /5 = 20%, that is, even subjects without any ability according to presupposed would "solve" an average of one fifth of all items. The problem is compounded by the fact that for subjects with at least inferior ability, not all answers are equally plausible, so that by the five commonly one, two, sometimes three according Falsifikationsstrategie [ here corresponds mutatis mutandis in as: exclusion process, see falsification; Author's note ] the right way out of consideration fall, which may increase the rate probability per item individually up to 50%.

Multiple Choice in international comparisons

Multiple - choice questions are also used in international school achievement comparisons such as TIMSS, PIRLS and PISA. Here, the U.S. standard format is used in which usually four to five answers are given, of which exactly one is counted as correct. In German-speaking countries, however, where this task format is seldom used, in the first PISA rounds over 10 % of students have ticked more than one response for each task.

A Canadian study shows that the benefit of North American students who are accustomed to from their school days MC tests, also in course examinations still can be detected.

Other problems

Multiple -choice tests convey ( partial ) knowledge of facts rather than knowledge. The people learn verifying answers instead of solving tasks. A person who has the right answer reliably finds out five possible, still may not be able to solve the task. For example, the subject solves the problem until such time as their solution matches one of the predetermined solutions. Makes the subject not the mistakes that have made the creator of the MC solutions intentionally to generate the wrong answers, the test person can find the right solution, without being able to solve the task itself independently.

Another problem is that the right solution in the statement of MC solutions often missing or supposedly wrong solution for those who are familiar with the matter, the right solution is. In this case, the Frustationspotential to the subject is very large. In the MC- test example for multiple choice in audio engineering (PDF, 63 kB), at least the answers to the first seven questions so inaccurate that you could easily add another response that would outclass the officially correct answer as clearly wrong.

A third problem is the correct understanding of tasks, both by ambiguities as well as by a lack of linguistic skills of the subject. You can always test a blend of expertise and proficiency in the language in which the task was set, even if the latter would not matter in practice, because the task is usually made ​​out of context and not from a task out in practice.

In SC- choice responses to the distractors can be very different in their area to the right answer, some are even deliberately obviously wrong, others wrong only in one easily overlooked detail. However, they are all equally valued, while in a free-text correction for some possibly would be given partial scores.

Measures against the rates

At German and Austrian universities Multiple -choice exams are very common. To prevent the students from the rates that the pass mark is set above the random probability or a negative scoring system used, or both.

The presented variants put either a binary decision ( " true" or " false") advance or single selection.

In the simplest and most common method, each response is received in terms of magnitude the same rating, but correct positive and false negative. This system is controversial, and has also been indicated already in court, because by the devaluation of correct answers is not the knowledge, but the risk appetite of the auditee is evaluated.

Alternatives that will further reduce the positive effect of testwiseness awarded a on amount higher (negative ) score for wrong answers as a (positive ) for correct answers.

Frequently go tasks with Malus worst, with zero points in the overall standings, even if the points total would actually be negative. Thus, the transcript can be kept positive. However, this requires a task in which a task consists of several multiple-choice questions. Such tasks are often used in tests containing mainly other types of tasks.

Legal review

In Germany there are now a variety of judgments, indicating the limits of the use of the multiple-choice method, as it is called in German lawyers. Many judgments relate to an absolute pass mark, which has meant that the number of existing specimens can vary considerably from year to year in part; other judgments relating to the assessment of tasks.

Overall, it is therefore advisable to deal with the case before multiple-choice tasks are designed.

Examples

The number of correct answers is not specified. Right are 1st and the 4th answer. 2, 3 and 5 serve as distractors.

From the text of the question is clear that only one answer can be correct ( the 2nd ).