Advantages Of Objective Test Items 3v427

Warren Loyd E. Pacana BEE-4 ADVANTAGES OF OBJECTIVE TEST ITEMS 

Survey a wide variety of material, in very little time o

SAMPLING --students can write 60 multiple choice in an hour but only 1 essay

o

means they are generally more reliable CAN'T BLUFF their way around the question --> teachers hate giving 0 for

essay even when it doesn't say anything on topic 

Writing skills, spelling and neatness don't get in child's way



my friend the dyslexic loves multiple choice.... by same token, spelling, neatness don't get in marker's way either



old Social Studies exams, 50% of the variance in student's mark could be ed for by spelling and mechanics even though these were not supposed to matter -- so prejudicial



Can use statistical analysis on individual questions or groups of questions to identify:



strengths and weakness of test (before you hand it back)



specific strengths and weakness of individual students



topics that need to be retaught; areas where course needs work



Unbiased in sense that teachers' preconceptions of student's work can't influence marking



Answer key is unambiguous so two observers come to same conclusion, unlike essay where same essay can go from 0 to 100% "there's that right-wing jerk arguing in favor of Hitler again"



Fairly useful as pre-tests



since it is fast and you're probably not looking for higher level thinking so much as knowledge of the area



or diagnostic testing where you can reuse year after year because individual not class



Fast to mark:



so you can do large groups, especially when the teacher's time is limited [but then again it’s a pain in the behind to write, so it balances out in the end]



Cheap to mark--> machine score --> why government uses them so much.

DISADVANTAGES OF OBJECTIVE TEST ITEMS 

Most types are limited to factual recall Not necessarily the case --> I will teach you how to get higher level thinking out of objective questions most people don't have the know-how to make objective items which go beyond mere recall of facts even those who know how, often lack the time to build good ones o



a higher order multiple choice can take 3 or 4 hours per question to write--> not worth the trouble unless you know you're teaching this for 20 years consequently, often low level recall only

Students need the opportunity to practice and demonstrate writing skills ---> if there is an over-reliance on objective testing, students will not have this opportunity. o

Before provincial exams I had a university student who didn't know that a sentence started with a capital and ended a period. When I asked how he got through high school, he said that he only had to write four papers in three years; he failed those, but was brilliant on everything else...now we don't get that problem because teachers have to use essays and therefore the problem fixed before it gets to me.

o

Objective items should therefore be balanced with other forms of evaluation --> Why there is written response on Diploma Exams 



multiple-choice good enough to predict student ability, but need to include to enforce writing on teachers

Objective items often test skills other than those that the teacher intended e.g. --> cause and effect relationships, when teacher thought was straight recall



Can become a straight reading test If time is limited, while student spends little time writing, spends a lot of time reading and thinking.



May be inappropriate for Grades 1-3; grade 4 is iffy o (except True/False) o requires too much reading comprehension and abstract thinking "Piaget says 'No'" o

matching, multiple choice definitely beyond them; maybe short answer

o

MAYBE true/false if careful about reading level -->Ultimately, it may be better to use non-test instruments for these grades



Wording is crucial because only one specific and predetermined answer is acceptable. --> Easy to become "guess what I am thinking" question



VERY time consuming to construct



Reusing tests may be problematic but since time consuming to write, you'll want to reuse them --> item banking is ok o

Suggests that you're not doing diagnostic teaching because you should be custom making exam to current group...

o

means that a "leak" can be devastating

o

case at U of A where student wrote to her old campus to get copy of old exams to "study"



Promotes guessing (gets 25% for choosing all 'a's; 50% on T/F)



Test-wiseness often a problem means a student can get the right answer without really knowing. Not suitable for language skill testing



Regardless of whatever test salesmen try to tell you 

Does not reveal thinking process o

in math, for example, the teacher wants to see WHERE child went wrong as well as if he or she got the right answer

o

if students can't display higher thinking, teachers can't observe it, so limited diagnostic information available

 

Often fail to provide a learning experience for students because only deal with recall Over-reliance on objective item tests will encourage rote memorization Again, there is a danger of sabotaging the neat things you are doing in your classroom if students learn that none of that higher thinking class discussion counts and all you are testing for is names and dates "What year did Picasso begin his Blue period"

Test Advantages and Disadvantages of Essay type of Test Advantages 1. No guessing o have to generate answers rather than select them o students can demonstrate their knowledge within broad limits 2. Allows divergent thinkers to demonstrate originality, creativity o in contrast to multiple-choice which are terrible for divergent thinkers who always chose the answer that isn't in the list of alternatives provided o of course, leaves open danger of diverging a bit too far... divergent thinking can be problem for marking in same way as for short answer Example: famous court case of physics student and barometer. 3. Reduced lead time required to produce o less stuff to type, run-off if caught short o write 'em on the blackboard if you have to 4. Less work to ister for smaller number of students o easier to mark ten essays than to build a multiple-choice test for ten students; advantage decreases with class size o But also have to consider long-term: teaching this course again next year? 5. Can be rich in diagnostic information o one topic in real depth

Disadvantages 1. Impossible to mark objectively o I personally have seen scores go from 0 to 100 per cent on the same paper defensibly so --> creative, original response vs. off topic o that's rare, but not uncommon to vary 35-40% o tons of studies on this area --> fascinating stuff o some markers emphasize content, some style o same essays, but introduced spelling mistakes, made huge difference, even when markers specifically told to disregard spelling o same with handwriting- Not valid measure of social or English if handwriting gets in the way

o

teacher expectations more important than actual performance eg., guy who sent lesser known stories by Dostoevsky, Hemingway, etc. into CBC short story contest...

o o

so "good class" or "poor class" halo effects: student's earlier work prejudices expectations Two students give same wrong answer: -"Johnny probably meant this when he wrote that" -"Susan got that wrong, just like I knew she would"

another kind of halo effect is where in the stack your paper appears:  if read an ok paper after five fantastic standout ones, it looks terrible by comparison; but if you just read 5 really ghastly papers, it looks pretty good 2. even different times of day make a difference o

-late afternoon papers scored lower because tired; but bluffers may do better because too tired to catch 'em out one study said might make a /fail difference to about 10% of the papers depending on time of day marked

3. --> first paper to be read often sets standard --> Or if you decide later to lighten up, the first paper's in trouble teachers who know content are easier graders than those who don't same teacher asked to mark the same paper the next day will often give completely different scores  not just English either --> studies suggest math is just as bad Can sample only limited range of course CONTENT  what you gain in depth you lose in breadth  it is possible to come up with questions that get the student to combine different content BUT make sure that it's YOU doing the combining here, not simply loading it on the student Time consuming for student to write  several students have talked to me about having an essay on their exam and allowing 25 minutes --> that's long enough for one or two paragraphs, depending on grade level, not full-fledged essay  unlike m-c, students not only have to interpret the question, they have to:  recall everything they know, sort out what might be relevant  they have to plan their response  they have to think up a catchy opening sentence --> getting started writing is VERY time consuming in itself --certain inertia here --BLANK PAGE SYNDROME  

o

o

--some teachers taught their students to restate the question in first paragraph, just to get them started, but is now a bad idea because turns markers off then they have to devote time to the actual mechanics of physically writing  they have to stop to think about spelling, or look up words in dictionary Time consuming (and mind numbing) to mark  after reading 42nd essay on Hamlet, your brain will explode  to do a proper job, often end up reading essays twice Contrary to expectation, most essay questions do not get at higher thinking skills  mostly just ask "tell me everything you've memorized"  too often marked for factual content plus style, not displays of higher thinking Encourages first draft approach to writing  good writing requires time to think, organize, write and revise  often, the need to 'beat the clock' discourages reflection, thoughtful answers 

o

o

o

--> Certainly kills revision o

Puts a heavy emphasis on student's writing skills  much of student's time devoted to writing not thinking  poor writers are slaughtered, no matter how well they know content  writing ability has more to do with class/social background than with mental ability  



 

sociologists note it is mostly style of middle & upper class that gets rewarded, even though "ain't never" communicates equally clearly Mike Hall, editor of Ft McMurray magazine wrote Writing Competency Test for university, fails because in answering paper on question "Should there be a death penalty" used the personal pronoun "I" Ultimately, have to evaluate writing or students won't learn to do it, but term paper may be better approach to this  BUT term paper unlimited work, exam setting limits to two hours and then it’s over have to use essay format for objectives that specify write or supply information if you take Bloom literally, can't really test "knowledge" because by definition their ability to incorporate into an essay takes you up a couple of steps to application -- though I suppose it would be possible to count objectives like "naming" as "knowledge" but relatively few opportunities in essay Only use essays when essays are called for