"Cary Kittrell" <cary at afone.as.arizona.edu> wrote in message
> <questions. For example, H07 was a four part multiple choice question
where
> <at least 25% of them should have selected the right answer if they had
just
> <*guessed*, yet only 9.3% of them got it right
> <http://christianparty.net/timssphysics.htm> <
> <What would you call this, cary? How about "negative knowledge"? No
other
> <girls managed to score this low, yet American girls managed to secure
that
> <cherished dead last spot multiple times over. G04 was also a four part
> <multiple choice question, yet only 21.6% of American girls got it right,
> <which is 3.4% lower than if they'd just guessed. No other girls scored
this
> <low, and 54% of French girls DID get it correct. G08 was a five part
> <multiple choice question, so 20% would have gotten it correct if they'd
just
> <guessed--yet only 10.9% of American girls got it correct, 9.1% less than
if
> <they'd just guessed. Yet half of Russian girls managed to get it
correct.
> <
> <What in your expert opinion, cary, is responsible for such rampant
"negative
> <knowledge"?
> <
>> Oh that's easy enough. My answer is:
>> QUESTION 1) The current President of the United States, the man
> now sitting in the White House, is:
>> a) George W. Bush
> b) Millard Fillmore
>> comments: for all practical purposes, 100% of typical U.S.
> students will get this one right. Using John's approach, we next
> subtract 50 percentage points for guessing. The result is that
> John will now claim that only 50% of U.S. students got this
> question right. This example shows how John's method under-
> represents group P1.
>
Dead wrong. What is SO difficult about this, cary? If all students get it
right, then you have evidence that ZERO percent of them guessed, in which
case you cannot estimate how many just guessed.
>> QUESTION 2) Modern astronomy and phyics have shown that the Earth
> revolves around the Sun, and has discredited the more ancient
> idea that the Sun moved about the Earth.
>> a) true
> b) false
>> comments: only a handful of science nerds will get the right
> answer here (b); nearly all students will reflexively put down
> "a", which is wrong (no privileged frames of reference).
> Essentially zero percent of students will get this one right. John
> will then subtract 50 percentage points for guessing, and report
> that MINUS 50% of students answered question two correctly, just
> as he got a minus score for eight of twenty-eight questions in the
> 12th grade girls example. The illustrates how John's method
> manages
> to under-represent even the zero-points contribution of group P2.
>
Dead wrong, again! If only 10% got it correct, then 90% got it wrong, which
means you have evidence that they had the WRONG answer, and thus that they
did not guess You can't estimate how many of them guessed if they didn't.
> Question 3) How many species are there in the genus Crotalus?
>> a) 26
> b) 27
>> comments: except for a handful of herp fanatics, all students will
> guess on this one, giving a score of 50%. This is the only case
> in which an argument may be legitimately made for subtracting 50
> percentage points for guessing. This is group P3.
>
You stated this improperly. If a handful of "herp fanatics" would have
selected the correct answer, then the distribution would not have been
50/50. Let's assume that a) is correct and that 60% selected a), and that
40% selected b):
X = total guesses
.5X = correct guesses
.5X = incorrect guesses = 40%
X = 80%
.5X = correct guesses = 40%
Correct answers - correct guesses = 60% - 40% = 20% = those who knew the
answer.
It's not at all correct to state that "an argument may be legitimately made
for subtracting 50 percentage points for guessing", is it, cary?
> (the correct answer is "b" -- assuming you're willing to accept
> that _Crotalus lannomi_, based on a single roadkill in Jalisco,
> is truly a new species of rattlesnake)
>> So, we find that:
>> question correct scoring John's method
>> 1 100 50
> 2 0 - 50
> 3 0 0
>> John's claim that "zero percent of American 12th grade girls solved
> simple math problems" implicitly assumes that all girls were guessing
> on all questions, which is completely unwarranted. This explains his
> puzzlement over why scores were lower than simple guessing would
> produce -- many did not guess, but he penalized them anyhow.
American 12th grade girls scored lower on ONE THIRD of these TIMSS questions
than if they'd just guessed, so they were never "penalized", because there's
no way to calculate guesses.
But when 25% would get the answer correct if they'd just guessed, and only
30% got the correct answer, then the score could be and was *adjusted* for
guesses.
One more time:
X = total guesses
.25X = correct guesses
.75X = incorrect guesses = 70%
X = 93 1/3%
.25X = 23 1/3%
Percent who understood the problem = percent who got the correct
answer -percent who guessed = 30% - 23 1/3% = 6 2/3%
It's not just a matter of subtracting the 25%, which would have been only
5%. The adjustment for guesses isn't intended to be a "penalty". This is
the only way you can view these scores objectively.
John Knight
