"Parse Tree" <parsetree at hotmail.com> wrote:
>The SATs are not well constructed. Generally, guessing penalties don't
>work, and there are numerous reasons for this. Firstly, guessing penalties
>rely on a completely random selection, which is rarely the case.
Any improvement you can make on random selection reflects some sort of
knowledge or logic.
>Actual
>selection is dependent on what they've been taught and how they work it out
>on their own.
Duh. That is what the SAT is trying to measure - what you've learned and how
you apply it.
>You know, many teachers teach things incorrectly, because
>they are not qualified in the subject they are lecturing on.
Even if this were true (I don't believe you), that is not the fault of the
SAT, which is trying to measure your preparation for college. Learning the
wrong stuff is not good preparation for college.
>> TIMSS is not designed so that ANY kid could get a perfect score or
>> anywhere near a perfect score, and I've never read a report that suggests
>> that any kid did so. There are easy problems and there are hard problems,
>> and the problems are weighted by the difficulty that they were found to
>> present to the entire test population.
>>That's probably because children who answered every question correctly were
>penalized for not showing enough work, or showing the work that the markers
>wished to be shown.
For the roughly 1/3 of the problems for which data was released, they have
statistics on each question as to what percentage of the kids answered each
answer, and what percentage of the kids made particular kinds of errors on
showing their work.
The material is largely in PDF form, and spread over multiple documents so it
is hard to convey via Usenet. I'll have to be selective to get text only
problems.
Here is one question that kids tended to score poorly on, that works in text:
>D12. Brighto soap powder is packed in cube-shaped cartons. A carton measures
>10 cm on each side. The company decides to increase the length of each edge
>of the carton by 10 per cent. How much does the volume increase?
>A. 10 cm3
>B. 21 cm3
>C. 100 cm3
>D. 331 cm3
Only 31% of the kids in the world got this correct.
For the US, of 2699 kids, 26.9% answered A, 24.0% answered B, 25.2% answered
C, and 16.7% answered D. 3.6% did not answer, and the remainder did
something other than a normal answer.
As a result 16.6% of US girls got this correct, and 16.9% of US boys. By the
nincompoop's standards, the US kids of both genders had negative
understanding of the question. The more logical explanation is that they
understood, but solved it incorrectly.
Pity the poor South Africans, who had only 6.4% of their kids get this
correct, most of the kids choosing A or C.
Another multiple choice question:
>K2. In how many ways can one arrange on a bookshelf 5 thick books, 4 medium
>sized books and 3 thin books so that the books of the same size remain together?
>A. 5! 4! 3! 3! = 103 680
>B. 5! 4! 3! = 17 280
>C. (5! 4! 3!) x 3 = 51 840
>D. 5 x 4 x 3 x 3 = 180
>E. 2**12 x 3 = 12 288
16.4% of US kids answered A, 35.9% answered B, 18.3% answered C, 27.3%
answered D, and 1.4% answered E, with 0.7% omitting the question (and the
rest being invalid answers). In this case, the girls slightly outdid the
boys, with 16.8% correct to the boys 16.0%.
As an example of a "show your work" question:
>K12. A translation maps A (2,–3) onto A' (–3,–5). Under the same translation, find
>the coordinates of B' , the image of B (1,4).
>Correct Response
>10 (-4, 2). No work shown, or only points are shown in a diagram such that
>method cannot be determined.
>11 (-4, 2). Method: A diagram that shows more than points is drawn showing the
>geometrical method used such as mid-point, slope, or change in x- and ydirection.
>12 (-4, 2). Method: The coordinates of the translation vector are (-5, -2); the
>translation vector (-5, -2) is added to B (1,4) to obtain B' (-4, 2).
>Note: If diagram is shown and the translation vector is indicated, also use
>code 12.
>19 Other correct responses with method are shown.
>Incorrect Response
>70 Response incorrect. No work shown.
>71 (6, 6). Method as in code 12 but uses incorrect translation vector, (5, 2).
>72 Method as in code 12 with correct translation vectors (-5, -2) but with error in
>subtraction of negative numbers.
>73 Method as in code 11 with an understandable diagram consisting of more than
>just points. At least one coordinate of B' is incorrect.
>79 Other incorrect responses with method shown. (If no method/work shown, code
>70.)
>Nonresponse
>90 Crossed-out, illegible, or impossible to interpret.
>99 BLANK
52% of kids in the world got this correct, including 37.5% of US girls and
46.5% of US boys. 14.1% of the US kids answers were coded 70, 17.8% were
coded 79. 4.7% of the US kids got 71, 72, or 73 indicating that they had the
right idea. Austrians by comparison had 10.7% get one of the "right idea"
answers, and only 12.9% were coded either 70 or 79. Czech kids had more that
showed no work than the US.
15.9% of the US kids were code 10, which means that they showed no work but
got the correct answer anyway.
>K13. The number of bacteria in a colony was growing exponentially. At 1 pm
>yesterday the number of bacteria was 1000 and at 3 pm yesterday it was 4000.
>How many bacteria were there in the colony at 6 pm yesterday?
>Correct Response
>10 32 000. No work shown.
>11 32 000. States explicitly that the number of bacteria doubles every hour or
>shows sequence (pattern) of numbers of bacteria in 1 hour intervals: 1 000,
>2 000, 4 000, 8 000, 16 000, 32 000.
>12 32 000. States that the numbers form a geometric series with common ratio
>r = 2 OR uses Sn = arn-1 for r = 2 OR uses an exponential equation in the
>general form of y = A(ak) with A = 1000, a = 2, and K = 5.
>13 32 000. Uses an exponential equation involving e such as y = 1000 (ekt),
>k = 0.6931, t = 5.
>19 Other correct responses.
>Incorrect Response
>70 Answers other than 16 000 and 64 000. No work shown.
>71 16 000 or 64 000. Exponential equation or pattern has been recognized
>correctly but there is a numerical error.
>72 Responses other than 16 000 and 64 000 where a correct exponential has been
>used but there is a numerical or algebraic error.
>Examples: sn = arn-1
>y = A (ak)
>73 Responses where the exponential function of the form y = A(ex) has been used
>but a numerical or algebraic error is made.
>79 Other incorrect responses.
>Nonresponse
>90 Crossed-out, illegible, or impossible to interpret.
>99 BLANK
27% of the kids of the world got this one right. 18.1% of US girls and 29.9%
of US boys; 10.9% of US kids got the 71, 72, 73 codes. Israel had more than
50% get it correct, and Sweden just less than 50%
The hardest released math problem was only correctly answered by 10% of the
kids in the world:
>K14. A string is wound symmetrically around a circular rod. The string goes
>exactly 4 times around the rod. The circumference of the rod is 4 cm and its
>length is 12 cm.
>Find the length of the string. Show all your work.
There is a diagram for those who cannot get the idea from the text, which
actually is sufficient to convey the diagram. Imagine a toilet paper roll
with a string spiraling down it |\=\=\=\=| so that the string attaches at the
top of the visible part of the tube at both ends and wraps around exactly 4
times as it goes down the length of the tube.
>Correct Response
>20 Length of string = 20 cm. Method:
>• The surface of the rod is represented as a rectangle 4 cm by 12 cm.
>• Four parallel congruent segments are drawn in the rectangle indicating the
>position of the string.
>• Length of one segment is calculated using Pythagorean theorem
>3 4 2 2 + = 5. Total length of string = 4 x 5 cm = 20 cm.
>21 Length of string = 20 cm. Method:
>• Half of surface of rod represented as rectangle 2 cm by 12 cm.
>• Eight congruent segments drawn in the rectangle indicating position of string.
>• Length of one segment calculated using Pythagorean theorem
>2 15 2 2 +. = 2.5. Total length of string = 8 x 2.5 cm = 20 cm.
>22 Length of string = 20 cm. Method used:
>• Situation represented either by rectangle 16 x 12 with string as its diagonal
>OR by right triangle with sides 16 and 12 and string as its hypotenuse.
>Pythagorean theorem used to calculate length of string 16 12 2 2 + = 20 cm.
>29 All other fully correct solutions.
>Partial Response
>10 Length of string = 20 cm. No work shown.
>11 Surface of rod represented by rectangle with correct dimensions and position
>of string correctly indicated, but numerical error in the calculation of the length
>of string.
>19 All other partially correct solutions with correct method and minor error.
>Incorrect Response
>70 Incorrect answer. No work shown.
>71 Length of string = 16 cm. Argument: It is the same as 4 circles.
>72 Length of string = 28 cm. Argument: "If the string were wound 4 times around
>the same place, its length would be 4 x 4. But since it "moves" along the rod
>which is 12 cm long, we must add these 12 cm to the length of the string."
>73 Estimation methods:
>Length of 1 revolution estimated or stated but not calculated; then it is
>multiplied by 4.
>Examples: 1 revolution is approx. 6 cm long, length of string is 4 x 6
>= 24 cm .
>1 revolution is (4 + 1.5) cm long, length of string is 4 x 5.5
>= 22 cm .
>Length of string must be greater than 16 cm (it would be 16
>cm if it were 4 circles) and/or
>Length of string must be less than 16 + 12 = 28 cm .
>16 cm < L < 28 cm)
>74 String is represented by a curve, e.g. parts of a circle or an ellipse.
>79 All other incorrect attempts with some work shown.
>Nonresponse
>90 Crossed-out, illegible, or impossible to interpret.
>99 BLANK
Only 1.0% of US girls and 6.2% of US boys got this correct. We were tops in
the world in having 56.2% get a code 79. Sweden did best with 23% getting it
correct.
>> TIMSS also included many problems that were NOT multiple choice, BTW, and you
>> could not get full credit unless your work was shown and contained the key
>> steps expected in the solution.
>>This is also bad. While better than multiple choice, it still allows a
>large quantity of discretion. In many disciplines there are numerous ways
>to come up with a solution to a problem. Statistics itself has this.
>>Also, key steps implies a great deal of cultural and related bias.
Not in mathematics and science, which is culture-free (in theory). But see
the answers and codings above.
>The
>amount of work shown by someone confident and familiar with type of question
>is quite a bit different from that shown by someone not confident.
If they get the wrong answer and don't show their work, then they were
falsely overconfident.
>Seems to be quite a bit of memorization for these tests.
Actually there is a couple of pages of formulas at the beginning of the
tests. But you do have to know how to solve problems.
More importantly, to the extent that memorization is required, it is required
equally of all kids, so this does not invalidate the international
comparison. Cultural differences may affect the scores, but that is
precisely the sort of thing that TIMSS was intended to detect - is that
something about American (or whatever) culture or education that aids or
harms ability to solve the problems.
We came up short, but the nincompoop's explanations have nothing to do with
the problem.
lojbab
