18 Oct 2005

Title: PSLE papers set to differentiate pupils

1. We wish to thank the writers in the recent discussions on the flawed mathematics question in the PSLE examination for their feedback and comments.

2. As Mrs Charis Mun (“Two marks for invalid question not way to go”, 13 Oct) points out, the question had a very straightforward solution. All pupils had been taught the basic skills required to solve the problem, given the information provided. The difficulty would have arisen for pupils who chose to check their answers using alternative methods, and found inconsistent answers.

3. Mrs Mun felt that the awarding of two marks to all pupils gave unfair benefit to those who would have been unable to solve the problem even if the dimensions in the diagram were correct.

4. There is no perfectly equitable way of compensating pupils for the flaw in the question. As the question is a multiple choice item, we cannot distinguish between pupils who gave the wrong answer because they lacked the skills to solve the problem, and those who knew how to solve the problem but ran into difficulty when they checked their answer using an alternative method. The best approach in such circumstances is to treat all pupils equally, which is why the SEAB decided to award all pupils the two marks which the question carried.

5. We will study Mrs Mun's suggestion of testing the understanding of mathematical concepts and problem-solving ability separately from computation ability. However, her suggestion of rewarding pupils based on the time they take to complete an examination paper is not feasible. There are many reasons why some pupils take longer than others to complete a paper, besides having a weaker understanding of the subject. Some may choose to check their answers, or examine a question from more angles than others do. Examinations are quite unlike a 100 metre race, where everyone knows exactly how to complete the race and the time taken to complete it becomes the only discriminator of ability.

6. The best approach to differentiate pupils of diverse abilities is therefore to set questions of varying difficulty levels. As Ms Choo Yin Lai (“Prepare children for unusual circumstances”, 13 Oct) also observed, the examination paper cannot be set at too easy a level.

7. Madam Rosalind Lim Kwang Suan (“More realistic time frame needed for maths paper”, 17 Oct) felt that there is insufficient time for most pupils to complete the PSLE mathematics paper. In formulating the design of examination papers, SEAB conducts trials with groups of pupils representing all abilities, and seeks feedback from teachers to ensure that most pupils can cope with the papers within the time allocated. Our examiners also ensure that the calculations required are not tedious by using numbers that could be manipulated without the need for a calculator. Our experience with the PSLE mathematics paper shows that the time allocated for the paper has in fact been sufficient for most pupils.

8. Mrs Mun asked whether the mechanics of computing T-scores leads to different subjects having different weightage in pupils' aggregate PSLE score. Pupils are ranked according to their performance in all the subjects at the PSLE. One way of doing this is to add up all the raw marks obtained by a pupil. However, this will not be desirable as the subject with the largest spread (or standard deviation) of raw marks would have inordinate influence on the rank order of pupils. To overcome this, raw marks in a subject are converted to T-scores (Transformed Scores), so that the mean and standard deviation for all subjects become 50 and 10 respectively. Mathematically, the formula is:

where X is the raw mark; is the mean of the raw marks; and SD is the standard deviation of the raw marks

9. The T-score for all the subjects are then added up to give a pupil's Aggregate Score. Computing the Aggregate Score in this manner underscores the equal importance placed on pupil performance in each PSLE subject.

10. However, for this method of computing Aggregate Score to work well, the mean and standard deviation of the raw marks for a given subject should not be too extreme. They should neither be too high nor too low, as pointed out by Mrs Mun. This situation can be avoided by designing examination papers so that they are neither too easy (resulting in a high mean) nor too difficult (resulting in a low mean), and that there are questions to differentiate between pupils of different abilities well i.e., there is a good spread of raw marks. Examination papers should also be comparable in difficulty from one year to the next so that standards are maintained over time, and teachers know how to prepare their pupils for the examinations.

11. Mr Leung Weiwen (“More care needed with exam papers”, 14 Oct) claims that there was an error in a previous PSLE Social Studies paper. This paper was offered by pupils in the Gifted Education course. We have ascertained that neither the topic nor the specific question mentioned by Mr Leung has been included in the PSLE Social Studies paper.

12. We take great care in setting examination questions. We would like to assure all parents and pupils that in the development of question papers, there are several rounds of checking both of each question and for the paper as a whole. These checks are carried out at every stage of the production cycle of the question paper by different teams of experienced setters, moderators and examiners. Unfortunately, in the case of the flawed PSLE mathematics question, the inconsistency of the data given was overlooked.

13. While even the best regarded examination boards have run into such problems from time to time, we aim for zero error. We will review our processes and introduce additional checks, where appropriate, to prevent any future recurrence of errors.

Raymond Lim
Director, Corporate Services
Singapore Examinations and Assessment Board