Need to develop scientific tests for examinations

---

Continuation from Thursday March 14

need to develop scientific tests for examinations

Any change in the structure, content, methodology and objectives of a subject needs five years notice to the schools. It is only 5 years later that the change is allowed. This is the accepted norm in this case only, at the examination centre the student is confronted with this shocking state of affairs. It is obvious that no pre-test has been executed. At a pre-test all these questions would have got eliminated. I am absolutely correct that this is not a scientifically constructed test. Absolutely it is not objective. The GCE O/L (London) exam is conducted in many countries and every test item indicates its reliability and validity.

Two of the most common defects of the papers are that the test items are not presented in the best order of difficulty and that the overall difficulty of items and consequently the whole test is not such as to produce optimum discrimination among the students tested. The relationship between difficulty and validity of the tests and of individual items is of vital importance; yet has grossly been overlooked by the test constructor. To start an exam with such test items is a hazard. Two perfectly valid items of equal difficulty are quite useless, that is no more use than one item for discriminating persons of different ability. The factor of general ability at the tests and the factor at general difficulty of the tests are really the same factor. To say that item A is more difficult than item B is the same as saying that a person of greater ability is needed to answer. It is important that the test items should be arranged in what appears to be gradually increasing difficultly. As these were not so then some students may not have reached these items and be put off by the difficulty of some earlier items in not taking them seriously. Tests should start with a few easy items in order to give testing confidence.

It is first necessary to survey all the various abilities that are representative of abilities and amenable to measurement, and to ensure that those measure the general factor than specialized knowledge. Consideration of the abilities to be tested is important but is severely neglected.

 It is regretted that descriptions of what is to be tested, so often takes the form of lists of unit. The other glaring defect is that the marking is not objective. Maximum objectivity can be achieved constructing the test scientifically. We should follow the 6 principles advocated by Cyril Burt. We could seek the assistance of Princeton University. Billions spent as stated by the Commissioner of Examinations could have been utilised for a Test Item Bank. Test Items have not adhered to that principle. This can be combined with other, subjective evidence and used to improve the test by identifying defects in items, some of which can be considered and selecting the best items and rearranging them in an improved order of difficulty.

In the contextive testing, the term objective has a different meaning, it refers to those tests which consist of a number of items, the correct responses to which are all precisely predetermined. The important point is that, to be regarded as objective, an item must have a precisely predetermined correct response, no matter what form it takes or what educational objective it assesses. This means that the marking of the type requires no judgment. In many objective tests, they are scored by machines. In the context of objective tests as indicating complete objectivity is only in the marking or scoring the test. Each item shall have one and only one predetermined correct answer. This fact is grossly overlooked by the test constructor. For example in a number of items a variety of acceptable answers are given, thereby loosing the objectiv in the selective test. This is not a serious disadvantage when this type of item is used in classroom tests. In any public examination it is preferable to have the same number of options.


In objective Test Items, New type items, more possibilities should not be included, since they carry implications. An item should never be made deliberately more difficult by using unnecessarily complex phraseology, since they are likely to be invalidated.



The other quality is that these should discriminate reliably. The discrimination is dependent on the form of the test, the length of the test and the difficulty of the items. The test therefore should start with very easy items and they should by arranged in correct order of difficulty. The proportion of items of different levels of difficulty depends on the purpose of the test and the type of total test score distribution required.

In this article I wish to examine two large and probably insoluble problems in education mainly selection and prediction of educational success-and-suggest that we are at present taking a regressive path which is likely to lead to drastic consequences I have agitated for the last four decades, with no avail one desideration of education policy comes from the needs of society. We cannot exist without a proper supply of teachers, scientists, mathematicians, lawyers, technicians, writers - the list is endless. Given these hard facts, which are not in dispute, there is obvious need for some principle of selection. We are asked to rely on what are known to be unreliable instruments and to reject what are known as reliable instruments.

In Sri Lanka the most notorious aspect of education is the examination system. Examinations at all levels are nearly always found wanting. The tests are not constructed scientifically, as a result they are not valid and reliable. Thus Sri Lankans are asked to rely on what are known to be unreliable instruments of assessment. It denies social justice and deprives society of the talents of deserving students, which are urgently needed in a culture dependent on a large extent on high intelligence. Scientifically, constructed tests serve to spot objectively unused talents and thus draw attention, to a problem, where educationists begin to realize the value of such tests, which alone can improve existing measures and reduce the error inherent in all examinations. Without the help of such tests, advancement to higher development of society, with better educated groups of society will not be possible.

Thus will be bringing to the top a large number of students of mediocre ability, while keeping submerged many people of superior quality. This rise of a new mediocracy in society is socially unjust, nationally disastrous and ethically unacceptable. Immediate steps to resort to scientific test construction and develop a Test Item Bank for varied types of examinations should be the aim. In America, Scholastic Aptitude Test selects students from all over the world to Howard University and other prestigious Universities. It has maintained a high correlation and America has attracted exceptionally intelligent students to those Universities. We should venture forthwith in developing such instruments of measuremental to select persons of high intelligence to avoid the birth of a mediocrity in Sri Lanka. The G.C.E. O/L and A/L exams are conducted all over the world, by constructing the test scientifically. Every question indicates the reliability and validity of the test item.

Unfortunately much of the popular discussions of the problem and indeed of the whole subject of testing arises from a sheer misunderstanding of the terms employed and the objects in view. That is partly due to Sri Lanka’s peculiar history perhaps we should also get the needed impetus for better research on testing on a much larger scale, which alone can improve existing measures and reduce the error inherent in all scientific measurement. Examinations rightly designed and intelligently used can be a useful factor in the educational process. The defects of examinations highlight three factors contributing to this massive mess. The country as a whole is ignorant of the deciding evaluation based on raw marks. No one challenges the validity and reliability based on raw marks. The Examination Department as a matter of urgency should go in for scientific testing with the prime objective of creating a Test Item Bank. The harmful role which examinations play in education can be overcome by this. These should be scientifically constructed. It should be stressed that inventing test items is highly skilled work requiring expert knowledge of the subject matter of the test, technical skill, facility in expression and above all imaginations. It should be appreciated that every question on public examination indicate the reliability and validity of a question. This is indicated in the G.C.E. A/L (London) Examinations.

Lack of knowledge on management is another glaring defect for the enormous malpractice in the process of evaluation that has created myopic reactions among the public. The Examination Department hardly reveals that they practice or malpractice management. As a result they are ill prepared for the tremendous challenges that confront them. It is therefore imperative that action is taken to make people capable of joint performance, to make their strengths effective and their weakness irrelevant. It is the reason that mismanagement is one of the critical determinant factors that is creating a chaotic situation in conducting examinations in Sri Lanka. This kind of mess is being repeated every year and the harmful effects will bring adverse results on our educational culture in the decades to come. The question as to what action to be taken to rescue these children from their likely fate of everlasting educational damnations is still an open question.