The potential targets of school reform include the curriculum (what should be taught), instruction (how teaching occurs and who does it), and testing (the method by which learning is measured). Since I've already discussed instruction, I'd like to use this installment to discuss what needs to be changed about testing, a process currently as obsolete as classroom instruction itself.
Much of this obsolescence results from the educational testing industry's continued reliance upon the assumptions underlying the classic psychometric model which nurtured, and was advanced by, early attempts at developing tests of intelligence and aptitude. Assumptions that may have been tenable for constructs that (a) were of unknown etiology, (b) possessed item domains that could not be exhaustively specified, and (c) were believed (incorrectly) to be both stable and immutable to instruction. Of equal importance, a perfect score (or a score of zero) on the resulting tests could not be interpreted quantitatively because it was impossible for any human to have attained perfection on (or be completely devoid of) the attributes in question.
School learning, on the other hand, possesses none of these constraints, although commercial "achievement" tests are constructed as though it does. This has proved most unfortunate since the real purposes of educational testing are to identify how much of the curriculum each student has mastered and what else needs to be taught (or re-taught).
How Tests of School Learning Should be Constructed
Since we know what causes learning (instruction) and what should be taught in our schools (the curriculum), the assessment of school learning is hardly rocket science. Assuming that our testing time frame continues to be a school year, "all" we need do is:
1. Specify the entire elementary school curriculum via the use of instructional objectives in enough detail to permit multiple test items to be written for each objective.
2. Construct school learning tests by some defensible, transparent procedure from this list of instructional objectives and test items. These tests can be based upon single academic subjects (e.g. mathematics, reading comprehension) or more specialized instructional topics within a subject (e.g., sight reading vocabulary).
3. Administer each test twice (preferably via computers so that results can be immediately available, securely stored, and appropriately distributed), once at the beginning of the year and once at the end. (There is no need to assess identical objectives or items twice since it is not difficult to ensure equivalence if the same sampling procedures are employed. It is necessary to administer the tests twice during the same year, however, since the school currently has little control over what occurs during the summer months.)
The difference between the beginning of the year and the end of the year test scores will then represent the absolute amount of the curriculum any given student learned during that year. Most of this will be a function of school instruction, although there is (and always will be) some error associated with inferences based upon test scores due to sampling, how a student may have felt during one of the testing sessions, exposure to extra school instruction, and so forth. No measurement ever developed (physical, psychological, or educational) is conducted without some degree of error.)
Tests scores constructed in this manner have a number of interesting characteristics, some of which are:
1. There is no need to employ standard scores (such as grade equivalences, percentile ranks, and so forth). All of these algebraic manipulations, including the normal curve itself, are used solely to rank order scores for the purpose of comparing them to one another, which is necessary only when a test is incapable of measuring the absolute amount of something. Tests assessing school learning should be capable of producing meaningful zeroes (indicating none of the curriculum was learned) and perfect scores (i.e. 100% mastery of the curriculum). In other words, scores produced by such tests would represent the amount of learning that takes place. To reduce them to a simple rank ordered list would be idiocy.
2. Tests such as these have direct instructional implications in their own right since each item represents one instructional objective and a missed item therefore represents a specific unlearned concept that needs to be taught (or re-taught). The most meaningful unit of analysis therefore often becomes an individual student's response to each individual item.
3. Because all interested parties (teachers, parents, and students) could have access to both the instructional objectives that should be taught and the types of items which will be used to assess learning: (a) teachers should teach the test and should employ mock tests based upon similar (but not identical) items to ascertain which students have mastered which objectives immediately following instruction; (b) parents would also have the capability of assessing their children's progress during the school year; and (c) students would become familiar with both the testing process and the types of items they will face at the end of the year.
4. Resources such as these (i.e., the existence of a comprehensive translation of the curriculum via instructional objectives, each of which would be accompanied by sample test items) might also speed the develop of supplementary digital tutoring software, make supplementary human tutoring more effective, and reduce the effects of differential test taking skills contributing to the socio-economic testing gap.
From my perspective the only unanswered question is why there should be any resistance, outside the testing industry, to substitute such a process for our currently obsolete testing model.
For a more comprehensive discussion of school testing, see Chapter Eight of TOO SIMPLE TO FAIL: a Case for Education Change (Oxford University Press) [bausell@gmail.com]
