Scoring Validity in Action

Abstract

In Chapter 6 we looked at elements of context validity that need to be considered at the test design stage and made a number of points in relation to test development which could potentially impact on the reliability of our tests. Hughes (2003: Chapter 5) examines a number of these specifically in relation to reliability. He provides a set of guidelines for making the test task itself more likely to produce reliable scores:

• take enough samples of behaviour;

• do not allow candidates too much freedom of choice;

• write unambiguous items;

• provide clear and explicit instructions;

• ensure that tests are well laid out and perfectly legible;

• make candidates familiar with format and testing techniques;

• provide uniform and non-distracting conditions of administration;

• use items that permit scoring which is as objective as possible;

• make comparisons between candidates as direct as possible.

And in relation to the scoring of test performance itself:

• provide a detailed scoring key;

• train scorers;

• agree acceptable responses and appropriate scores at the outset of scoring;

• exclude items which do not discriminate well between weaker and stronger student;