A definition of reliability
Reliability of scores from selection measures is a characteristic necessary for effective HR selection. In our earlier example, the computer programming aptitude test was apparently a unreliable measure; it is this unreliability in test scores that makes it difficult for Robert to know each applicant's actual level of programming aptitude. A host of definitions have beengiven for the term relicability. In our discussion, we will touch on several of these. But for now we want to consider a fundamental definition of the concept. In the context of HR selection, RELIABILITY simply means the degree of dependability, consistency or stability of SCORES on a MEASURE used in selection researchg (either predictors or criteria). In general, reliability of a measure is determined by the degree of consistency between two sets of scores on the measure. In our earlier selection example, we would have expected the programming aptitude test to have yielded similar results from one testing period to the next if the test produced reliable data. Because the results were not similar, we would probably conclude that the test contained errors of measurement. Thus a careful study of the reliability of the test should be made.
Errors of Measurement
- Reliability deals with errors of measurement. A measure that is perfectly reliable is free of errors. None of our selection measures whether predictor or criterion, will be free of measurement errors (although such errors tend to be less frequent with measures of physical attributes relative to measures of unobservable psychological characteristics).
- Selection measures designed to assess important KSAs may be prone to error due to test taker, examiner and the situation in which testing takes place.
- Greater amount of measurement error, the lower reliability of a selection mesure.
- Less errors= higher reliability.
- If errors of measurements can be assessed a measurereliability can be determined.
What are errors of measurements
When we use selection devices (e.g. tests) we obtain numerical scores on the measures. These scores serve as a basis for selection decision making. Because we are using scores as a bsis for our decision, we want to know the 'true' scores of applicants for each characteristic being measures. E.g. Administer a mathematics ability test we want to know the true match ability of the test taker.Unless our measure is perfectly reliable, we will encounter difficulties in knowing these true scores. In fact, we may get mathematics ability scores that are quite different from the individuals' true abilities.
Why do we get errors
- The score obtained on a measure (OBTAINED SCORE) consists of two parts.
- 1. True component
- 2. Error component.
- the componenets of any obtained score (X) can be summarized by the following Xobtained=Xtrue=Xerror
- xobtained=obtained score for a person or measure
- Xtrue= true score for a personon the measure; that is, the actual amount of the attribute measured that a person really possesses.
- Xerror= error score for a person on the measure; that is, the amount that a person's score was influenced byfactors present at the time of measurement that are unrelated to the attribute measure. These errors are assumed to represent random fluctuations or chance factors.
- The notion of a score being compsed of true and error parts is basic axiom of measurement theory.
- based on an ideal conception. It is the score individuals would obtain if external and internal conditions to a measure were perfect.
- Imagine that an individual takes a test measuring a specific ability many different times. With each testing,his or her scores will differ somewhat; after a large number of testings. the scores will take the form of a normal distribution. The differences in scores are treated as if they are due to errors of measurement.The average of all test scores best approximates the test taker's true ability. We might think of a true score as the mean or average score made by an individual on many different administrations of a measure.
- The idealized situation does not exist. The notion of a true score, however, helps to define the idea that there is a specific score that would be obtained if measurement conditions were perfect. Because a true score can nevr be measured exactly, the obtained score is used to estimate the true score. Reliability answers this question: How confident can we be that an indiviual's obtained score represents his or her true score?
- A second part of the obtained score is the error score. This score represents errors of measurement - factorsthat affect obtained scores but are not related to the characteristic, trait or attribute being measured. These factors, present at the time of measurement distort respondents' scores either over or under what they would have been on another measurement occasion to the next. Fatigue, anxiety, or noise during testing that distracts some test takers but not others are only a few of the factors that explain difference in individuals' scores over different measurement occasions.
- How much error exists in a score is an important attribute of the selection measure. If scores on a selection measure are due to chance or reflect an inconsistant or unpredictable error, we cannot have much confidence in the selection device. If we use a test that claims to measure general mental ability, but the reported reliability is so low that the test scores consist mostly of error then our so called 'intelligence test' cannot possibly measure mental ability. With excessive error, a selection measure will not be useful.If there is little error present, the measure may be applicable in selection.
- errors can affect predictors (eg tests) as well as criteria (eg. performance evaluations)
- Important to reduce errors in measurement as much as possible to show true differences rather than chance (or error) differences.
- Selections based on reliable measures are more accurate to suitability and job future performance as well as being fairer to the applicants.
examples of sources of errors of measurement contributing to unreliability in HR selection measures
- Source of error - individual responding to a selection measure.
- Condidion which error may occur - individual's physical and mental health, motivation, mood, level of stress, understanding of instructions, content of the items etc affect how the individual responds to a selection measure.
- Example of error - An applicant has an accident on theway to take a test. During the test he is distracted by worries that his car insurance will not cover all of the expenses.
Error example 2
- Source of Error - Individual administering a selection measure
- Condidtions error may occur - Where the administrator of a selection measure affects the responses of the individual completing the measure.
- Example of error- Two employment interviewers interview the same job applicants using the same interview questions. One interviewer frequently smiles and nods approvingly during the interview; the other does not.
Error example 3
- Source of error: Individual scoring a selection measure
- Conditions error may occur: where judgment and subjectivity on the part of the scorer of a selection measure play a role in scoring.
- Example of error: A supervisor does not attend a training session prior to using a new performance appraisal instrument. Thus when giving ratings, whe completes the performance evaluation forms as soon as possible with no consistent or precise rules for evaluating each employee's actual job performance.
Error example 4
- Source of error: physical conditions under which a selection measure is administered.
- Conditions error may occur: where heating, cooling, lighting, nois etc. affect how an individual responds to a selection measure
- Example of error: While taking a test, respondents are interrupted by a power failure that affects both the cooling and lighting in the room.