A test cannot be valid if it is unreliableAssessment methods and tests should have validity and reliability data and research to back up their claims that the test is a sound measure. 

Reliability is a very important concept and works in tandem with Validity.  A guiding principle for psychology is that a test can be reliable but not valid for a particular purpose,  however, a test cannot be valid if it is unreliable. Read more about Reliability...read more about reliability

Assessment Validity

Assessment methods including personality questionnaires, ability assessments, interviews, or any other assessment method are valid to the extent that the assessment method measures what it was designed to measure. 

There are different aspects of validity and they differ in their focus. 

The aspects of validity that have an impact on the actual scientific application of the assessment are:

The two less relevant aspects of validity:

The most important validity to those interested in the usefulness of tests for predicting work-related outcomes is Predictive Validity.

Predictive validity is the extent to which a test or questionnaire predicts some future or desired outcome, for example work behaviour or on-the-job performance.  This validity has obvious importance in personnel selection, recruitment and development. 
  • Predictive validity is of particular interest to psychologists and HR professionals as it allows us to extrapolate the results of the test taken today to a meaningful outcome of what we want to know about the future behaviour of an employee. 
  • Predictive validity is about whether a test can predict some meaningful future outcome accuratelyPredictive validity is usually measured by the correlation between the test score and some appropriate criterion.  The criterion could be performance on the job, training performance, counter-productive behaviours, manager ratings on competencies or any other outcome that can be measured.
  • A validity coefficient (denoted by r) is a correlation between a test score and some criterion measure (such as performance).
  • Read more about the predictive validity of different assessment methods...read more about the different predictive validity coefficients of different assessment methods
  • A test may appreciably improve predictive efficiency if it shows any significant correlation with the criterion, however low. 
  • The significance of the correlation is a measurement of the probability that the relationship between the two sets of data is due to chance. 
  • A larger pool of related data decreases the probability that a correlation is due to chance and therefore the cut-off for the level of a significant r is usually less as the sample size increases. 

Correlations between teo variables

Predictive Validity Findings of Different Assessment Methods

The following table summarises some of the general research findings around the predictive validity of the different selection methods available:

Assessment Method

Predictive Validity for
Job Performance

GMA (IQ) Tests

.65

Integrity Tests

.46

Employment Interviews (structured)

.58

Employment Interviews (unstructured)

.60

Conscientiousness

.22

Biographical data

.35

References

.26

Job experience

.13

Assessment centres

.37

Peer ratings

.49

Years of Education

.10

Emotional Intelligence

.24

Work sample tests

.33

Emotional Stability

.12

Source: Schmit, Oh & Shaffer (2016) - Working paper

Assessment method

Predictive validity

Criterion measure

Integrity Tests

.58

Counter-productive work behaviour

Integrity Tests

.51

Overall job performance

Source – Comprehensive meta-analysis of integrity test validities by Ones, Viswesvaran & Schmit (1993).

Multiple Assessments and the Impact on Predictive Validity

When we combine assessments in a battery we can increase the validity of the testing if the tests are of approximately the same validity and have low inter-correlations.  the Validity of a method tell us how well it measures what it is designed to measure

Guilford & Fruchter (1978) summed up the different effects of lengthening tests and including more tests in a battery as follows:

  • “In general, if there is a choice between lengthening of tests in a battery to make them more reliable and adding more tests of different kinds that contribute unique valid variances, the decision should certainty go to the second alternative.” 
  • We can therefore increase the validity of testing by using more than one assessment method. 

    • Schmidt, Oh & Shaffer (2016) found the two combinations with the highest multivariate validity and utility to predict on the job performance were GMA (IQ test) and an integrity test (mean validity of .78) and GMA (IQ test) and a structured interview (mean validity of .76).

Test Length and Validity

The following table illustrates how validities increase as test length increases.  The calculations are based upon typical reliability and validity figures of .70 and .40 respectively for a 5 minute test.  The difference in validity between a 5 minute test and a test of infinite length is only a .078 difference (.478-.400).

Test Time (Minutes)

Validity (r)

1

.270

2

.332

3

.365

4

.386

5

.400

6

.410

7

.410

8

.418

9

.424

10

.430

11

.434

12

.437

13

.440

14

.443

15

.445

Test of infinite length

.478

Read about the Reliability of Assessments - click here...read more about reliability of psychological assessments

 

WE ARE MEMBERS OF:
NZ Psychological Society
SIOP - Society for Industrial & Organizational Psychology
ITC - International
Testing Commision
NZ Association of Psychological Type
APA - American Psychological Association

Candidate Information
Click here

Quotes

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go to Top

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Go to Top