EPPP - Psychological Assessment - Introduction, Intelligence Issues

Three-tier system for categorizing test user qualifications:

Level A - may be administered and interpreted by nonpsychologists 

Level B - require some technical knowledge of test construction and use (and completion of) supporting psychological and educational subjects

Level C - should be administered only be individuals with at least a Master's degree in psychology and at least on year of supervised experience under a psychologist

include age, developmental level, gender, reading level, language proficiency, physical abilities, motivation, anxiety, familiarity with testing procedures, and cultural and socioeconomic background

Reliability - the degree to which a test scores are from from the effects of measurement error 

Validity - the degree to which a test measures what was designed to measure

used to describe two characteristics of a test 

1. a test is said to be standardized when the testee's words and acts, the apparatus, and the scoring have been fixed so the scores collected at different times and places are fully comparable. 

2. A test is referred to as standardized when it has been administered under standard conditions to a representative sample for the purpose of establishing norms.

permit comparison between an examinee's test performance and the performance of individuals in the norm group. 

Ex. - percentile ranks and standard scores

AKA domain referenced and content-referenced scores

Permit interpreting an examinee's test performance in terms of what the examinee can do or knows with regard to a clearly defined content domain or in terms of performance or status on an external criterion (e.g., predicted rating on a measure of job performance).

Ex. - an examinee's performance is usually reported as percent of the test items answered correctly, and the percent is compared to a pre-established cutoff (e.g., 80% correct) to determine if the examinee has reached mastery or requires additional instruction

Provided by ipsative scales, which permit intraindividual comparisons - i.e., comparisons of an examinee's score on one scale with his scores on other scales. 

Ex. - an examinee's scores on one the Edwards Personal Preference Schedule provide information on the relative strength of his basic psychological needs.

focuses on overt and covert behaviors that occur in specific circumstances and may utilize behavioral interventions, behavioral observations, cognitive assessment (e.g., thought sampling, protocol analysis) and/or psychophysiological measures.

a type of behavioral assessment that entails determining the function or purpose of a behavior by identifying its antecedents and consequences. 

Goal is to identify strategies for decreasing or eliminating a target behavior by eliminating the antecedents and consequences that are maintaining the behavior and provide support for alternative behaviors

Derived from Vygotsky's method for evaluating a child's mental development and involves an interactive approach and deliberate deviation from standardized testing procedures to obtain additional information about the examinee and/or determine if the examinee is likely to benefit from assistance or intruction.

Involves providing an examinee with additional cues, suggestions, or feedback and is ordinarily done after standard administration of the test to preserve the applicability of the test's norms.

based on empirically validated relationships between test results and specific criteria and make use of a multiple regression equation or similar statistical technique.

Tends to be more accurate

based on the decision-makers intuition, experience, and knowledge. In other words, classifications, diagnoses, and predictions about behavior rely on the clinician's judgment.

Tends to be less accurate

Interviews can be used as young as age 6. Two important goals, establishing rapport and maintaining cooperation. The following techniques may be helpful:

Use descriptive statements - include objective comments about appearance and non-negative comments about behavior or demeanor (you look happy today). Helps define expectations and helps with motivation. 

Use Reflection - mirror what the child says and can be either literal or interpretive. Helps clarify and organize thoughts and feelings 

Provide Labeled Praise - indicates approval and helps guide and encourage the child to behave in a particular way (you're doing a good job of telling me what happened).

Avoid Critical Statements - Criticism elicits negative emotional reactions and defensiveness and disrupts development of rapport.

Must consider acculturation, racial/ethnic identity, and language proficiency; the availability of appropriate norms; the cultural equivalence of the content or construct measured by the test; and the availability of alternatives that are more appropriate for the client's background. The following guidelines should be considered:

Purpose of the assessment - professional should be clear about the purpose and should use only techniques that are likely to accomplish that purpose and benefit the person

Test content - must be sensitive to the cultural loading in current assessment procedures and practices and be aware that, currently, there are no truly culture-fair or culture-free tests

Alternative methods - Whenever possible, use culturally sensitive procedures and assessments either as alternatives to standard tests or in conjunction with them 

Ethnic norms - be aware of normative behavior within different racial and ethnic groups and interpret an examinee's responses within the appropriate cultural context 

Role of the examiner - self-monitor level of assessment expertise with respect to members of racial and ethnic minorities and should be cognizant of any cultural differences that might interfere with the establishment of rapport.

one of the most common ways to collect data from a large group of people. May include personality traits, moods, thoughts, attitudes, preferences, and behaviors

use to make decisions about a client; for example, assigning diagnoses and creating treatment plans. 

Also use this approach to draw conclusions from the empirical research of others to make decisions about mental health treatment. 

• Plus - volume of information 
• negative - tends to be inconsistencies that can arise from various information sources.

Used to observe physiological functions (heart rate, skin perspiration, facial muscles, etc.) which can often describe the emotional state of an individual. 

Can include self-report to discover what the person experienced. 

• Plus - the ability to investigate and analyze momentary experiences without intervening in the interaction while it occurs
• negative - the monetary expense required, and that not every momentary experience can be important to every domain being studied.

aka observational study - a method of collecting evaluative information in which the evaluator watches the participant in their natural environment without altering that environment. Used when other data collection methods are not effective. Can be overt of covert

most appropriate when standardized information needs to be gathered, and result in quantitative data

Looks at natural occurrence and provides qualitative data.

different from other forms of assessment as they involve social interactions. Can ask open or close ended questions, each category can lead to various answers and this add to the quality of the data. 

Structured Interviews - typically easy to replicate as there are a fixed set of closed questions that are used, which means it is easy for test reliability. Not flexible, and answers may lack detail

Unstructured Interviews - more flexible and thus the questions can be adapted as needed, and generate vast amounts of qualitative data, because of the details included in the answers. Time consuming, and interviewers need to be trained which can be expensive

administration and scoring 

norms

Criterion

ipsative

antecedents

testing the limits 

computer adaptive testing (CAT)

empirically validated

the clinician's judgment 

more

descriptive

open-ended

loading (content)

no consistent effect (inconsistent results)

data collection 

technology

Based on observations that various measures of intelligence correlate to some degree with each other. 

Proposed a general intellectual factor (g) and argued that performance on any significant cognitive task depends on g plus one or more specific factors (s) unique to the task

Crystalized - (Gc) refers to acquired knowledge and skills, is affected by educational and cultural experiences, and includes reading a numerical skills and factual knowledge 

Fluid - (Gf) does not depend on specific instruction, is relatively culture free, and enables an individual to solve novel problems and perceive relations and similarities.

distinguishes between three levels or strata of intelligence. 

Stratum III is g (general intelligence); stratum II consists of eight broad abilities including fluid intelligence, crystalized intelligence, and general memory and learning; stratum I consists of specific abilities that are each linked to one of the second stratum abilities.

McGrew combined elements of of the Horn-Cattell and Carroll approaches to derive the CHC model 

developed on the basis of extensive empirical research and serves as the framework for the BABC-II and Woodcock-Johnson IV.

Distinguishes between ten broad-stratum level abilities and over 70 narrow-stratum abilities that are each linked to one of the broad.

Part of Guilford's structure of intellect model 

Convergent - relies on rational, logical reasoning and involves the use of logical judgment and consideration of facts to derive the correct solution to a problem 

Divergent - involves nonlogical processes and requires creativity and flexibility to derive multiple solutions

defines "successful intelligence" as the ability to adapt to, modify, and choose environments that accomplish one's goals and the goals of society and proposes that it is composed of three abilities - analytical, creative, and practical

The traditional view of intelligence is too narrow, and distinguishes between 8 types of cognitive ability - linguistic, musical, logical-mathematical, spatial, bodily-kinesthetic, interpersonal, intrapersonal, and naturalistic 

everyone has all intelligence to some degree and the intelligences are not static but can develop with exposure and learning experiences

The relationships between heredity and intelligence is demonstrated by studies comparing IQ's of people with varying levels of genetic and environmental similarity.  

A frequently cited study is one by Bouchard and McGue (1981), and shows the greater the genetic similarity, the higher the correlation between IQ scores. 

Most reported heritability correlations for people in industrial countries range from .60 to .80 which means the between 32 and 64% of variability in intelligence is due to genetic factors. Must be interpreted with caution

• Correlations 
• Identical twins reared together - .85
• Identical twins reared together - .67
• Fraternal twins reared together - .58
• Biological siblings reared together - .45

The contribution of the environment to intelligence is suggested by several studies 

IQ test scores have been linked to socioeconomic status, educational opportunities, family environment, and nutrition

Studies have found that there is a relationship between family size, birth order, and IQ, with children's IQ scores decreasing from the child that is born first to the child that is born last. 

This model explains the firstborn child's advantage in terms of changing intellectual environment in the family: In contrast to their younger siblings, firstborns do not initially have to share their parents' attention, are exposed to more adult language, and are more likely to act as "tutors" for their brothers and sisters.

Research before year 2000 found that IQ test scores consistently increased over the previous 70 years in the US and other industrialized countries. This increase is called the Flynn Effect 

Cannot be explained by genetics since it involves change over only one or two generations, and is believed to be the result of environmental factors 

After 2000 the Flynn Effect has reversed for people with IQ's over 110 but continued for IQ's 70-109

Crystalized intelligence increases until age 60 but fluid intelligence peaks in late adolescence and declines thereafter. Declines in fluid intelligence have been attributes to declines in processing speed

Used a cross-sequential design that combined cross-sectional and longitudinal methodologies. Findings concluded that a cross-sectional design is more likely to find early age-related declines in IQ because it is more vulnerable to cohort (intergenerational) effects - i.e., to the confounding effects of educational and other differences between different age groups 

study showed that, for most people, only perceptual speed declined substantially prior to age 60.

1. many items and tasks included in cognitive ability tests emphasize information processing and people over 50 experience a decline in the efficiency and speed with which they manipulate information. 

2. physical health is one of the strongest predictors of the maintenance of cognitive functioning in later adulthood. Of particular importance is cardiovascular functioning, which has an impact on information processing speed and other cognitive functions 

3. For some older adults, declines in cognitive ability are due to disuse and are, therefore reversible.

do not differ significantly in terms of average performance on IQ test, but there are some difference in specific abilities.

Females - better on measures of verbal ability, especially during school years, and are less likely to have reading disability 

Males - better on measures of spatial and math skills, with spatial skills showing the largest gender gap 

Some argue environmental factors play a role in this.

Whites tend to outperform Blacks by about 1 standard deviation point 

Some argue IAQ tests are biased against Blacks 

• 2 types of test bias:
• Slope Bias - occurs when there is differential validity - i.e., when the validity coefficient for a predictor differ for different groups - and as a consequence the predictor is more accurate for one group than another 

Intercept Bias - (unfairness) occurs when the validity coefficients and criterion performance for different groups are the same, but their mean scores on the predictor differ. As a result, the predictor consistently over or under predicts performance on the criterion for members of one of the groups.

g (general intelligence)

crystalized 

crystalized

practical

genetic

.85

.45

Flynn

60

cross-sectional 

cohort

cross-sequential 

perceptual speed

verbal

spatial

slop

predictor

D

Testing the limits is one type of dynamic assessment and involves providing an examinee with additional cues, suggestions, or feedback.

Answer A: The term "functional behavioral assessment" refers to a type of behavioral assessment that entails determining the function or purpose of behavior by identifying its antecedents and consequences.

Answer B: Test-teach-retest is a type of dynamic assessment that involves following the initial assessment with an intervention designed to modify the examinee's performance and then reassessing the examinee.

Answer C: Graduated prompting is a type of dynamic assessment that involves giving the examinee a series of verbal prompts that are graduated in terms of difficulty level.

A

In most unstructured interviews, information variance is caused by the wide differences in content and phrasing due to factors such as theoretical orientation and style of the interviewer.

Answer B: Criterion variance is the method for removing extreme values from a set based upon its variance. This type of variance was used when developing the criteria for structured interviews.

Answer C: To establish convergent validity, it is important to show that measures that should be related are related.

Answer D: To establish discriminant validity, it is important to show that measures that should not be related are not related.

B

Examples of norm-referenced scores are percentile ranks and standard scores. These types of scores permit comparisons between an examinee's test performance and the performance of individuals in the norm group.

Answer A: Criterion-referenced scores are also known as domain-referenced and content-referenced scores. They permit interpreting an examinee's test performance in terms of what the examinee can do or knows with regard to a clearly defined content domain or in terms of performance or status on an external criterion.

Answer C: Self-referenced scores are provided by ipsative scales, which permit intraindividual comparisons (i.e., comparisons of an examinee's score on one scale with their scores on other scales.

Answer D: Content-referenced scores are the same as criterion-referenced scores and permit interpreting an examinee's test performance in terms of what the examinee can do or knows with regard to a clearly defined content domain.

C

Reliability refers to the degree to which test scores are free from the effects of measurement error.

Answer A: Validity refers to the degree to which a test measures what it was designed to measure.

Answer B: Content validity is the extent to which a measure covers the construct of interest.

Answer D: Criterion validity is the extent to which an individual's scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with.

A

Multiple regression equations are developed by correlating each test or subtest with a criterion. The higher the correlation, the greater is the weight in the equation. The correlation of the entire battery with the criterion measure indicates the battery's highest predictive validity.

Answer B: Actuarial risk assessments are a statistically calculated prediction of the likelihood that an individual will pose a threat to others or engage in a certain behavior (e.g., violence) within a given period. The data in this type of prediction is from a specific, measurable variable (e.g., gender, age) that has been validated as a predictor.

Answer C: A clinician-based prediction is based on the decision maker's intuition, experience, and knowledge. In other words, classifications, diagnoses, and predictions about behavior rely on the clinician’s judgment.

Answer D: In a multiple hurdle approach, the individual, or applicant, must pass each step of the selection process to advance to the next step. Failure at any step disqualifies the individual from further consideration.

C

Intercept bias, or unfairness, occurs when the validity coefficients and criterion performance for different groups are the same but their mean scores on the predictor differ.

Answer A: Statistical predictions are based on empirically validated relationships between test results and specific criteria, and they make use of a multiple regression equation or similar statistical technique.

Answer B: Slope bias occurs when there is differential validity and, consequently, the predictor is more accurate for one group than another.

Answer D: Clinical predictions are based on the decision maker's intuition, experience, and knowledge. In other words, classifications, diagnoses, and predictions about behavior rely on the clinician's judgment.

B

Crystallized intelligence is reflected in an individual's general knowledge, vocabulary, and reasoning based on acquired information.

Answer A: Fluid intelligence does not depend on specific instruction, is relatively culture-free, and enables an individual to solve novel problems and perceive relations and similarities.

Answer C: Convergent thinking relies on rational, logical reasoning and involves the use of logical judgment and consideration of facts to derive the correct solution to a problem.

Answer D: Divergent thinking involves nonlogical processes and requires creativity and flexibility to derive multiple solutions.

B



Self-report is one of the most common ways social scientists collect data from a large group of people. This information may include personality traits, moods, thoughts, attitudes, preferences, and behaviors. 

Answer A: Multi-informant reports are a form of data collection that compile information from several sources (e.g., family, friends, clinicians, etc.). 

Answer C: Direct observation is also known as “observational study.” It is a method of collecting evaluative information in which the evaluator watches participants in their usual environment without altering that environment. Direct observations may be structured or unstructured. Answer D: Psychophysiological measures are data collection methods that involve the monitoring of a physiological function, such as heart rate or perspiration, as it correlates to a client’s psychological or emotional state.

A

Descriptive statements give attention to the child, encourage the child the continue doing what they are doing, and point out to the child what behavior is expected of them. An example of a descriptive statement would be "You look happy today."

Answer B: Reflective statements mirror what the child says and can be either literal or interpretive. Reflection helps clarify and organize the child's thoughts and feelings.

Answer C: Labeled praise indicates approval and helps guide and encourage the child to behave in a particular way.

Answer D: Criticism elicits negative emotional reactions and defensiveness from a child and disrupts the development of rapport. Better tactics for altering undesirable behavior are to make rule-based corrections, invitational statements, and to ignore the behavior.

C

Structured interviews are typically easy to replicate as there is a fixed set of closed questions that are used, which means it is easy to test for reliability. They have some limitations, including that they are not flexible and the answers may lack detail.

Answer A: Structured direct observations are most appropriate when standardized information needs to be gathered and result in quantitative data.

Answer B: Unstructured interviews are more flexible, and thus the questions can be adapted as needed and generate vast amounts of qualitative data because of the detail included in the answers.

Answer D: Unstructured direct observations look at the natural occurrence and provides qualitative data. Weaknesses of this approach include participant and observer bias.