must show relationship between IV and DV.
Threat to Internal Validity: History
The specific events that occur between first and second measurements...longer time in between= greater threat
Threat to Internal Validity: Maturation
Changes occuring within the experimental subjects that are due to the passage of time.
Threat to Internal Validity: Testing
effects of taking a test upon the scores of a second testing. people usually do better the second time they take a test.
Threat to Internal Validity: Instrumentation
changes in the calibration of a measuring instrument or changes in the observers/scorers may produce changes in the obtained measurements.
Threat to Internal Validity: statistical regression
when subjects are assigned to a group on the basis of their extreme scores; move toward the group mean during subsequent measurements.
extent to which causal inferences made in an experiment can be generalized to other times, settings, or groups of people
Pre-Experimental (Exploratory) Designs
lack random assignment and control group
One group post-test only
single group is studed only once, subsequent to some agent or treatment presumed to cause change
-provides only a single measurement of what happens when one group of people is subjected to one treatment experience
One-Group Pre-Test - Post-Test Design
single group is studied twice (once before intervention, once after)
O X O
Cannot establish causation
Post-Test Only with Non-Equivalent Groups (Static Group Comparison)
Can't establish causality, doesn't control for threats to internal validity. No control group.
- group that has been given intervention is compared with another group that has not.
- X O
True Experimental (Explanatory) Design
uses randomization and other techniques to control threats to internal validity
Pre-Test - Post-Test Control Group Design (with Randomization)
- Exp. Group: R O1 X O2
- Control Group: R O1 O2
Random assignment equalizes the comparison groups, which eliminates the threats to internal validity, except certain patterns of attrition.
Post- Test Only Control Group Design
- Exp Group: R X O
- Control Group: R O
No Pretest....assume randomization will give us equivalent comparison groups. Cannot estimate exact amount of change due to intervention
Simple Interrupted Time-Series Experiment
(Simple Time Series Design)
O1 O2 O3 O4 X O5 O6 O7 O8
Researcher observes subject several times before/after intervention. Determines how long effects of intervention last after receiving services. Problem with causation.
Approximations of experimental control in nonexperimental settings. Used when not possible to randomize subjects into experimental/control groups. Not randomized. Groups call treatment/comparison groups.
Solomon Four-Group Design
Can tell whether changes in DV are due to some interaction effect between the pretest and exposure to experimental stimulus (X)
- Experimental Group: RO1 X O2
- Control Group: RO1 O
- Experimental Group: R X O2
- Control Group: R O2
Controls for testing effects...all threats to internal validity are addressed. Must use a large number of study participants.
Nonequivalent Control (comparison) Groups Design
used in program evaluation (no possible randomization)
- Treatment Groups: O1 X O2
- Comparison Group: O1 O2
Interrupted Time-Series Experiment with a Non-Equivalent Control Group (Multiple Time Series Design)
similar to time series except researcher has comparison group.
- Treatment Group:
- O1 O2 O3 O4 X O5 O6 O7 O8
- Comparison Group:
- O1 O2 O3 O4 O5 O6 O7 O8
What is the most valid source of data?
A series of measurements of the client's condition prior to treatment.
Internal validity is enhanced when baseline has enough measurement points to show a stable trent in problem. (5 - 10 points recommended)
data about target behaviors is collected during assessment phase of therapy (before intervention)
- Strength: data collectors aware they need to measure behavior so they can take the time to do it accurately
- Weakenss: treatment gets delayed, clients might start working on target behavior before intervention phase
Data about target behavior is reconstructed from memory or records
- Strength: can start intervention immediatly (good for crisis situations)
- Weakness: have to rely on memories to reconstruct occurance. Not always reliable.
Basic AB Design
basic single subject design
- Strength: clearly reveals if any change between baseline/intervention phases
- Weakenss: doesn't distinguish if intervention caused the change.
- Strength: More support to rule out spurious factors if target behavior returns to baseline level when intervention is removed. Strong internal validity.
- Weakness: ethical concerns (might be bad to remove a successful intervention and return clients to pretreatment condition), can't be done with
Multiple Baseline Designs
- A1 B B B
- A1 A2 B B
- A1 A2 A3 B
Done with 3 different people and the intervention is applient at 3 different times.
- Strength: No ethical concern + Strong validity; can address multiple problems of clients
- Weakness: may have seepage of effects from one setting to another unless behaviors, client, etc are independent; order of intervention may have impact on another client behavior; takes a lot of planning and mayb be hard to implement
Intensity of intervention is changed as you move through the process. Once a client reaches an intermediate goal, he/she can move onto next level on intensity, which expects more from the client. used in physical therapy
: does not remove intervention (more ethical); client can set intermediate objectives during assessment phase
- Weakness: limited to behaviors that take long time to change; how much change can actually be shown in each phase?
Baseline - first intervention - second intervention
- Strength: Flexible design; allows one to make changes if intervention is not working
- Weakness: potential carryover effect from B to C.
A B A C A B C
Combines the effects of two successful interventions into one powerful intervention.
Simple Random Sampling (SRS)
each element has equal probability of being chosen for the sample
hard to use if you have a complex sampling frame; may not be the most accurate method
Taking every Kth element in a sampling frame. Sampling interval determined by dividing population size by desired sample size
Divide population into smaller strata, prior to drawing the sample and then separate random samples are drawn from each strata
Cluster or Multistage
Final units to be included in the sampling are obtained by first sampling larger unitis (clusters) in which smaller units are contained.
used when there are too many elements that compose a population
efficient but more prone to sampling error than other designs because sampling error is introduced in each step of the design.
Nonprobability based sampling designs
investigator does not know the probability of each population element being included in the sample.
less accurate but if you don't have the resources for a probability sample its the next best thing.
taking whichever elements that are readily available to the researcher
start with a few cases of the specific type one wants to study and have them lead to more cases, who lead to more...
dividing a population into various categories and setting quotas on the number of elements to be selected from each category. Once a quota is reached, no more elements from that category are put in sample. Similar to stratified. Used when you can't establish sampling frame.
Types of Pre-Experimental Designs
- One Group Post-Test
- One Group Pre-Test - Post Test
- Post Test only with Non-Equivalent Groups
Types of True Experimental (Explanatory) Design
- Pre-Test --Post Test control Group
- Post Test only control group
- Solomon Four Group
Types of Quasi Experimental Designs
- Non Equivalent Control Groups
- Simple Interrupted Time series experiment
- Interrupted time series experiment with a non equivalent control group
allows researchers to use their prior knowledge about the topic