The confidence that an experimental treatment or condition made a differnce and that rival explanations were systematically ruled out through study design and control.
The ability to generalize the findings from a research study to other populations, places, and situations.
A way to rule out threats to internal validity through statements that the threat is not a reasonable one.
Often called alpha and referred to as the level of significance, this error is the rejection of a true null hypothesis; the researcher erroneously draws a conclusion that the intervention had an effect.
Type I error
Often called beta and related to the power of a statistical test, this error is the acceptance of a false hypothesis; the researcher erroneously draws a conclusion that the interention had no effect.
Type II error
The magnitude of the impact that the intervention or variable is expected to have on the outcome.
Threaten validity because of events or circumstances that occur during data collection.
Threatens validity because the changes that occur in subjects do not occur as a result of the intervention, but because time passed.
Threatens validity due to the familiarity of the subjects with the testing, particularly when retesting is used in a study.
Threatens validity becaus the instrument or data collection procedure has changed in some way.
Threatens validity because subjects may perform differently because they are aware they are in a study or as a reaction to being treated.
Threatens validity due to the introduction of bias through selection or composition of comparison groups.
A threat to internal validity resulting from loss of subjects during a study
The capacity to confidently generalize the results of a study form one group of subjects to another population group.
Threatens validity because subjects react to something because it is novel or new and not to the actual treatment or intervention itself.
Threatens validity due to the interaction with the researcher conducting the study or applying the intervention
The ability of qualitative research findings to be applied to other samples and other settings.
Applicability and transferability
The likelihood that qualitative research outcomes or events will happen again given the same circumstances
Cross checking conclusions using muliple data sources, methods, or researchers to study the phenomenon.
Checking the accuracy of the observations and conclusions directly with subjects.
The process of explicitly reflecting on and documenting the researchers biases.
Detailed documentation of sources of information, data, and design decisions related to the qualitative research study.
Before determining if there is a casual relationship between the outcome and the intervention, three conditions must be met:
- Changes in the presumed casue must be related to changes in the presumed effect
- The presumed cause must occur before the presumed effect
- There are no plausible alternative explanations
Ways to rule out alternative explanations for outcomes by using statistical analyses:
- Testing null hypotheses
- Determining the probability of Type I and Type II error
- Calculating and reporting tests of effect size
- Ensuring data meet the fundamental assumptions of the statistical tests
Factors that jeopardize internal validity:
- Historical effects
- Maturation effects
- Testing effects
- Instrumentation effects
- Placebo effect
- Multiple treatment effects
- Subject selection
- Experimental mortality
Factors that jeopardize external validity:
- Population validity
- Selection effects-most likely to effect.
- Time and historical effects
- Novelty effect-because it is new
- Experimenter effect-interaction with researcher
The intent of qualitatie research is to interpret rather than test interventions
Strategies to promote the validity of qualitative research:
- Prolonged or varied field experiences
- Verbatim accounts
- Triangulation-multiple sources
- Participant feedback or member checking-
- Bracketing the researcher's bias-documenting the bias-being aware of it
- Documentation of an audit trail-a record of the rationale for each choice
Three ways to deal with threats to validity:
- Eliminate the threat
- Control the threat
- Account for the threat
An internally valid research study is one in which there is sufficient evidence to support the claim that the intervention had an effect and that nothing else was responsible for the outcome.