-
Research design
- the outline, plan or strategy used to investigate the research problem
- - specifies such things as how to collect and analyze the data
-
The goal in research is to
use the strongest design that is possible, ethical, and feasible for your research question.
-
Strong research designs
include pretests (measure the DV), control groups, and random assignment
-
Weak experimental designs
- designs that do not control for many extraneous variables and provide weak evidence of cause and effect
- - One group posttest only
- - One Group Pretest- Posttest Design
- - Posttest-Only Design with Non-eqiv Groups
-
One-group posttest-only design
- administration of a posttest to a single group of participants after they have been given an experimental treatment condition
- - X= experimental manipulation
- - O = measurement of the DV
-
One-group posttest-only design is rarely useful because
- - the design allows no evidence of what the participants would have
- scored on the dependent variables had they not received the treatment
- - Design does not have a no-treatment control group
- - Should be viewed as a faulty design
-
One-Group Pretest-Posttest Design
- design in which a treatment condition is interjected between a pretest and posttest of the dependent variable
- - provides a small improvement over the one-group posttest-only study
- because of the many uncontrolled rival hypotheses that could also
- explain the obtained results.
-
One-Group Pretest-Posttest Design is weak because
- - not so much because the sources of rival hypotheses can affect the
- results, but because in most cases we do not know if they did.
- Does not allow us to control or to test for the potential influence of these effects.
-
One Group Posttest only and One Group Pretest-Posttest disadvantage
is that we cannot know if the independent variable influenced the dependent variable.
-
Posttest-Only Design with Nonequivalent Groups
- -
- Design in which the performance of an experimental group is compared
- with that of a nonequivalent control group at the posttest
- - by
- adding a control group, this design addresses some of the threats to
- internal validity, history, maturation, regression artifact, and
- attrition
-
Problem with Posttest Only Design with Non-eqiv Groups is
- -
- that the comparison group might differ in important ways from the
- participants in the experimental group. This threat is called SELECTION
- - you want your groups to differ only on the independent variable
- - matching is not as sufficient with controlling for many variables like random assignment does
- - a weak design
-
The Weak Experimental Research Designs are WEAK because
they do not provide a way of isolating the effect of the treatment condition; rival hypotheses are not eliminated
-
Strong Experimental Research Designs
- - have greater internal validity.
- -
- That is, they provide more assurance that the effect of the independent
- variable on the dependent variable has been isolated and tested.
- - In order to achieve internal validity, we must eliminate potential rival hypotheses.
- - This can happen by two primary means: control techniques and a control group
-
RCT (Randomized controlled trial)
Experimental design with random assignment to experimental and control groups
-
Counterfactual
what the experimental group participants' responses would have been if they had not received the treatment
-
Control group serves as a source of
comparison and as a control for rival hypotheses.
-
Strong Research Designs can take the form of
- between-participants
- research design in which participants are randomly assigned to
- different groups and participate in a single condition
-
Between - Participants Design
- - groups produced by random assignment, and the different groups are exposed to the different levels of the IV
- - Groups are composed of different people
- - Random assignment eliminates most of the internal validity threats.
-
Posttest- Only Control-Group Design
- -
- Administration of a posttest to two or more randomly assigned groups of
- participants that receive the different levels of the independent
- variable
- - The research participants are randomly assigned to as many groups as there are experimental conditions
- - The nonequivalent posttest-only design lacks random assignment this design does not.
- -Including
- a randomized control group causes the threats to internal validity such
- as history, maturation, instrumentation, testing, regression artifact,
- attrition, selection and additive effects to be controlled
-
Strengths and Weaknesses of the Posttest- Only Control- Group Design
- -
- randomization is the best control technique for achieving equivalence
- but does not give complete assurance that the necessary equivalence has
- been attained.
- - lacks a pretest which means you dont get a way to check on the success of the randomization process.
- - Lacks increased statistical power associated with pretest.
-
Within- Participants Design
- - all participants receive all conditions
- - use counterbalancing
-
Strengths of Within-Participants Designs
- -
- Participants serve as their own control and variables such as age,
- gender, and prior experience remain constant over the entire experiment
- - Conditions can't differ because some kinds of people are in one condition but not in another
- - If counterbalancing is used, this within design controls for all of the common threats to internal validity.
- - Therefore, maximally sensitive to the effects of the independent variable
- - Within does not require as many participants as between
- -
- powerful technique of control in within because participants serve as
- their own control, they are perfectly matched in the various treatment
- conditions, increases the sensitivity of the experiment
-
Weaknesses of within-participants designs
- - taxing on participants because they have to be present for all conditions
- - the confounding influence of a sequencing effect, a sequencing rival hypothesis is a real possibility
- -
- counterbalancing controls only linear sequencing effects; if the
- sequencing effects are nonlinear (differential carryover effects), then a
- confounding carryover sequencing effect will remain
- - generally harder to deal with than between-participants design
-
Mixed Designs (Between and Within)
- - In between, random assignment used, each group receives only one level of the IV
- - In within, all levels of IV are administered to all participants
- - Pretest-posttest control-group design is an example of a mixed design
- - Mixed design must have at least one between-subjects IV and at least one within-subjects IV
-
Pretest-Posttest Control-Group Design
- -
- Randomly assigned to two or more treatment conditions and pretest is
- administered, then the treatment conditions are administered and last,
- the posttest is administered.
- - Strong on internal validity
-
Advantages of including a pretest
- o
- Allows researcher to check to see how well the randomization process
- worked (random assignment. When pretest is included, researcher does not
- need to assume that it worked properly
- o Researcher can determine if a ceiling effect or floor effect is likely to occur
- o
- If experimental and control groups are slightly different on the DV
- measure at the pretest, the researcher can use a statistical technique
- called analysis of covariance to statistically control for these pretest
- differences.
- o This statistical technique adjusts for pretest
- differences, but it also provides a more accurate and powerful test of
- the differences between the experimental and control group posttest
- scores
- o To gain an empirical demonstration of whether an overall
- change in response occurred from pretesting to posttesting. Most direct
- way of gaining such evidence of change is to see if there is a
- statistically significant difference between pretest and posttest change
- scores of the experimental and control groups
-
Ceiling effect
situation where participants' pretest scores on the dependent variable are too high to allow for additional increases
-
Floor effect
situation where participants' pretest scores on the dependent variable are too low to allow for additional decreases
-
Analysis of covariance
a statistical procedure in which group means are compared after adjusting for pretest differences
-
External validity
the degree to which the results of a study generalize
-
disadvantage of pretest
- - participants might change in some way because they were given a pretest.
- - external validity can sometimes be weakened when a pretest is included in the design
- -
- the results might generalize best to people who have taken a pretest,
- and might not generalize as well to those who have not taken a pretest
-
Factorial Designs
- - When there is more than one independent variable of interest, a factorial design is the experimental design of choice.
- - In posttest-only control group, within- participants, and pretest-posttest control group designs have only one IV
-
Factorial design
- - two or more IVs are studied to determine their separate and joint effects on the dependent variable
- - The IVs in factorial designs can be between-subjects, within-subjects variables, or a combo of both (mixed design)
-
Cell
combination of levels of two or more independent variables
-
Cell mean
the average score of the participants in a single cell
-
Marginal mean
the average score of all participants receiving one level of an independent variable
-
Main effect
the influence of one independent variable on the dependent variable
-
Interaction effect
when the effect of two or more IVs on the DV is more complex than indicated by the main effects
-
Two-way interaction
the effect of one IV on the DV varies with the different levels of the other independent variable
-
No interaction rule
if the lines are parallel, there is no interaction
-
Factorial Designs Based on within-subjects independent variables
- -
- The unique feature of a factorial design composed of two
- within-subjects IVs is that all participants experience (at different
- times) all combinations of the IVs.
- - Use counterbalance when you have within-subjects IVs, to eliminate order and carryover effects
-
Factorial designs Based on a Mixed Model
- Advantage: need fewer participants
-
Strengths of Factorial Designs
- - enables us to include as many independent variables, no limit on IVs
- - Advantages:
- o Allows experimenter to manipulate more than one IV simultaneously, therefore more precise hypotheses can be tested.
- o Researcher can control a potentially confounding variable by building it into the design as an independent variable
- o Enables researcher to study the interactive effects of the IVs on the DV
- o This last one is the most important because it enables us to hypothesize and test interactive effects.
- o Testing main effects does not require a factorial design, but testing interactions does.
-
Difficulties associated with increasing number of IVs in Factorial Designs
- o Increase in the number of participants required
- o Increased difficulty of simultaneously manipulating the combinations of independent variables.
- o Higher-order interaction effects are significant. Interactions of an even higher order tend to become unwieldy
-
There are several factors to consider in making the design decision:
- o
- The nature of the research problem, the specific research question, the
- extraneous variables that must be controlled, and the relative
- advantages and disadvantages inherent in alternative designs
-
Considerations that are under your control when constructing an experimental research design:
- o Should I use a control group
- o Should I use multiple treatment comparison groups
- o Should I use a pretest
- o Should I use just one or multiple pretests
|
|