What are the necessary characteristics of a causal relationship?
covariance
temporal precedence
internal validity
random assignment of subjects
manipulation of an IV
Pseudo variables supports claims of causality
True or False?
False: age and gender do cannot support claims of causality
How can regression beta values be used to investigate causality?
whichever beta value is largest compared to the rest of the beta values is most likely to have a casual relationship
beta values can have both positive and negative values (pos/neg relationships)
Types of Longitudinal Studies:
cohort studies
panel studies
record linkage
prospective studies
retrospective studies
Cohort Study
chart the lives of people with shared characteristics who experience the same life events at the same time
Panel Studies
focuses on a household
Record Linkage Studies
link together administrative records for the same individual over time
Prospective Studies
follow an individual's life
Retrospective Studies
Individual is asked to recall their life
Types of Correlations:
cross-sectional correlation
autocorrelations
cross-lag correlations
Cross-Sectional Correlation
test whether to see if two variables are correlated when measured at the same point in time
Autocorrelations
determines the correlation between one variable and itself when measured at two different points in time
Cross-lag Correlations
determine whether the earlier measure of one variable is associated with the later measure of the other variable (helps establish temporal precedence)
Apparent correlations are a function of:
random chance
fabrication of data
an actual casual relationship due to the third variable
What should authors do after finding significant effects in their pilot data?
They should not keep the pilot data and conduct a new experiment with the same design but different sample and data
What should authors do if they want to publish a paper with a known confound?
qualify what they found
be honest and say there was a confound in the publication
full disclosure and include in the limitations that there was a confound
What should authors do if they conduct multiple analyses on their data?
conducting multiple analyses on data the risk of making a type 1 error increases
lower alpha values so that a type 1 error is less likely to occur
Why are reversal designs potentially unethical?
some might question the ethics of withdrawing a treatment that appears to be working
may be unethical to use a treatment that is not empirically demonstrated to be effective
Interaction Effect
whether the effect of the original independent variable depends on the level of another independent variable
Crossover Interaction
the original IV depends on the new IV and causes a graph of the interaction to intersect
ex: preference for hot food or cold food?
hot>cold for pancakes
cold>hot for ice cream
Spreading Interaction
DV (how often dog sits) is higher at one level of the original IV (saying sit) combined with the new IV (holding treat) and is lower or non-existent at the other levels (saying sit/nothing and not holding a treat)
What happens to a main effect when an interaction is found?
when a study shows both a main effect and an interaction, the interaction is always more important
Main Effect
the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable
How do researchers discuss a two-way interaction?
in a three-way design, there are three possible two-way interactions
to inspect these interactions, one must construct 2x2 tables, compute the means of each cells and investigate the difference in differences
the effect of 1 variable DEPENDS on the level of the other variable
look for "it depends" or "only when"
How do they discuss a three-way interaction?
in a design with three independent variables, the three-way interaction tests whether two-way interactions are the same at the levels of the third independent variable
Selection Effects
bias introduced by the selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved
ex: participants choose to be in a certain group
Solutions for Selection effects:
random assignment and matched sets (participants sorted lowest to highest and grouped into sets of two)
Carryover Effects
any lingering effects of a previous/other experimental condition affecting a current experimental condition
ex: in a study that tests if caffeine use has an effect on test scores, an individual in the "no caffeine" group regularly used caffeine prior to study
Solution for Carryover Effects
counterbalancing: presenting IV to participants in different sequences, A to B vs B to A
Observer Effects/Bias
results of an experiment biased due to experimenters' expectations on a particular task, creating an implicit demand for the participants to perform as expected
Solution for Observer Effects/Bias
blind studies (masked design)
Hypothesis
a proposed explanation based on something that you observe as a starting point for further investigation (a prediction, something you test)
Theory
an in-depth statement describing general principles about how variables relate to one another within a scientific phenomenon
Law
a universal single statement that is accepted to be true usually involving math
Characteristics which define a true experiment
random assignment of subjects
manipulation of an IV
correlation
temporal precedence
internal validity
Systematic Variability
makes internal validity a problem by not controlling for a variable so it affects all participants at all levels of the study in the same way
nonrandom: flaw in design
ex: showing up for a study where some people got 9 hours of sleep vs. 5 hours of sleep and pairing all those with 9 hours of sleep together in one group
Unsystematic Variability
has a random or haphazard variable which can obscure differences in the independent variable
individual differences: some people do great with 5 hours of sleep and other don't
random
noise
variable spread evenly
Mediator
definition: reason for effect
X affects Y because X leads to M which leads to Y (X/Y relationship is implied because of M)
mediator must be a casual result of IV and a casual antecedent of DV
ex: exercise -> calorie intake -> weight loss
(exercising more increases your calorie intake which influences weight loss)
Moderator
definition: contextualizing the effect
cannot be a casual result of IV
influences to what extent IV affects DV
ex: exercise -> weight loss (gender, age and previous weight are moderators because exercise does not cause them but they influence how much exercise affects weight loss)
Multivariate Design
study involves more than two variables
Posttest Only
participants are randomly assigned to the independent variable level and measured once on dependent variable after exposure to the independent variable
Pretest/Posttest
participants are assigned to the IV measured twice on DV (once before being exposed to the independent variable and once after being exposed to the independent variable)
Repeated Measures Design
participants are measured on the dependent variable more than once; after exposure to each IV condition
Concurrent Measures Design
participants are exposed to all IV levels at the same time and single attitudinal or behavioral precedence is dependent variable
Pilot Study
study using a separate group of participants completed before conducting the study: checks how well independent variables are operationalized
Matched Groups Design
one in which the experimenters will measure all variables of the participants that could have an effect on the measured variable
will match like participants and split them evenly between all different groups so that variation between the groups decreased
increases internal validity
Factorial Design
crossing two IVs and studying the effects of each possible combination of IV
must have participant variable
Stable-baseline
when a researcher observes behavior for an extended baseline period before beginning an intervention
proves that the behavior was not changing before the intervention
Multiple-Baseline
researchers stagger their introduction of an intervention across a variety of contexts, time or situations
Reversal Design
researchers observe a behavior before and during a treatment and then stop the treatment to see if the behavior issue returns/reverses
Quasi-experiment
a study similar to an experiment except that the researchers do not have full experimental control (they may not be able to randomly assign participants to independent variable conditions)
Nonequivalent Control Group Design
an independent-groups quasi-experiment that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups
Nonequivalent Control Group Pretest/Posttest Design
an independent groups quasi-experiment that has at least one treatment group and one comparison group.
participants have not been randomly assigned to the two groups and at least one pretest and one posttest are administered
Interrupted Time-Series Design
a quasi-experiment in which participants are measured repeatedly on a dependent variable before, during and after the interruption caused by some event
ex: A factory wants to measure worker productivity. They measure productivity every week, halfway through the experiment they shorten shifts from 10 hrs to 8 hrs. Following the intervention productivity improves. Pretest was for the 10hrs, week of reduction was during “interruption”, measurements after that are post-interruption.
Nonequivalent Control Group Interrupted Time-Series Design
a quasi-experiment with two or more groups in which participants have not been randomly assigned to groups; participants are measured repeatedly on a dependent variable before, during, and after the “interruption” caused by some event, and the presence or timing of the interrupting event differs among the groups
ex: same thing as above but this time, do it before christmas for the first floor of the factory , and after christmas for the second floor.
Wait-list Design
an experimental design for studying a therapeutic treatment, in which researchers randomly assign some participants to receive the therapy under investigation immediately, and others to receive it after a time delay
Small-N Design
a study in which researchers gather information from just a few cases
Single-N Design
a study in which researchers gather information from only one animal or one person
Sources of Error
statistical fluke
situation noise
individual differences
errors in initial study design
lack of/improper execution of replication
conscious fraud
unconscious bias
mistake in data analysis
multiple DVs
underpowered
measurement
no effect in study
Parametric Study
looks at the impact of changing certain parameters within the study
What defines a quasi-experiment?
researchers do not have full control
participants are not randomly assigned
used when researcher is interested in IV that cannot be randomly assigned
ex: personality traits to assess intelligence: participants assigned based on personality
What defines a small-N study?
researchers use a small sample size bc they expect a large effect size
obtain a lot of info from a few cases rather than a little info a lot of cases
each participant is treated separately
data represents each individual rather than groups
Internal Validity (N-study)
manipulation check
within-subject that repeated measured behavior before and after intervention
External Validity (N-study)
triangulated by combining the results of the N studies with other studies
researchers can specify which population they want to generalize to
researchers may not be concerned with generalization at all
Construct Validity (N-study)
multiple observers
interrater reliability
Statistical Validity
do not use traditional statistics but still treat data appropriately to provide evidence
Disadvantages of N-study
internal validity issues in narrowing down what is specifically responsible for observed effects