matched pairs design
- a between-subjects design
- subjects randomly assigned as pairs after being matched on potentially confounding variables (the two groups have same set of characteristics)
- to control for subject variability
advantages/disadvantages of matched pairs
- advantages - no practice/fatigue effects across conditions - no carryover effects
- disadvantages - larger sample needed, need to know what variables need to be matched
within subjects design
- ultimate matched-pair design (each subject matched with themselves)
- Also called repeated measures design
- controls for subject-related variability
- use paired t-tests
- BUT carry-over effects are possible
advantages/disadvantages of within-subjects
- advantages - controls for subject variability, more power, fewer subjects required (in principle)
- disadvantages - practice/fatigue effects, awareness of changes across conditions, order effects (can be subject to task demands - know what is being manipulated)
how to deal with order effects
- vary sequences of conditions
- to vary the order systematically so that the order effects was out
- create sequences so that
- 1. every conditions appears in every possible position in the order the same # of times
- 2. every condition precedes every other condition as many times as it follows that condition
- then randomly assign participants to sequences
advantages/disadvantages of counterbalancing
- advantages - balance order, any differences due to fatigue/practice or subject variability should was out
- disadvantages - requires more subjects, analysis is more difficult
types of counterbalancing
- 1 - each subject gets all possible orders (reverse counterbalancing, block randomization)
- 2 - each subject gets only one possible order, but across subjects all orders accounted for (complete counterbalancing)
- A-B, B-A
- particular individual does each order
- present different orders - one is just reverse order
- order randomized within blocks - subjects receive all blocks
- ABC, BAC, CBA, ACB
- would be a really long experiment!
- = N! orders
- each subject gets one order - but across all subjects all orders are accounted for
Simple Latin Square
- partial counterbalancing
- each condition appears in each position in the sequence
- DOES NOT fulfill "every condition precedes every other condition as many times as it follows that condition"
balanced latin square
- full counterbalancing
- only works for even # of conditions (2x2, 4x2)
- fulfills both counterbalancing conditions
- at least one between subject IV and one within-subject IV
- Still do ANOVA - same general test for factorial!
- summarizing a set of empirical data
- central tendency (mean, median, mode)
- variability (the spread, SD)
- drawing conclusions about the broader population
- - infer what your sample means about the larger population
- - if you aren't interested in the broader population - inferetial not necessary!
variability and 3 sources
- represents background "noise"through which we have to detect the experimental "signal"
- NEED to reduce extraneous variability
- sources: within groups (different individuals contribute to the data), between groups (because of IV's), between groups (because of extraneous variables - DO NOT WANT)
do we have enough confidence that between groups variability is larger than variability within groups
how likely is it that the IVs had an effect over and above the variability due to random sampling?
- based on probabilities!
- "inferences under uncertainty" - we make statements about how probable it is that the population means are different
- start with a hypotehsis that is opposite of what we are looking for (null) and see if data is improbable given that hypothesis
- null vs. alternative hypothesis - any differences due to treatments
- any given sample is just one of many
- b/c of the many different samples that could be made with the population - their actual distributions will be different because of noise variability
- if you average all possible samples - end up on average with distribution that looks like population
alpha = .05
- the probability that teh difference you are observing in your data is from a different population
- reasonable doubt = 5%!
Type I error
- null is true, but we reject null and assume that there is an effect
- nothing there but we think there is!!
- equal to alpha
Type II error
- we failed to reject the null - but there truly is an effect!
- something there but we don't think there is!
- equal to Beta
- 1-Beta (1-probability of type II error)
- ability of independent manipulation to detect an effect! - to reject null when should reject it!
how your independent variable caused some change in the dependent variable
- analysis of variance
- when there are more than two groups!
rely on random assignment/random sampling to make sure the groups don't differ in ability
- can we generalize from results
- results then must be internally valid and replicable! - a
- 1. to different samples - are your conclusions not limited to your specific sample?
- 2. to different procedures/operationalizations? - how does your research mesh with others?
- 3. to the world at large
how would you increase external validity?
- aggregations - multiple subjects, trials, vary the stimuli, measure it in different ways
- multivariate measurement (multiple dependent variables and see hwo they pattern together)
- nonreactive studies - non-experimental
- naturalistic and field studies - non-experimental - people more likely to act normally
more you increase external though, you may decrease internal!
Power - depends on...?
- N (sample size)
- small N = higher probability of type 2 error
- which would be a higher power!??
Qualitative vs. Quantitative Research
- Both involve observations of behavior!
- Quantitative - more control, less open-ended, reactivity, manipulation, inferential, numbers
- Qualitative - less control, more open-ended, naturalistic, spontaneous, descriptive, categories
Why do survey research?
- to assess how people feel abotu an issue
- to examine relationships across responses
- to dispel myths
- to gather scientific information
what kinds of problems do you want to avoid in survey research?
- want one issue per item
- want to avoid bias/leading questions
how to construct surveys
- make alternatives clear (mutually exclusive and exhaustive)
- beware of social desirability - use neutral statements!
- beware of acquiescence/response bias - when respondents tend to agree to any statement
- think about item format/sequencing (but beware of context effects!)
- representativeness - does sample exhibit same distribution of characteristics as the intended population to be studied?
- randomness - does each individual in population have equal likelihood to survey
Different kinds of sampling
- probability sampling (simple random, stratified, cluster)
- non-probability sampling (non equal probability) - convenience/haphazard, quota sampling (consider population characteristics)
- mail surveys
- one-on-one interviews
- telephone interviews
- internet surveys
- examines degree of relationship between 2 behaviors/events
- changes in one are associated with changes in other (multivariate)
- measures the extent of co-variation between two (or more) dependent variables
- typically reported in terms of "r"
types of coreelational studies
- observational research
- survey research
- archival research
correlation can be misleading
- Restrictive range (by restrict range of the data = different interpretation)
- outliers (can make it look stronger or weaker)
- nonlinear - a U-shaped relation or a hump (correlations assume simple lines!)
correlation vs causation
- correlation is necessary for determining causation but is not sufficient
- X --> Y
- Y --> X
- or a third variable problem (Z could cause X and Y)