Research Methods

General Scientific -- at some point we believe that something is real.

Paradigmatic -- Which methods make sense and which don't. They may evolve over time.

Domain -- How generalizable is it?

any thing or concept that can take on more than one value

A concrete variable is something that you can take a direct measure of.

e.g. how tall are you

and abstract variable is something that can not be directly measured, you must measure other things that correlate with the thing you are interested in.

e.g. how smart are you

A hypothetical construct is an abstract concept inventer to refer to a variable that can not be directly observed.

e.g. Intelligence

To make abstract concepts concrete enough to study.

a variable that that must be present for a relationship between to variable to occur.

e.g. if parents are home kids won't get in the liquor cabinet, but if parents are home they will.

There is a relationship between the two variables, but it changes depending upon if there is a moderator present.

e.g. more births in Stockholm during the fall probably because more people are staying in during winter.

• Cause must preced effect
• Cause and effect must covary
• There must be no other plausible explanation for the effect.

• Data in number form
• Often indicates quantity or frequency

• Data reflects categories
• Phenomenological research that focuses on what the meaning of things are.

• Experimental
• Field Experiments
• Quasi-experimental
• Correlational
• Case Study
• Single Subject

• Researcher is in control and manipulating the independent variable.
• Attempt to determine causality
• e.g. irritate people then ask them to deliver a shock to their partner. sometimes have a gun present near shock switch. measure if shocks are bigger with gun present.

• experiments done in the real world instead of the lab.
• harder to control some things
• e.g. cover someone in blood and send them out to a park watch and see how many people are willing to help them

• Nonrandom assignment of participants
• Still try to look at whether we can detmine causality, but not as strongly as we can with experimental research.
• e.g. divide a classroom up by kids last names and complete an education study.

• also called naturalistic
• looking at relationships between one or more predictors and one or more criteria.
• Do not manipulate an IV
• can not determine causality
• e.g. looking at the correlation between SAT scores and college GPA

• looking at one case or one individual
• may be difficult to generalize unles syou have multiple case studies
• Can not attribute causality

• Single individual is the focus, but it can be experimental if you manipulate an IV
• Good to use in private practice

• Cross sectional
• Longitudinal
• Cohort Sequential
• Time Series
• Prospective
• Retrospective

• Look at a variable at a particular point in time, but across groups
• We can show that there are relationships, but can not show causality
• e.g. looking at word knowledge using groups at different ages.

• Look at a variable in the same subjects over a period of time.
• can show a causal relationship
• e.g. measure attachment in babies, then again at 5, 10, 15..etc.

Try to speed up longitudinal by getting groups at different ages that have common factors and looking at variables

• Look at things after the fact to see if we can tell what caused them.
• e.g. food poisoning. if a bunch of people got sick, what do they have in common

• Truth test -- is it believable
• Utility test -- how useful is it

• Content
• Criterion
• Construct

• Does this cover what we think it should cover?
• Do the things we are looking at logically get to the thing we are trying to study?
• e.g. if we want to know about intelligence does measuring math ability get to it?
• often demonstrated through agreement between experts.

• Does this measure correlate with other measures that we think it should?
• must be calibrated against a known measure or against itself over time

Testing the measure over time to see if the same results are found

• comparing the measure to another establish measure.
• e.g. comparing the results of your intelligence test to the results of an established intelligence test. If you see similar scores you have concrete criterion validity.

• The degree to which two things are not similar
• e.g. math knowlege and reading knowledge may not correlate.

• The degree to which things that should be similar are.
• e.g. math knowledge and math knowledge should correlate

• Deals with the consistency of a measure
• How well will we be able to reproduce a score?

• Test-retest
• Alternate forms
• Internal consistency
• Inter-rater

• repeat measures twice and see if we get similar results
• e.g. measure people's height twice
• e.g. give people a word knowledge test today and then tomorrow
• must be careful that subjects may remember some of what was on the first test and this could affect their scores on the second

• Make up two tests that are supposed to measure the same thing and give the subject both of them.
• may be difficult to create two tests that are really the same but different

• measure the correlation between different items on the same test.
• split halves test...write one test split it in two and see if halves correlate

• a statistical analysis that tells us how well items agree with each other.
• If they agree we have high reliability

• If we get measures from different sources how much to they agree.
• e.g. if you have two people observing the same behaviors are their rating similar?

• who do we expect to be able to generalize the finding to?
• e.g. reinforcement can be generalized across a large population, but tourette's may not.

The subset of people from the population who are actually being studied

everyone in the population has the same probability of being in the sample

Sample parallels the population on important things

• Who should we obsrve
• When are we observing them
• What are we going to observe
• Where are we going to observe them

• Find the population
• Determine sampling frame

a list of all members of the population

• Split the population into groups based on attribues, then sample randomly from the groups
• Difficult to get a truly random sample, and the population you want to generalize to may be incredibly large.
• e.g. divide the population into male and female and then randomly sample

• Natural groups exist, so we take a random sample from that.
• e.g. middle school kids may be questioned for marketing research

• take every nth person in the sampling frame.
• may run into bias if there are periodic or cyclic ordering in the sampling fram
• e.g. take every third patient that signs in at a doctors office.

• subjects are picked because they are easy to get.
• may leave out important groups, or get sampling bias because certain people tend to participate
• e.g. psychology students at univeristy

• see if particpants can put you in contact with other people that might be good participants.
• e.g. people being treated for a rare disease may know other people with the same disease.

• You want to look at a specific group so you purposely select people from that group.
• e.g. asking to use schizophrenics at a state hospital.

the ability to detect an effect

• the size of the effect
• the size of the type I error
• the variability
• reliability of the measures used

• False positive result
• denoted by alpha
• equals the significance level

• False negative
• beta
• 1-beta is the power

• the variability of the thing being studied
• e.g. look at the difference int he size of coins. Pennies and nickels are easy to tell abpart because they are all basically made the same size, not much deviation from the norm.

• Who says so
• How do they know
• What's missing
• Did someone change the subject
• Does this make sense

two or more variables are combined so the effect of them can not be separated.

• Natural -- age, race, cohort
• Treatment -- some don't want to take drugs
• Measurement -- if we give a questionnaire out that is higher than participants reading level we may not be measuring what we think we are.

the ability to make causal attribution about variables based on the design and results of the study

The ability to generalize the results from the study samply to another population or situation

• History
• Maturation
• Testing
• Instrumentation change
• Statistical regression
• Selection effects
• Reactivity

• During an experiment subjects may be exposed to life events that impact the outcome.
• e.g. 9/11, or starting school for little kids

• Maturation effects occur over the normal passage of time. normal growth.
• e.g. kids may have more word knowlege at the end of and experiment than they did at the beginning simply because they are maturing.

• completing pretests may affect outcome.
• e.g. if we look at word knowledge subjects may remember some of the words from the pretest during the posttest

• changing what you are using to measure, instructions, or methods can all affect the outcome of a study.
• e.g. observers may not agree overtime, you need to recalibrate their definitions of behaviors

• group mean tends to get closer to the norm upon retesting
• e.g. if you have a group that is on the extreme end of anxiety they will tend to be closer to norm when retested.

• the sample may be biased.
• nonrandom assignment can affect outcome
• random assignment may not be as representative as we thing.
• e.g. age, race, and SES can affect outcome

• evaluation apprehension -- don't want to look bad infront of wife
• Novelty effects -- may not behave the same after weeks of study.

• blind or masked experiment.
• This means that the person doing the research doesn't know group assignments.

• Good subject
• Negativistic subject
• Faithful subject
• Apprehensive subject

subject wants to give information consistent with what they perceive the goals of the study to be.

subject provides information contrary to their perceived goal of the study.

subject tries to follow the instructions to the letter

subject is concerned about performance and being judged.

• cues that are inadvertently part of the experimental situation that influence how subjects react.
• e.g. a baseline measure on PTSD symptoms based on self report alerts the participant the we are trying to reduct PTSD synptoms

both the frontline researcher and the participants are unaware of group assignments

• experimenter's expectations regarding the results may influence the outcome
• e.g. if experimenters are aware of what group subjects are in, they may inadvertently treat them differently.
• e.g. clever hans. horse was stomping unitl experimenter reacted, not because he could count

• reduce the cues you give people
• figure out how to motivate them to give you a more realistic answer
• role play control groups

• Structural -- what are our methods
• Functional -- real world vs. experiment
• Conceptual -- does it matter to people

• Use of control group
• Random assignment to groups
• Researcher manipulates the IV and observes impact on DV
• maximum possible attribution of causality

proves causal relationship between what we are doing and what is hapening

Causality is already established, but we want to know if it will work for our particular client.

to evaluate reliability over time.

when items are taken from a theoretical population of items

when symptoms, behaviors etc. are not causally related to each other

• are the items in a test consistent with the definition
• agreement between experts
• alternative method

• do the scores correlate with other scores that it should
• does an ability measure correlate with performace

do items converge or diverge?