How much does one variable change across observations & what is typical or average
How two or more variables change together
Describe relationships among observed variables and presume the relationship is consistent and universal.
- Include assumed causes & unobserved mechanisms that explain how and why variables are related (whether consistently lawful or not)
predict what will occur in a specific situation based on prior observations and/ or the logical implications of theories.
What makes some theories better? (more plausible)
- Make unique & specific predictions
- Consistent with established theories
- Explain the most data
- Simplicity (parsimony)
- Each assumption has a probability of bring wrong
- More assumptions=more probability of being wrong
- Unless assumptions verified by better prediction, explanation, consistency
5 Limitations of Personal Experience
- Expectation Bias: our expectations tend to influence the way we perceive and interpret events.
- The Confirmation Bias: We tend to only use experience and info that are consistent with our current ideas.
- Limited Data: We make inferences from very little info.
- No Baserate/Comparison Group: We often lack and overlook the importance of comparisons, when inferring a relationship between variables.
- Lack of Control: We don't systematically control the variables, which is needed to infer a causal relationship.
How research overcomes these limitations
- Defines variables in terms of specific, objective criteria.
- Uses settings where observations is optimal.
- Evaluates the reliability and validity of observations.
- Records and replicates observations.
- Collects larger, random sample & estimates chance.
- Uses appropriate comparisons
- Manipulates variables and uses statistical controls to test competing causal explations.
- What is the complete idealized essence of a thing?
- -Concepts are mystical ideals that may or may not exist
- -What should a word apply to?
- -No objective answer & evidence is irrelevant
- What are the real world properties of a thing?
- -Concepts correspond to the actual world.
- -They must be connected to what is observable.
- -Abstract ideas are shaped and reshaped empirical data
- An actual thing/event in the physical world
- Directly observable and measurable
- Idea or concept
- Not directly observable or measurable
- Either a particular variable or composed of multiple variables
To test theories, we must
- Define the abstract concepts in terms of observable variables.
- Then create operational definitions
- -The precise operations, methods, and procedures used to assign a quantitative value or a qualitative category to each observation.
An operational definition should...
- Specify exactly what was measured and how.
- Allow for placing each observation on a numerical scale along 1 or more dimensions.
- Continuous vs categories
- Presence vs Absence of a particular thing is Quantitative
- Numerical value does not always translate into exact number of "units"
- Abstract variables composed of multiple variables that cannot be separated.
- Inherently "Confounded"
- -Limits ability for explanation
- Why are they used so often?
- -Still allows us to describe relationships
Why is specificity so important?
- VERY IMPORTANTIncreases reliability
- Allows for evaluation of validity
- Allows others to replicate the work
- Reduces opportunity for fraud or expectancy bias
Refining our constructs
- Concepts are like theories
- Concepts should be modified in light of data
- -Reduces the influence of other variables
- -Scores better representation the underlying concept
- -More complex to use
- -Units of measurement no longer have concrete meaning.
- Observed score= True score + Random error + Systematic error
R: Random, inconsistent influences on measurements
Lack of control or specifying in the method
S: Systematic, consistent influences on measurements
- Calibration error in equipment or consistent error by observer
- Same amount and direction for each observation
Random Error & Reliability
Lots of R can reduce reliability.
How to reduce random error?
- To reduce amount of R in each observation:
- -Identify and eliminate possible random influences
- To reduce the effects of R on overall results:
- -Measure each observation multiple times and take average
- -Collect a large sample of observations
- Measuring what you think
- If unreliable, then its invalid
- -Lots of R mean O unequal to T
- -Reliability is rarely perfect
- But, reliable measures can still be invalid
- -Due to systematic error
- -Reliability is just the first step in ensuring validity
How do we assess Validity?
- Predictive Validity
- Does it predict scores on previously validated measures of the same thing?
- Why not just use the other measure?
Should not correlate with variables that are supposed to be unrelated
How do we improve Validity?
- Pick a new operational definition
- Improve reliability
- Create a Latent Variable
- We only know the observed score
- We can only estimate R and S
- W never know for sure that O=T
- So, we must take steps to minimize any potential error and its impact