How much does one variable change across observations & what is typical or average
How two or more variables change together
Describe relationships among observed variables and presume the relationship is consistent and universal.
- Include assumed causes & unobserved mechanisms that explain how and why variables are related (whether consistently lawful or not)
predict what will occur in a specific situation based on prior observations and/ or the logical implications of theories.
What makes some theories better? (more plausible)
- Make unique & specific predictions
- Consistent with established theories
- Explain the most data
- Simplicity (parsimony)
- Each assumption has a probability of bring wrong
- More assumptions=more probability of being wrong
- Unless assumptions verified by better prediction, explanation, consistency
5 Limitations of Personal Experience
- Expectation Bias: our expectations tend to influence the way we perceive and interpret events.
- The Confirmation Bias: We tend to only use experience and info that are consistent with our current ideas.
- Limited Data: We make inferences from very little info.
- No Baserate/Comparison Group: We often lack and overlook the importance of comparisons, when inferring a relationship between variables.
- Lack of Control: We don't systematically control the variables, which is needed to infer a causal relationship.
How research overcomes these limitations
- Defines variables in terms of specific, objective criteria.
- Uses settings where observations is optimal.
- Evaluates the reliability and validity of observations.
- Records and replicates observations.
- Collects larger, random sample & estimates chance.
- Uses appropriate comparisons
- Manipulates variables and uses statistical controls to test competing causal explations.
- What is the complete idealized essence of a thing?
- -Concepts are mystical ideals that may or may not exist
- -What should a word apply to?
- -No objective answer & evidence is irrelevant
- What are the real world properties of a thing?
- -Concepts correspond to the actual world.
- -They must be connected to what is observable.
- -Abstract ideas are shaped and reshaped empirical data
- An actual thing/event in the physical world
- Directly observable and measurable
- Idea or concept
- Not directly observable or measurable
- Either a particular variable or composed of multiple variables
To test theories, we must
- Define the abstract concepts in terms of observable variables.
- Then create operational definitions
- -The precise operations, methods, and procedures used to assign a quantitative value or a qualitative category to each observation.
An operational definition should...
- Specify exactly what was measured and how.
- Allow for placing each observation on a numerical scale along 1 or more dimensions.
- Continuous vs categories
- Presence vs Absence of a particular thing is Quantitative
- Numerical value does not always translate into exact number of "units"
- Abstract variables composed of multiple variables that cannot be separated.
- Inherently "Confounded"
- -Limits ability for explanation
- Why are they used so often?
- -Still allows us to describe relationships
Why is specificity so important?
- VERY IMPORTANTIncreases reliability
- Allows for evaluation of validity
- Allows others to replicate the work
- Reduces opportunity for fraud or expectancy bias
Refining our constructs
- Concepts are like theories
- Concepts should be modified in light of data
- -Reduces the influence of other variables
- -Scores better representation the underlying concept
- -More complex to use
- -Units of measurement no longer have concrete meaning.
- Observed score= True score + Random error + Systematic error
R: Random, inconsistent influences on measurements
Lack of control or specifying in the method
S: Systematic, consistent influences on measurements
- Calibration error in equipment or consistent error by observer
- Same amount and direction for each observation
Random Error & Reliability
Lots of R can reduce reliability.
How to reduce random error?
- To reduce amount of R in each observation:
- -Identify and eliminate possible random influences
- To reduce the effects of R on overall results:
- -Measure each observation multiple times and take average
- -Collect a large sample of observations
- Measuring what you think
- If unreliable, then its invalid
- -Lots of R mean O unequal to T
- -Reliability is rarely perfect
- But, reliable measures can still be invalid
- -Due to systematic error
- -Reliability is just the first step in ensuring validity
How do we assess Validity?
- Predictive Validity
- Does it predict scores on previously validated measures of the same thing?
- Why not just use the other measure?
Should not correlate with variables that are supposed to be unrelated
How do we improve Validity?
- Pick a new operational definition
- Improve reliability
- Create a Latent Variable
- We only know the observed score
- We can only estimate R and S
- W never know for sure that O=T
- So, we must take steps to minimize any potential error and its impact
What is the difference between the types of questions science can answer and those it cannot?
Some questions are about subjective preferences and not about what is factually true.
What are the non-scientific ways of knowing, why do we use them, and what are their limitations?
Emotion, Personal experience, Authority, Logic, and Science. Easy-no effort, sometimes is the only "evidence" we have, people prefer positive emotional states, and we prefer our various ideas to be logically compatible. Premises are rarely complete, what feels good has no consistent relationship to what is true, faulty perceptions, and authorities may lie for their own good.
How do various limitations of our personal experiences llimit their usefulness as evidence or data when drawing conclusions?
Authority, Emotion, and Personal experience are all highly prone to error and affected by bias and other factors unrelated to accuracy and truth. Also, they are not "self-correcting" no mechanisms by which errors are eventually corrected.
What are univariate and multivariate questions and how do they relate to the goals of descrption versus explanation?
Univariate questions are how much does one variable change across observations & what is typical or average? Multivariate questions are how two or more variables change together. Goal is to describe or explain the variability in observed objects or events. The explanation is the WHY? to the question (which is multivariate)
What role does empirical observation play in science and how is it used to test theories?
Empirical observations are publicly observable, anyone could witness and measure it. The thing or event itself does NOT need to be directly observed, only objects or events that are logically related.
What aspects of scientific method make it self-correcting?
Systematic and Empirical observations are aspects of the scientific method that are self correcting.
What are the necessary features of a scientific theory and the criteria for evaluating good and bad ones?
- Scientific theories: a detailed causal explanation of observed events that is testable against evidence and can predict future events.
- make unique & specific predictions
- consistent with established theories explain the most data
- simplicity (parsimony)
What is the logical argument that justifies preferring theories with greater "simplicity"
- Simpler = more plausible
- each assumption has a probability of being wrong, more assumptions = more probability of being wrong, and unless assumptions verified by better prediction, explanation, and consistency.
What are the requirements of an operational definition and what steps do we take tog o from an abstract concept like happiness to an operational definition?
Concepts correspond to the actual world. they must be connected to what is observed. abstract ideas are shaped and reshaped empirical data.
What are the defining features of and the limitations of qualitative variables?
Qualitative variables are abstract variables ccomposed of multiple variables that cannot be separated. they limit ability of explanation.
How do random and systematic measurement errors differ, what
causes them, how can we assess them, how can we reduce them, and how do they each relate to reliability and validity?
Random error has inconsistent infuences on measurements. systematic error has consistent influences on measurement. Lots of R can reduce reliability, its consistent if you assess the same thing multiple times. to reduce R identify and eliminate possible random influences. reliability measures R and validity measures S.
For each of the methods for assessing reliability and
validity, how would the results differ between a reliable and unreliable measure, and between a valid and invalid measure?
How and why does sample size affect both our measurement of
an individual observation as well as our samples of observations?
The sample size may not reflect the population as a whole. sample size can reduce random sampling error or differ population if too small.
Why do the different types of sampling error occur, how do they influence our results, and how can we reduce these influences?
They occur without measurement error. they can hide real relationships between variables or produce " fake" relationships
What are the various descriptive statistics used to describe a sample on a particular variable, and which statistics apply to which kinds of measurement scales and why?
Central tendency & dispersion (variability)
Qualitative & Quantitative
- Ordinal Scale: Exact quality between each category unknown, irrelevant, or inconsistent.
- Interval Scale: Quantity between each value remains constant
- Ratio Scale: Can compare scores in terms of ratios
most frequent; highest number
The highest point where no more than 50% of acores are below it
Why 3 different types of "average"
the mean & median only apply to some scales
On a curve graph
the highest point is the mean
- comparing 2 possible scores
- lowest and highest scores
The typical amount that acores differ from the mean.