ex. gender, race, where you live, political affiliation
much test data are qualitative
ex. responses to a given test item can be right or wrong or a,b,c, or d
quantitative data
test scores, reaction time, heart rate, brain-wave measurements
lures/distractors
wrong answers on a multiple choice test
item analysis
if a test has only two answers (the right and wrong answer) then indicating which answer was selected is basically the same as indicating whether the correct answer was chosen
Two kinds of test theory
classical test theory (older)
IRT (item response theory) newer replacing classical test theory
item discrimination
the ability of the item to tell the difference between different people
an item that everyone got right does not discriminate
an item that everyone got wrong does not discriminate
items that are too easy or too difficult have low discriminations
item-test/item whole correlation
the correltation between a particular item and the scores on the entire test
what does it mean when the test-item correlation is high
it means that ppl who got the item right tend to have high test scores, and ppl who got it wrong tend to have low test scores.
the item discriminates between ppl who knew the material and ppl who didnt
the test item correlation is used as an index of item discrimination
average test item correlation
tells how much they all tend to intercorrelate.
can also be called measure of test coherence
measure of test coherence
whetherthe items all test the same thing
average intercorrelation
called Cronbach's Alpha and is the most standard measure of test reliability
Kuder-Richardson
the version of this formula that applies totests where there are right and wrong answers
difficulty
the percent correct for an item
reliability
the item-whole score correlation
process of test development
1. make up a bunch of test items and create a test
2. give the test to alot of ppl
3. get the correlations among the items
4. throw out items that correlatepoorly with the reamining items
5. if there are too few items left, creae sme new ones and start over
6. continue this until the test has enough items tha all have high correlations w/ each other
multivariate research
gathering many variables
testing and survey research
multivariate statistical analysis
taking complex data and reducing it to simpler forms
qualitative (multivariate methods)
categorical or classification data
ex. gender, race, poltical affiliation
quantitative (mulivariate methods)
test scores, reaction time, heart rate, brain-wave measurements
factor analysis
a method for construct validation
gives us a method to Xray inside the vlack box of abstract construct that we are interested in
properties of a correlation matrix
it contains the correlations or every item with every other item
it is square
the diagonal elements represent the correlation of each item with itsel, and so these are always 1
it is symmetrical
simplifying the correlation matrix
technique for packaging together variables into super variables
do this by looking for items that correlate highly with each other, and packaging them together
if looking at one trait all items should intercorrelate, and should get one super variable that can be name after the trait
True/False:
Factor analysis and PCA are similar techniques
true
factors
both factor analysis and principal components analysis boil down the correlation matrix to a smaller number of super variables