# 272 U3

 What are the 3 reasons for assessing IOA? Consistency of measurement Minimizes observer bias reflects the quality of operational definition What is the difference between agreement between observers and accuracy?  Is it possible to have one but not the other?  Explain. Agreement - how well data from independent observers correspond Accuracy - How well the data reflects what actually happens Yes it is possible, two people can make mistakes in the same way therefore you can have agreement but not accuracy definition could be too broad, thus you have accuracy but not agreement What is the difference between and reliability and validity? Can you have one without the other? Explain. Reliability - consistent measurement of the construct Validity - measurement reflects the construct yes. you can measure something reliably and it not reflect the constructand you can measure a construct inaccurately Is accuracy and validity the same? Why or why not? No, you can accurately measure a bx that has nothing to do with the overall construct measuring eye contact for reading comprehension Complete the following sentence using the terms “accurate” and “reliable.” If data are _________, they must be ___________. Why is the converse not true? Give an example. accurate, reliablethe converse is not true because two people can take data inaccurately in the same way which would make it reliable, but not accurate Do you have to check IOA during each phase of a study?  Why or why not? yes, IOA should be assessed in at least every phase of the experiment to ensure consistency Be able to calculate IOA using the frequency ratio method, Point-by-Point Agreement Ratio method (scoring agreements as occurrences and non-occurrences), and frequency-within-interval method. FREQ RATIO - smaller total/larger total X 100 Point By Point - Agreements/agreements +Disagreements X 100 Freq within interval ? pg 193 What does “chance” refer to?  How might one handle expected “chance” levels of agreement? level of agreement expected by randomly marking occurences compute IOA using agreements on nonoccurences In addition to “chance,” there are 4 other sources of bias that can obscure interpretation of interobserver agreement.  Describe each source of bias. reactivity of reliability assessment - bx changes when you know your work is being checked observer drift - change in the application of definitions of bx over time Expectancies and Feedback - when observers expectations of change and feedback from experimenters combine, it may influence data taken by observer Complexities of observations - the less complex the observations, the higher the agreement Define reactivity and observer drift. Reactivity - changes the people make when their bx is being monitored or evaluated Observer Drift -  change in the application of definitions of bx over time Given that interobserver agreement provides no information about either accuracy or reliability, why do you suspect that this is the most common approach to evaluating measurement procedures and data? requires the least amount of response effort  and is the best representation of the construct of accuracy and reliablity When using an interval agreement procedure for calculating IOA, explain how the percent agreement can be inflated by especially frequent or infrequent responding. especially frequent refers to that fact that agreement will be high because of the number of intervals in which the behavior occurs.  If it is occurring a lot high agreement may happen  INfrequent responding referes to agreements based on non-occurrence which would be the same for low freq bxs but in reverse. Authoratlborn80 ID202073 Card Set272 U3 Description272 U3 Updated2013-05-04T05:52:24Z Show Answers