Experimental Methods and Design 2

Post-Hoc comparisons are comparisons run after data collection.

Number of possible comparisons c = (a)*(a-1) with a=# of groups. To determine pairwise comparisons just divide by 2

Fisher’s LSD (very liberal) -> Modified Bonferroni-> Bonferroni-> Tukey’s HSD (very conservative)

The probability of at least one Type I error (a) for a set of many comparisons. aFW = (c)(a) with c= # of comparisons

Conducting a large number of statistical tests, without theoretical reasons, and then “fishing” out the significant test results.

• 
• Anything greater than Fisher's LSD is significant.

• T= Grand Total
• t= # of grand totals (always 1)
• A= Total for each condition
• a= # of levels for the IV
• AS= Individual scores
• s= # of participants per condition
• as= Total # of participants
• [T]=T²/as
• [A]=(A₁²+A₂²)/s
• [AS]=AS²

• Planned before the study is executed to test a specific hypothesis.
• Only difference in how they are conducted is the alpha level used to determine significance.

• There must be 1 coefficient for each mean
•     – Means not included in the comparison have a coefficient of      zero

• There is always two sets of means being compared
•      – means in the same set have the same sign (+ or -)

Coefficients must add to zero

Orthogonal comparisons reflect independent or non-overlapping pieces of information.

 The outcome of one comparison gives no indication whatsoever about the outcome of another orthogonal comparison.

Non-orthogonal contrasts can be hard to interpret.

• Control of individual differences (aka subject differences)
• More likely to find an effect
• Fewer participants needed
• Efficiency gains (less time-consuming and expensive)… 

• Sequence effects
• 2) Carryover effects
• a) Order
• b) Practice
• c) Interference
• d) Differential

• Confounds are extraneous variables that vary in some non-random way across conditions
• 1. Sequence
• :– People change as they spend time working on tasks
• – They can become bored, fatigued, or more excited.
• 2. Carryover:
• – Responses to one condition influence the responses to a later condition
• .– Different from sequence effects. Not the passage of time, but the impact of one condition on another.
• 2) Carryover effects
• a) Order
• b) Practice
• c) Interference
• d) Differential