Research methods weeks 5-8

• 33%
• 21%

ELITE1 showed losartan had 50% less risk of mortality, but mortality was a secondary endpoint. 

In ELITE2, the primary endpoint was mortality, and it showed that mortality was unchanged with losartan. 

Thus don't use secondary endpoints to make judgements.

• analyzed 115 articles published from 1990-2003 in major journals and speciality journals that had over 1000 citations.
• -49 reported evaluations of health care interventions
• 45 claimed the interventions were effective
• by 2004, 5/6 non-randomized studies, and 9/39 randomized trials were contraindicated or exaggerated

80%

• spironolactone vs placebo
• very stringent inclusion/exclusion criteria
• Spironolactone was protective in the study

In real life, the patients had a higher risk of hyperkalemia

• Why? 
• Pts not monitored ascarefully, lower compliance, interaction with other drugs (Ace-inhibitors)
• 24% got hyperkalemia, compared to 1.7%

spironolactone is protective in heart failure, but only in a select few patients who meet inclusion criteria.

• Confounding [non randomized, lack of detailed clinical info]
• Availability and use of drugs [ cox-2 have a restricted status, non-selective are available OTC]
• Duration of use was short-term
• Generalizability of patient population

A set of observations on the values that a variable takes at diff. times

Goal: to develop a mathematical model that describes pattern to allow future prediction.

Take the trend, and subtract the cyclical components, the remaining portion are the random fluctuations.

error term associated with any observation isrelated to error term of other observations.

• ex. Durbin-Watson statistic
• Ljung-box statistic

mean and variance of stochastic processes are consistent overtime, and the covariance depends only on distance or lag between the 2 time periods. 

ex. Augmented Dick-Fuller test

Excludes patients with history of disease, just recent diagnosis/onset

• Time sequence; exposure precedes outcome
• collect info on all relevant predictions/confounders
• minimizes recall bias
• estimates incidence
• can assess multiple outcomes

• Large # of people needed
• loss to follow up
• expensive, and resourceintensive
• status of subject may change (switch between control and exposure)
• may miss subclinical forms of outcome
• inefficient for studying rare outcomes
• Prospective: time consuming
• Retrospective: limited control over sampling and choice/quality of predictor variables

• Population: diverse vs. highly selected
• Intervention: patient/provider vs randomized
• Follow-up: longer followup vs short follow-up
• Analysis: sophisticated multivariate techniques for confounding adjustments vs. simple-to-sophisticated

Exposure is a cause of an outcome if exposure at a given level results in a different out come than would have occurred without that level of exposure

• Guide to predict, prevent, diagnose , treat
• Treatable or reversible cause
• Clinical and research

• Temporality
• strength of association
• dose-response
• reversibility
• consistency
• biological plausability
• specficity: Causation is likely if a very specific population at a specific site and disease with no other likely explanation.
• analogy

• Ensure entire group not experienced outcome
• observed time period needs to be meaningful in disease context
• complete follow up

• chance (random error)
• bias (in selection or measurement)-> systematic error
• confound (another type of bias)
• effect modifier (interactions)

• systematic error in creating intervention groups, causing them to differ with respect to prognosis
• groups differ in baseline characteristics due to ways in which participants were selected for study or assigned to study groups
• -occurs at design phase, may impact external or internal validity

• randomization
• restriction
• matching
• stratification
• adjustment/standardization
• sensitivity analysis

Study sample taken from population of interest should be representative of the population

• -inclusion/exclusion criteria
• -volunteer bias
• -referral centre bias

• one group is more susceptible to outcome than another (aka susceptibility bias)
• due to differences in extent of disease, presence of co-morbidity, prior treatments.

• patients drop out or move from one group to another 
• loss-to-follow up
• cross-over

Can occur at design, conduct, or analysis phase

• observation
• classification
• information
• detection
• ascertainment : differential way data is collected
• recall
• screening bias (aka lead-time bias)

• blinding
• exposure and outcome data should be collected similarly
• standard outcome definition
• choose an objective/hard outcome

Estimated intervention effect is biased because of some difference between comparison groups apart from the planned intervention, such as baseline characteristics, prognostic factors, or concomitant interventions

• Actively exclude or control for confounding variables
• cohort studies: matching
• RCT: stratification
• controlling for confounding by measuring known confounders and including them as covariates in multivariate analysis.

Strength and direction

• strength: robustness of findings when bias is controlled
• direction: bias less important if association is significant despite bias in opposite direction

• real, spurious, indirect
• spurious: due to selection bias, measurement bias or chance
• indirect: due to confounding

• exclude those likely to be lost (moving, unwilling)
• obtain info to allow future tracking
• during follow-up: periodic contact with subjects
• checking vital stats from OHIP/registries

• Similar to RR but time-to-event analysis, has an associated time value. 
• Usually shown as a survival rate

"Twice as many people developed HT in 6 months"

• STROBE statements
• NOS

• Strengthening the Reporting of Observational Studies in Epid
• -international collaborative initiatives of epid, methodologists, statisticians involved in conduct and disseminiation of observational studies

aim is to assess the quality of non-randomize studies with its design, content, and ease-of-use

• uses a star system which judges a study in 3 perspectives:
• selection of study groups
• comparability of groups
• ascertainment of either exposure or outcome of interest, for case-control or cohort studies, respectively.

to developan instrument providing an easy and convenient tool for quality assessment of non-RCT to be used in systematic review

linked smoking to health problems using case-control study

against bias

• Case-control nested within a cohort study
• most data are collected before outcome occurrence, so less likely to be impacted by recall bias
• ex. peanut allergy study by AVON longitudinal study group

• study a risk factor/multiple risk factors
• disease of interest is rare (impractical for cohort)
• long latency period between exposure and disease
• random assignment is unethical or impossible
• need an answer quickly (bypassses need to collect info on large # of people who wont get the outcome)

exposures; diseases

Case-control

When the baseline probabilities of the outcome are low (<0.1-0.2)

• Thepopulation that the investigator wishes to target, with the cases being the subjects who develop the disease within the base. The population is defined geographically and temporally
• adv: easier to sample for controls from a primary base
• disadv: it is challenging/impractical to ascertain all cases in a primary base

determination of the study base

• choose all incident cases in the source population
• -recall of past exposures may be more accurate
• -temporal sequence is easier to assess.

• biases towards longer survival among participants 
• differential recall of risks
• exposures status may change with onset of each disease

Direct random sampling from source population

• Population-base
• hospital/medical practice controls
• neighbourhood/friends controls
• relatives
• proxy respondents
• deceased controls

in cases where the probability of disease diagnosis depends on access to medical care, then it would be appropriate to include hospital controls

• Random sample of population from which cases came
• usually defined by geographic borders
• valid if cases consist of all individuals who developed disease of interest in a defined population

• usually patients seeking care for other conditions (controls should have a variety of conditions)
• disadv: not always possible to identify source population of the cases
• -additional biases of specialized medical facilities

• source population of cases poorly defined and presumably healthy controls are desirable 
• neighbours tend to seek care at similar places
• neighbours are similar with respect to socioeconomic status and determinants of health

measurement of exposure should be comparable in cases and controls

• confounding should not be allowed to distort estimation of effect
• restriction, matching, adjustment

• Limits #of eligible subjects (ex.only males aged 40-50)
• helps control selection bias
• susceptible to residual confounding effect (risk still varies within age 40-50)
• limits generalizability and does not allow evaluation of the restricted factors

• Ensure that grous do not differ for confounders (ex. for every active male between 40-50, there will be an inactive male aged 40-50)
• helps control selection bias
• requires control of confounding at both design and analysis of study. 
• advantages: allows matching geographically to control socioeconomic/ethnic factors
• diadvantages: overmatching: match on factors that may themselves be related to exposures.

• Occurs when a survey respondent answers a question
• influenced by correct answer, but also respondent's memory
• ex. response bias: socially acceptable response

• Type of information bias orginated when sensitivity and specificity of procedure to detect exposure and/or effect is not perfect.
• i.e. exposed/diseased classified as non-exposed/non-diseased or vice-versa

very little improvement past case:control of 4:1

remember that increasing sample size only improves precision, not validity.

• Good for studying rare condition/disease
• Good for long latency between predictor and outcome
• Useful for hypothesis generation since relatively inexpensive and fast as disease already developed
• can evaluate multiple etiological factors/exposures at once

• cannot estimate incidence/prevalence
• can only study one outcome, as that is how popn is sampled
• prone to various errors and biases

• detailed and quantified description of a population
• systematic collection of data
• The research question does

Measure characteristics of a particular population, either at a fixed point in time, or comparatively over time

attemptto test a theory; i.e. explore and test associations between variables.

• subject of study, gains or losses produced by impact of research study
• 'cause' of changes in the dependent variable that will be manipulated or observed, then measured in dependent variable
• includes error variables that may confound results of the study (ideally you want these variables to be randomly distributed)

• hold them constant
• exclusion: only use females to diminish effects of gender

Research questions-> decide on information needed -> decide on prelim. analysis [examine resources, review existing lit. -> decide on sample+ chose survey method -> Design questionnaire -> Pilot survey -> amend questionnaire and sample -> main survey -> edit code and tabulate -> analyze -> final report

SA: postal, online, delivery and collection

I: structured interview, telephone questionnaire, focus group

• sample widely distributed geographically
• subjects need to be given time to think
• subjects have moderate/high interest in subject
• questions are written close-ended

• Delivered by hand to each respondent and collected later
• -allows direct contact with potential respondents, which may lead to higher response

Adv: low cost, easy to design and administer, anonymous-> honesty

• Disadv: volunteer sample (little control over who responds)
• sampling error (non-internet users)

adv: higher response rates, can ask open ended questions for detailed responses, additional probes can be asked

disadv: time consuming, expensive, sensitive topics-> less accurate

• allow for a variety of views to emerge
• group dynamics can often allow for stimulation of new perspective (allows for the basis of a survey)

• most people have a phone
• higher response rates

questions need to be short

• each member of population has known non-zero probability of being selected
• allows calculation of sampling error
• ex. random, systematic stratified

• members selected in non-random way
• sampling error unknown
• ex. convenience, judgement, quota, snowball

• Random: each member has equal and known chance of selection
• Systematic: also called 9th name selection [as long as no hidden order in list of eligibile participants, just as good as random]

• reduces sampling error even compared to random sampling
• identify relevant strata, then random sampling to select subjects from each stratum until # of participants is proportional to its frequency in the population

subset of population that shares atleast one common characteristic

• convenience: mostly during preliminary phases (ex. friends and family)
• judgement: an extension of convenience; researcher makes a judgement that this one sample is representative of the entire population (ex. use one city to sample the country)
• quota: non-probability equivalent of stratified sampling: identify your strata and proportions as represented in population, then use judgement sampling to select required number from each stratum
• snowball: used when desired sample characteristics are rare; relies on referrals from initial participiants to find new participants

• contact members of sampling frame and ascertain whether they belong to the required sample
• design questionnaire/interview in such a way that ineligible respondents are identified early and screened out

• If there is no easily applied "objective" measurement tool
• ex. determining a patient's status, assessing quality of life, enhancing educational programs

• Objective: can be verified, real answer exists
• Subjective: ask about personal perception, no factual answer. 
• Open-ended: allows information gathering, tough to analyze and time consuming
• Close-ended: useful for hypothesis testing, but may lose important info. time effecient, lots of response formats

• dichotomous
• multichotomous
• verbal frequency scale (all the time, fairly often, never etc. )
• list
• ranking
• likert scale
• graphical rating (continuous scale on a line)
• non-verbal

• Measures strength of feeling or perception
• optimal number is 7+/- 2 
• Often analyzed as interval data, but is actually nominal

• Double negatives
• Double Barelled: ask for thoughts about 2 things at once
• Leading: suggests the answer the research is looking for (may end in weren't you, don't you, etc. )
• Loaded: contains a controversial assumption

• Pre-test on convenient other
• Pilot on small sample from target popn.

• reliable 
• valid
• feasible
• acceptable

• produces error variance
• places upper limit on validity (requires reliability for validity,but not the opposite)

• raw score= true score + error 
• sources of error: misinterpretation, biases, inexperience, inter-rater differences

Tests constructed to differentiate between objects

•            a^2true
• [(a^2true+a^2error)
•      n

• reliability is not a fixed property of the scale
• change popn-> change reliability
• more items/raters increase reliability
• need trepeat measures to estimate error
• test that doesn't discriminate is useless!

>0.8: based on one time administration of survey, is there good correlation between related domains? - does not account for day-to-day or observer variation into account so usually an optimistic reliability

• Stability >0.5: 
• reproducibility when administered on different occasions:
• ex. test-retest: same result on 2 occasions
• intra-rater: agreement between ratings made by same rater on diff. occasions
• inter-rater: agreement between 2 different raters

• Wrong conclusions, unethical
• Face
• Content: does it tap relevant + disregard irrelevant ideas (based on expert opnion, existing tools etc. )
• Criterion: are results consistent with other measures [divided into concurrent and predictive]
• Construct: can we predict differences based on constructs? requires first 3 types of validity

• Concurrent: examine relationship between criterion measure and scale at time of administration
• predictive: examines relationship between scale and future outcomes (ex. compared to a gold standard)

• extreme groups: t-tests
• change: 2 way anova
• criterion: pearson's correlation

• Avoid ambiguity
• Consider reading level
• How much training required?
• How easy to score (avoid weighing responses, be aware of unintentional weighting)
• Cost

• be brief (but remember reliability proportional to # of items)
• use only items for which there is a variety of responses
• be aware of social desirability bias
• those who administer the scale willing to do so?

• Avoid intrusive questions
• emphasize confidentiality

• identify most appropriate respondents
• use multiple forms of contact
• develop easy to complete questionnaire with instructions
• conduct on-site interiews to taylor question to participant cognitive ability
• be cautious about financial incentives

• understand question
• recall relevant attitude, belief, behaviour
• inference and estimation (decomposition and extrapolation; end-digit bias)
• Map answer on response alternatives
• edit the answer (what people think and what they tell you)

• Social desirability
• Deviation
• Hello-goodbye
• End aversion: no extreme responses
• Positive skew: favourable response
• Halo effect
• framing: choice between 2 alternatives depends on how they're framed

• keep task simple
• maintain motivation of respondents (choose only those interested, motivation higher earlier so keep it short, ask people to explain their answers)