.
Based on lecture given by Dr. Thiwanka Wijeratne.
Another resource is "Basic Clinical Epidemiology"
Approach to Critical Appraisal
- PICOT Statement
- Randomization Worked? (Look at Table 1)
- Blinding? (who? what?)
- Outcome (reliable etc)
- Type of Analysis (intention to treat? per-protocol?)
- Generalizable? (Sampling, Eligibility)
Approach to Analysing Studies
- PICOT --> Identify what the study is studying
- Internal Validity
- Biases
- Selection Bias - A systematic error in creating intervention groups (causing them to differ with respect to prognosis)
- Measurement Bias - Any aspect of the way information is collected in the study that creates a systematic difference between compared population that is not the studied association (i.e. recall bias, lead time bias, etc..)
- Confounder -> A Type of bias -> (i.e. coffee is a confounder when studying smoking and MI, b/c comes with smoking and can influence MI)
- 3 Features:
- Predictor of Outcome
- Associated with Exposure
- Not intermediatry between exposure & outcome
- Options to address confounders:
- Adjustment Analysis
- Regression Techniques
- Stratification
- 3 Features:
- Biases
- External Validity
- Population of study and how it applies to real world.
- Sample Size (the larger the size, the more relevant patients)
- Alpha Error - (aka Random Error) - Falsely seeing a difference due to random chance (no difference exists)
- Example: Table 1 (baseline characteristics in randomization). Usually (1 in 20 times) will find a difference, so if more than 20 variables in Table 1, there is a chance one will be significant.
- Can use Bon Feroni correction.
- DO NOT use P-values, instead use "Standardized Mean Difference" to see if two groups are homogeneous. (b/c if you have enough patients and variables you will find a difference where one does not exist).
- Example: Table 1 (baseline characteristics in randomization). Usually (1 in 20 times) will find a difference, so if more than 20 variables in Table 1, there is a chance one will be significant.
- Beta Error - (aka Underpowered) - Not capturing a true difference due to insufficient sample size and/or study power.
- Structure
- RCT
- Randomization (
- Allocation Concealment (whether investigators and participants know their treatment).
- I.e. Randomization: allocate each person to A or B (random), but did not conceal treatment
- Allocation Concealment: Give each participant a number, which corresponds to a treatment (unknown).
- Blinding
- Outcome Measurement
- Observational Studies
- Case Control - start with outcome and look at exposure
- Start with one exposure, look at multiple outcomes.
- Cohort - start with exposure, and look at outcome
- Start with one outcome, and look at multiple exposures (i.e. MI)
- Retrospective vs. Prospective
- Accessibilty of information
- Retrospective --> already been done (go back to 2010, get patients, and look prospectively).
- Prospective --> don't have data yet, start with patients, and follow them to get data.
- Case Control - start with outcome and look at exposure
- RCT
Types of Studies
- Randomized Control Trials (RCT)
- Proves causation
- Pros:
- Only study design that establishes causation
- Cons:
- Restricted population, VERY expensive, very focused question
- Observational Studies
- Proves association
- Pros: Can study adverse effects, rare diseases
- Cons: Bias!
- Two Types:
- Case Control
- Cohort Studies
Study Characteristics
- Types of outcomes:
- Continuous
- Binary
- Time to Event
- Are outcomes clinically important? (valid, reliable (reproducible), interpretable, accurate, responsive)
- Type of analysis:
- Intention To Treat (participants stay in groups to which they were allocated)
- Per Protocol (able to change)
- Type of randomization
- Sequence-based is poor form (i.e. assign each one sequentially. Can bias)
- Must have "Allocation Concealment!" (aka blinding)
- Must define who is blinded --> patients, treating physicians, analysts, paper writers
- Do not use terms like "double blinded", "triple blinded" etc..
Diagnostic Tests
- Sensitivity
- Specificity
- PPV
- NPV
- Likelihood ratios
- Ratio of:
- Probability of positive test in pts with disease
- Probability of positive test in pts w/o disease.
- LR+
- sn / (1- sp)
- Probability of positive test of pts with disease (a/a+c) OVER probability of positive test in pts without disease (b/b+d) ---> simplifes to sn / (1-sp)
- LR -
- Probability of negative test in pts with disease (c/c+a) OVER probability of a negative test in pts w/o disease (d/b+d)
- (1-sn)/sp
- Ratio of:
- ODDS RATIOS
- Utility: find post-test probability from pre-test
- Two ways:
- Calculate:
- Probability --> to pre-test Odds --> (multiply by LR+ or LR- ) -->to post-test Odds --> post-test probability
- Opre = Ppre/(1-Ppre)
- Ppost=Opost(1+Opost)
- Use Nomogram
- "2-5-10" rule
- LR = 2 then probability should increase by ~ 15%
- LR = 5 --> 30%
- LR = 10 --> 45%
- etc...
- Calculate:
Treatment Effect
-
Outcome + Outcome - Exp A B Contr C D - Experimental Event Rate (EER) --> A/ A+B
- Control Event Rate (CER) --> C/ C+B
- Relative Risk (Risk Ratio)
- Probability of given event in exposed group vs. control.
- EER / CER
- Relative Risk Reduction
- RRR = (CER - EER / CER)
- Absolute Risk Reduction
- ARR = CER - EER
- Number Needed to Treat
- NNT = 1/ARR
- NOTE:
- Case control --> cannot calculate absolute risk.
- - can only look at Odds Ratio
- Case control --> cannot calculate absolute risk.
- Odds Ratio
- OR = (A/B) / (C/D)
- Odds of target outcome within treatment group vs controls (i.e. odds within treated group vs odds in untreated group).
Systemic Reviews & Meta-Analysis
- Systematic Reviews
- A systematic review is a scientific tool that can beused to appraise, summarise, and communicate theresults and implications of otherwise unmanageablequantities of research.Meta-Analysis
-
- Extension of systematic review --> actually use pooled numbers to make assessment across many studies.
- Process:
- Research Question
- Search for studies
- Review studies for inclusion
- Assess quality of sutides
- Summarise
- Qualiatively - Tables, Text
- Quantatively
- Forrest Plots
- Puts studies together.
- Look at model used to combine information:
- Fixed Model --> If all studies looked at same patients
- Random Effects Model - If heterogeneous population
- See if all studies have are heterogeneous
- Clinical Heterogeneity --> look clinically and decide if studies are the same (i.e. gastroparesis --> diabetics & crohn's disease)
- Statistical Heterogeneity --> Chi^2
- Q-statistic --> P value statistically significant?)
- I^2 > 40% --> then statistically heterogeneous
- If heterogeneous populations
- Measure it
- Random effects model (takes into account differences between studies)
- Subgroup analysis
- Meta-Regression
- Funnel Plot
- Looks at publication bias
- Should look like a tree (should be triangle) --> but if studies are biased, then will see only one side of the triangle. (i.e. only positive results are reported). goes to a point, so the more N in a trial, then narrows to a point.
Power Analysis
- Important to select an appropriate sample size to ensure the study has enough statistical power to avoid:
- A.) Type I error - incorrect rejection of null hypothesis ("false positive")
- B.) Type II error - incorrectly retaining a false null hypothesis ("false negative")
- Variables:
- N = sample size
- alpha = Type I error chance
- beta = Type II error chance
- 1-beta = Power
- To calculate any above variable, you need the others..
- Two ways to do power analysis
- A priori analysis
- - Most ideal. Done before the trial, estimating the needed alpha, and 1-beta
- Post-hoc analysis
- - Less ideal. Once you have N, and alpha, it calculates 1-Beta
- Compromise power analysis
- - When N is too large to be feasible. Then have to compromise on the alpha or beta.
- A priori analysis
Useful Resources
- CONSORT STATEMENT
- Checklist for reporting RCTs
- STROBE STATEMENT
- Checklist for reporting observational studies
- PRISMA STATEMENT
- Preferred reporting items for systematic reviews and meta-analysis
- MOOSE STATEMENT
- For systematic reviews of observational studies
- DOWNLOAD
- RevMan5
- Mendeley: free reference manager
Comments