Here are links to some excellent reviews from the “How to read a paper” series by Trisha Greenhalgh, senior lecturer for the British Medical Journal. Also please scroll down to view some tips I have been able to find online from other websites to help you evaluate scientific studies in a more informed and appropriate manner.

How to read a paper: The Medline database
Trisha Greenhalgh
BMJ 315 (7101), 180-3 (19 Jul 1997)

How to read a paper : getting your bearings (deciding what the paper is about)
Trisha Greenhalgh
BMJ 315 (7102), 243-6 (26 Jul 1997)

How to read a paper: Assessing the methodological quality of published papers
Trisha Greenhalgh
BMJ 315 (7103), 305-8 (02 Aug 1997)

How to read a paper: Statistics for the non-statistician. I: Different types of data need different
statistical tests
Trisha Greenhalgh
BMJ 315 (7104), 364-6 (09 Aug 1997)

How to read a paper: Statistics for the non-statistician. II: "Significant" relations and their pitfalls
Trisha Greenhalgh
BMJ 315 (7105), 422-5 (16 Aug 1997)

How to read a paper: Papers that report drug trials
Trisha Greenhalgh
BMJ 315 (7106), 480-3 (23 Aug 1997)

How to read a paper: Papers that report diagnostic or screening tests
Trisha Greenhalgh
BMJ 315 (7107), 540-3 (30 Aug 1997)

How to read a paper: Papers that tell you what things cost (economic analyses)
Trisha Greenhalgh
BMJ 315 (7108), 596-9 (06 Sep 1997)

How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses)
Trisha Greenhalgh
BMJ 315 (7109), 672-5 (13 Sep 1997)

How to read a paper: Papers that go beyond numbers (qualitative research)
Trisha Greenhalgh and Rod Taylor
BMJ 315 (7110), 740-3 (20 Sep 1997)
------------------------------------------------------------------------------------

KEY TIPS IN EVALUATING A SCIENTIFIC STUDY (excerpt from an online resource)

A mnemonic: People say fleas on every dog really seem unhealthy

1. Examine population..
2. Look at size of study.
3. Find source of funds if possible.
4. How are results obtained?
5. Are endpoints or actual diseases being studied?
6. How is prevention/improvement defined?
7. Look at raw numbers and beware if none are supplied.
8. Does the study support the conclusion?
9. Is the conclusion being used appropriately in subsequent claims.

1. Examine the population of the study from two different angles.
a) What is the relationship of the study population to the general population as a whole? Do the study participants represent a large cross-section of the general population or is the study confined to a very small isolated group? Have they been chosen at random? If not, who has been excluded or included?

b) Are controls and subjects evenly matched on all relevant variables? If not does the study point out the variables which may have affected the results but which they could not control.

2. Look at the size of the study population.
The larger the population the better. How long a time period does the study cover? Again, the longer the time the better.

3. Who is financing/supporting the study?
Who is supplying the medication if medication is being studied? This is difficult to establish since drug companies try to launder their money to hide their involvement. Assume that the drug company providing the medication has a vested interest in the results and read them with that caveat in mind.

4. How are the results being obtained?
a) Self-reporting or relying on memory of the participant is the least reliable method. Retrospective studies that rely on old medical records are marginally more reliable but they assume that the old medical record is accurate – a somewhat questionable assumption. Current ongoing objective testing of the participants is the most reliable method.

b) Is there a distinction made between in vivo (i.e. results in living human beings) and in vitro (i.e. results in test tubes and petri dishes). Very frequently results in test tubes and petri dishes do not translate into results in living human beings. Similarly results on animals are a step removed from results on humans and frequently will not translate.

5. What is being studied?
Surrogate end points (factors associated with the disease) or the actual disease? If endpoints rather than disease, then are there studies showing that the end points are actually connected to the disease? Those studies must also be evaluated.

6. How is improvement or prevention defined?
What is the practical significance of the improvement as the researcher defines it. Remember that statistical significance may not mean clinical significance.

7. Examine the raw numbers.
Beware of abstracts and studies reporting the results as percentages without indicating the raw numbers. Remember that 100% of x is 0 if x is 0. A 50% reduction in something is insignificant if the study shows that you have prevented 1 death in 100,000 people.
A recently published study on fosamax highlights this problem of percentages rather than raw numbers. After three years, according to the conclusion, women taking fosamax had lost 35% less height than women not on fossamax. Sounds impressive until you read the fine print – 35% is equal to 1.6 millimeters. Over 3 years. in thirty years the difference (assuming that it’s linear) would be less than 3/4 of an inch. And this assumes that measurement was sufficiently accurate to conclude that 1.6mm was actual and not a measurement error.

8. Does the body of the study actually support the conclusion?
This is frequently where the raw numbers vs percentages becomes crucial.

9. Finally even if the study itself seems logical and all seems to be in order, is it being used appropriately. Or is is being used to justify a course of action or to prove a theory which does not logically follow.