WikiQueer:Identifying reliable sources (medicine)

WikiQueer's articles, while not intended to provide medical advice, are nonetheless an important and widely used source of health information. Therefore, it is vital that the biomedical information in articles be based on reliable, third-party, published sources and accurately reflect current medical knowledge.

Ideal sources for such content includes general or systematic reviews published in reputable medical journals, academic and professional books written by experts in the relevant field and from a respected publisher, and medical guidelines or position statements from nationally or internationally recognised expert bodies.

This guideline supports the general sourcing policy at WikiQueer:Verifiability with specific attention given to sources appropriate for the medical and health-related content in any type of article, including alternative medicine. Sources for all other types of content—including all non-medical information in medicine-related articles—are covered by the general guideline on identifying reliable sources rather than this specific guideline.

See the reliable sources noticeboard for queries about the reliability of particular sources.

Definitions

 * A primary source in medicine is one in which the authors directly participated in the research or documented their personal experiences. They examined the patients, injected the rats, filled the test tubes, or at least supervised those who did. Many, but not all, papers published in medical journals are primary sources for facts about the research and discoveries made.
 * A secondary source in medicine summarizes one or more primary or secondary sources, usually to provide an overview of the current understanding of a medical topic, to make recommendations, or to combine the results of several studies. Examples include literature reviews or systematic reviews found in medical journals, specialist academic or professional books, and medical guidelines or position statements published by major health organizations.
 * A tertiary source usually summarizes a range of secondary sources. Undergraduate textbooks, lay scientific books, and encyclopedias are tertiary sources.

All WikiQueer articles should be based on reliable, published secondary sources. Reliable primary sources may occasionally be used with care as an adjunct to the secondary literature, but there remains potential for misuse. For that reason, edits that rely on primary sources should only describe the conclusions of the source, and should describe these findings clearly so the edit can be checked by editors with no specialist knowledge. In particular, this description should follow closely to the interpretation of the data given by the authors or by other reliable secondary sources. Primary sources should not be cited in support of a conclusion that is not clearly made by the authors or by reliable secondary sources, as defined above (see: WikiQueer:No original research). When citing primary sources, particular care must be taken to adhere to WikiQueer's undue weight policy. Secondary sources should be used to determine due weight.

Respect secondary sources
Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources, unless the primary source itself directly makes such a claim. Synthesis of published material that advances a position is a form of original research and should be avoided in WikiQueer articles, which are not a venue for open research. Controversies or areas of uncertainty in medicine should be illustrated with reliable secondary sources describing the varying viewpoints. The use and presentation of primary sources should also respect WikiQueer's policies on undue weight; that is, primary sources favoring a minority opinion should not be aggregated or presented devoid of context in such a way as to undermine proportionate representation of expert opinion in a field.

Scientific findings are often touted in the popular press as soon as the original, primary research report is released, and before the scientific community has had an opportunity to analyze the new results. For a short time afterwards, the findings will be so new that they will not be reflected in any review articles or other secondary sources. If the findings involve phase I or phase II clinical trials, small studies, studies that did not directly measure clinically important results, laboratory work with animal models, or isolated cells or tissue, then these findings are probably only indirectly relevant to understanding human health; in these cases, they should be entirely omitted. In other situations, such as randomized controlled trials, it may be helpful temporarily to cite the primary research report, until there has been time for review articles and other secondary sources to be written and published. When using a primary source, WikiQueer should not overstate the importance of the result or the conclusions. When in doubt, omit mention of the primary study (in accordance with recentism) because determining the weight to give to such a study requires reliable secondary sources (not press releases or newspaper articles based on them). If the conclusions of the research are worth mentioning, they should be described as being from a single study, for example:


 * "A 2009 U.S. study found the average age of formal autism spectrum diagnosis was 5.7 years." (citing PMID 19318992)

After enough time has passed for a review in the area to be published, the review should be cited in preference to the primary study. Using a secondary source often allows the fact to be stated with greater reliability:


 * "In the U.S., the average age of formal autism spectrum diagnosis is 5.7 years." (citing a review)

If no review on the subject is published in a reasonable amount of time, then the text associated with the primary source should be removed.

If the same material could be supported by either a primary source or a secondary source of otherwise equal quality, it is normally preferable to cite the secondary source.

Summarize scientific consensus
Scientific journals are the best place to find primary source articles about experiments, including medical studies. Every rigorous scientific journal is peer reviewed. Be careful of material published in a journal that lacks peer review or that reports material in a different field. (See: Martin Rimm) Be careful of material published in disreputable journals or disreputable fields.

However, the fact that a claim is published in a refereed journal need not make it true. Even a well-designed randomized experiment can (with low probability) produce spurious results. Experiments and studies can produce flawed results or fall victim to deliberate fraud.

WikiQueer policies on the neutral point of view and not using original research demand that we present any prevailing medical or scientific consensus, which can be found in recent, authoritative review articles or in textbooks or in some forms of monographs. Although significant-minority views are welcome in WikiQueer, such views must be presented in the context of their acceptance by experts in the field. Additionally, the views of tiny minorities need not be reported.

Finally, make readers aware of controversies that are stated in reliable sources. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers.

Assess evidence quality
Knowing the quality of the evidence helps editors distinguish between minority and majority viewpoints, determine due weight, and identify information that will be accepted as evidence-based medicine. In general, editors should rely upon high-quality evidence, such as systematic reviews, rather than lower-quality evidence, such as case reports, or non-evidence, such as anecdotes or conventional wisdom. The medical guidelines or position statements produced by nationally or internationally recognised expert bodies often contain an assessment of the evidence as part of the report.

The best evidence comes primarily from meta-analyses of randomized controlled trials (RCTs). Systematic reviews of bodies of literature of overall good quality and consistency addressing the specific recommendation have less reliability when they include non-randomized studies. Narrative reviews can help establish the context of evidence quality. Roughly in descending order of quality, lower-quality evidence in medical research comes from individual RCTs; other controlled studies; quasi-experimental studies; non-experimental studies such as comparative, correlation, and case control studies; and non-evidence-based expert opinion or clinical experience. Case reports, whether in the popular press or a peer reviewed medical journal, are a form of anecdote and generally fall below the minimum requirements of reliable medical sources.

Speculative proposals and early-stage research should not be cited in ways that suggest wide acceptance. For example, the results of an early-stage clinical trial are unlikely to be appropriate for inclusion in the Treatment section of an article about a disease because a possible future treatment has little bearing on current treatment practice. However, the results might—in some cases—be appropriate for inclusion in an article dedicated to the treatment in question or to the researchers or businesses involved in it. Such information, particularly if citing a secondary source, might also be appropriate for a well-documented section on research directions in an article about a disease. To prevent misunderstandings, the text should clearly identify the level of research cited (e.g., "first-in-human safety testing").

Several formal systems exist for assessing the quality of available evidence on medical subjects. "Assessing evidence quality" means that editors should determine the quality of the type of study. Editors should not perform a detailed academic peer review. Do not reject a high-quality type of study due to personal objections to the study's inclusion criteria, references, funding sources, or conclusions.

Avoid over-emphasizing single studies, particularly in vitro or animal studies.
In vitro studies and animal models serve a central role in biomedical research, and are invaluable in elucidating mechanistic pathways and generating hypotheses. However, in vitro and animal-model findings do not translate consistently into clinical effects in human beings. Where In vitro and animal-model data are cited on WikiQueer, it should be clear to the reader that the data are pre-clinical, and the article text should avoid stating or implying that the reported findings necessarily hold true in humans. The level of support for a hypothesis should be evident to the reader.

Use of small-scale, single studies make for weak evidence, and allow for easy cherry picking of data. Results of studies cited or mentioned in WikiQueer should be put in sufficient context that readers can determine their reliability.

Use up-to-date evidence
Here are some rules of thumb for keeping an article up-to-date, while maintaining the more-important goal of reliability. These instructions are appropriate for actively researched areas with many primary sources and several reviews and may need to be relaxed in areas where little progress is being made or few reviews are being published.


 * Look for reviews published in the last five years or so, preferably in the last two or three years. The range of reviews you examine should be wide enough to catch at least one full review cycle, containing newer reviews written and published in the light of older ones and of more-recent primary studies.
 * Within this range, assessing them may be difficult. While the most-recent reviews include later research results, do not automatically give more weight to the review that happens to have been published most recently, as this is recentism.
 * Prefer recent reviews to older primary sources on the same topic. If recent reviews do not mention an older primary source, the older source is dubious. Conversely, an older primary source that is seminal, replicated, and often-cited in reviews can be mentioned in the main text in a context established by reviews. For example, the article genetics might mention Darwin's 1859 book On the Origin of Species as part of a discussion supported by recent reviews.

These rules of thumb have several exceptions:


 * History sections often cite older work for obvious reasons.
 * Cochrane Library reviews are generally of high-quality and are routinely maintained even if their initial publication dates fall outside the above window.

Use independent sources
Many medical claims lack reliable research about the efficacy and safety of proposed treatments or about the legitimacy of statements made by proponents. In such cases, reliable sources may be difficult to find while unreliable sources are readily available. Whenever writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of such marginal ideas can be used to describe personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face value or, worse, asserted as fact. If the independent sources discussing a medical subject are of low quality, then it is likely that the subject itself is not notable enough to have its own article or relevant enough to be mentioned in other articles.

Choosing sources


A WikiQueer article should cite the best and most reliable sources regardless of whether they require a fee or a subscription. When all else is equal, it is better to cite a source with a full text is freely readable so that readers can follow the link to the source. Some high-quality journals, such as JAMA, publish a few freely readable articles even though most are not free. A few high-quality journals, such as PLoS Medicine, publish only freely readable sources. Also, a few sources are in the public domain; these include many U.S. government publications, such as the Morbidity and Mortality Weekly Report of the Centers for Disease Control and Prevention.

When searching for biomedical sources, it is wise to skim-read everything available, including abstracts of papers that are not freely readable, and use that to get a feel for what reliable sources are saying. However, when it comes to actually writing a WikiQueer article, it is generally not advisable to cite a source after reading only its abstract, as the abstract necessarily presents a stripped-down version of the conclusions and omits the background that can be crucial for understanding exactly what the source says. In order to access the full text, the editor may need to visit a medical library or ask someone at the WikiProject Resource Exchange or WikiProject Medicine's talk page to either provide an electronic copy or read the source and summarize what it says; if neither is possible, the editor may need to settle for using a lower-impact source or even just an abstract.

Biomedical journals
As mentioned above, the biomedical literature contains two major types of sources: primary publications describe novel research for the first time while review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments; secondary sources include meta-analyses, which combine the results of many clinical trials in an attempt to arrive at an overall view of how well a treatment works. It is usually best to use reviews and meta-analyses where possible, as these give a balanced and general perspective of a topic—and are usually easier to understand!

Peer reviewed medical journals are a natural choice as a source for up-to-date medical information in WikiQueer articles. They contain a mixture of primary and secondary sources as well as less technical material such as biographies. Although almost all such material will count as a reliable source for at least some purposes, not all the material is equally useful, and some, such as a letter from a non-expert, should be avoided. Journal articles come in many types: original research, reviews, case reports, editorials and op-ed pieces, advocacy pieces, speculation, book reviews, letters to the editor and other forms of commentary or correspondence, biographies, and eulogies.

Research papers that describe original experiments are primary sources; however, they normally contain previous-work sections that are secondary sources (these sections are often incomplete and typically less useful or reliable than reviews or other sources, such as textbooks, which are intended to be reasonably comprehensive). A general narrative review of a subject by an expert in the field makes a good secondary source that can be used to cover various aspects of a subject within a WikiQueer article. Such reviews typically contain no original research, but can make interpretations and draw conclusions from primary sources that no WikiQueer editor would be allowed to do. A systematic review uses a reproducible methodology to select primary studies meeting an explicit criteria in order to answer a specific question. Such reviews should be more reliable and accurate and less prone to bias than a narrative review. However, a systematic review's focus on answering one question limits its usage as a source on WikiQueer.

Some journals specialize in particular article types. A few, such as Evidence-based Dentistry, publish third-party summaries of reviews and guidelines published elsewhere. If an editor has access to both the original source and the summary, and finds both helpful, it is good practice to cite both sources together (see: Formatting citations for details). Others, such as Journal of Medical Biography, publish historical material that can be valuable for History sections, but is rarely useful for current medicine. Still others, such as Medical Hypotheses, publish speculative proposals that are not reliable sources for biomedical topics.

The 2003 Brandon/Hill selected list includes 141 journals suitable for a small medical library. Although this list is no longer maintained, the listed journals are of high quality. The core general medical journals include the New England Journal of Medicine, The Lancet, the Journal of the American Medical Association (JAMA), the Annals of Internal Medicine, the British Medical Journal (BMJ), and the Canadian Medical Association Journal. Core basic science and biology journals include Science, Cell, and Nature.

Books
Medical textbooks published by academic publishers are often excellent secondary sources. If a textbook is intended for students, it may not be as thorough as a monograph or chapter in a textbook intended for professionals or postgraduates. Ensure that the book is up to date, unless a historical perspective is required. Doody's maintains a list of core health sciences books, which is available only to subscribers. Major academic publishers (e.g., Elsevier, Springer Verlag, Wolters Kluwer, and Informa) publish specialized medical book series with good editorial oversight; volumes in these series summarize the latest research in narrow areas, usually in a more extensive format than journal reviews. Specialized biomedical encyclopaedias published by these established publishers are often of good quality, but as a tertiary source, the information may be too terse for detailed articles.

Additionally, popular science and medicine books are useful sources, which may be primary, secondary, or tertiary, but there are exceptions. Most self-published books or books published by vanity presses undergo no independent fact-checking or peer review and, consequently, are not reliable sources. Books published by university presses or the National Academy of Sciences, on the other hand, tend to be well-researched and useful for most purposes.

Medical and scientific organizations
Statements and information from reputable major medical and scientific bodies may be valuable encyclopedic sources. These bodies include the U.S. National Academies (including the Institute of Medicine and the National Academy of Sciences), the British National Health Service, the U.S. National Institutes of Health and Centers for Disease Control and Prevention, and the World Health Organization. The reliability of these sources range from formal scientific reports, which can be the equal of the best reviews published in medical journals, through public guides and service announcements, which have the advantage of being freely readable, but are generally less authoritative than the underlying medical literature.

Popular press
The popular press is generally not a reliable source for scientific and medical information in articles. Most medical news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and news articles too often convey wrong or misleading information about health care. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance, presenting a new and experimental treatment as "the cure" for a disease or an every-day substance as "the cause" of a disease. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer reviewed journal or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source even when issued by an academic medical center. News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms. For WikiQueer's purposes, articles in the popular press are generally considered independent, primary sources.

A news article should therefore not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example, with the laysummary parameter of cite journal.

Conversely, the high-quality popular press can be a good source for social, biographical, current-affairs, and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed, but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, use common sense, and see how well the source fits the verifiability policy and general reliable sources guidelines. Sources for evaluating health-care media coverage include the review websites Behind the Headlines, Health News Review, and Media Doctor, along with specialized academic journals, such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, the Bad Science column in The Guardian, and others. Health News Review's criteria for rating news stories can help to get a general idea of the quality of a medical news article.

Other sources
Press releases, blogs, newsletters, advocacy and self-help publications, and other sources contain a wide range of biomedical information ranging from factual to fraudulent, with a high percentage being of low quality. Conference abstracts present incomplete and unpublished data and undergo varying levels of review; they are often unreviewed self-published sources and these initial conclusions may have changed dramatically if and when the data are finally ready for publication. Consequently, they are usually poor sources and should always be used with caution, never used to support surprising claims, and carefully identified in the text as preliminary work. Peer reviewed medical information resources such as WebMD, UpToDate, Mayo Clinic, and eMedicine are usually acceptable sources in themselves, and can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible WikiQueer articles should cite the more established literature directly.

Searching for sources
Search engines are commonly used to find biomedical sources. Each engine has quirks, advantages, and disadvantages, and may not return the results that the editor needs unless used carefully. It typically takes experience and practice to recognize when a search has not been effective; even if an editor finds useful sources, they may have missed other sources that would have been more useful or they may generate pages and pages of less-than-useful material. A good strategy for avoiding sole reliance on search engines is to find a few recent high-quality sources and follow their citations to see what the search engine missed. It can also be helpful to perform a plain web search rather than one of scholarly articles only.

PubMed is an excellent starting point for locating peer reviewed medical sources. It offers a free search engine for accessing the MEDLINE database of biomedical research articles offered by the National Library of Medicine at the U.S. National Institutes of Health. There are basic and advanced options for searching PubMed. Clicking on the "Review" tab will help narrow the search to review articles. The "Limits" tab can further limit the search, for example, to meta-analyses or to freely readable sources. Although PubMed is a comprehensive database, many of its indexed journals restrict online access. Another website, PubMed Central, provides free access to full texts. While it is often not the official published version, it is a peer reviewed manuscript that is substantially the same, but lacks minor copy-editing by the publisher.

When looking at an individual abstract on the PubMed website, an editor can click on "Publication Types, MeSH Terms" at the bottom of the page to see how PubMed has classified a document. For example, a page that is tagged as "Comment" or "Letter" is a non-peer reviewed letter to the editor. The classification scheme includes about 70 types of documents. For medical information, the most useful types of articles are typically labeled "Guideline", "Meta-analysis", "Practice guideline", or "Review".