Skip to content

“Tips for Savvy Medical Web Surfing” – A Critique

February 21, 2008

CNN has an article today, Tips for Savvy Medical Web Surfing. To their credit, the spoke with Mary Ryan, President-Elect of the Medical Library Association. After all, medical librarians are professional experts in finding health information and assessing its quality. However, it would have been nice if they had listed the organization’s correct name, rather than erroneously calling it the “American Medical Library Association.”

This is kind of a long and wonky one, but I think it’s important to address.

A breakdown of the advice, and my response:
“Use search engines that screen out the garbage for you” – They suggest MedlinePlus, which is an excellent idea. It is a project of the National Library of Medicine, and leads you to generally reliable consumer-level information on numerous medical concerns.

Find smart bloggers with your disease – Yes, smart bloggers are probably linking to associations and information relevant to your condition. Not bad advice. Don’t take bloggers’ comments as your only source of info, though.

Invest 30 minutes in the pubmed.gov tutorial
I do have a problem with the advice given by Jan Guthrie, director of The Health Resource, a for-pay medical research service. Honestly, I probably wouldn’t have written this post if this bit hadn’t irritated me. In this section about searching for articles in the major medical literature database, PubMed, Guthrie advises reading just the beginning and end of the study, stating that “The conclusion will tell you whether the treatment they studied was effective, moderately effective, or not at all effective.”

This is simply not true as an absolute, and it is the exact opposite of how many expert medical librarians are trained to read papers. The abstract, introduction, conclusion and discussion sections of a paper most reflect the way the authors want to present their data, and may exaggerate findings or make statements that are not supported by the methodology and data. These are the very sections in which authors attempt to make their case for why their work matters – they tell you what the authors think their findings mean, but don’t actually prove it definitively. If you only want to know what an article is generally about, so you can bring it to your physician, fine, just read the conclusion. If you actually want to know if the treatment was effective, you have to read the entire paper, assess the methods the researchers used, determine whether it is applicable to the individual situation, understand whether the paper has important limitations, and so on. It’s simply not true that every paper making it into a peer-reviewed journal (even prominent ones) is well-designed and well-founded in its conclusions, and reading only the conclusions and believing them without further investigation is bad practice. Maybe it’s a fine distinction to some, what the authors say they did vs. finding out what they actually did, but I think it’s an important one.

Click on information about annual meetings – Yes, annual meetings of professional medical organizations often include presentations of new findings and can be good for keeping up with what is novel in medicine. The problem is generally that the findings have not yet been subjected to peer and editorial review, so it is difficult to assess the validity of the findings from these preliminary findings. I like to see the complete picture before deciding a researcher really knows what’s what, but it may be a good jumping off point for discussion with your doc.

When in doubt about a Web site, click on “about us” – You should do this anyway, whether you have initial doubts or not. It’s good to know where your advice is coming from, and whether there are conflicts of interest. For example, are the proprietors invested in selling particular supplements or procedures, or are they associated with some money-making or agenda-driven organization that may color what information they present? Find out before you go too deep down the rabbit hole.

Update: David notes a problem with the “search engine” section of the CNN piece.

About these ads
9 Comments leave one →
  1. February 24, 2008 4:14 am

    This is valuable information for those who like to surf the web for information on medical problems of interest.

  2. February 25, 2008 2:25 pm

    Very good points.

    My only concern is that the typical lay reader is not going to be able to generate a discriminating opinion from the “Materials and Methods” and “Data Analysis” sections of a technical research paper. Even acknowledging the problems of conflicts of interest, overstating of results, and elitist barriers to information, I suspect as an empirical matter that the impression even a reasonably knowledgeable lay reader will take away from the abstract and conclusion of a paper will leave them closer to a factually-informed understanding of the results than would reading the methods and data sections and trying to form their own conclusions.

    Technically-educated readers, or those who have highly self-educated themselves about a particular disease, the extant literature on it, and statistical analysis methods, will want to take a more critical stance. But it requires considerable expertise to be able to accurately identify and knowledgeably challenge technical shortcomings in a research paper. The average person looking for information, who mostly reads secondary sources, will likely be better off accepting the researchers’ word for their own conclusions than trying to critique them. This does ratify the elites’ stature as knowledge-keepers, and that is problematic, but, honestly, there is a difference between the elite and the lay masses.

  3. February 25, 2008 2:46 pm

    Kevin, I actually agree with you that the lay reader is going to have a hard time deciphering those elements of many papers, and that the abstract and conclusions are likely to be more readily understandable. I simply object to the author’s characterization that the conclusion “will tell you whether the treatment they studied was effective, moderately effective, or not at all effective.” It will tell you whether the researchers think as much, but I thought the CNN author glossed over the notion that these sections of papers are most prone to be misleading/incomplete, if that was understood at all. I absolutely agree that, as a jumping off point for discussion with a healthcare provider, reading the conclusion is just fine. I just wanted to nitpick the significance the author gave to these sections with regard to “demonstrating” anything about efficacy.

    Thank you for your comment!

  4. cleverpostingaddy permalink
    November 13, 2009 7:51 pm

    Look for information based on evidence. If it’s about a drug, check out older drugs of the same class, or other drugs used to treat that illness or condition, which have been on the market for over five years, at minimum. Check out the Cochrane Collaboration Library, google English (or French, if you can) Prescrire, read the Australian equivalent of the FDA, which is more trustworthy and less conflicted, run the study authors and ‘thought leaders’ (the guy quoted) names through CSPINET integrity database, google the authors universities, clinics and labs for industry conflict of interest.

    Take a look at the website mandate. Are they hitting it? For example if they puport to be advocating for women, but instead advoacate for a particular political party, don’t buy what they’re ‘selling’.

Trackbacks

  1. davidrothman.net » Blog Archive » Nitpicks: CNN’s “Tips for savvy medical Web surfing”
  2. Yes, Virginia, Liberals Do Live in Tennessee « The Crone Speaks
  3. Everybody Dance Now … « Newscoma
  4. Finding Credible Health Information Online: MedLibs Round 1.8 | Highlight HEALTH
  5. Searching for Credible Health Information Online?: Ask Rachel | Our Bodies Our Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 2,102 other followers

%d bloggers like this: