Scientific community takes credibility hit

How good is your research??

This was on CNN.com today:

<!–QuoteBegin>

Quote

Research: Third of study results don’t hold up

Wednesday, July 13, 2005; Posted: 12:06 p.m. EDT (16:06 GMT)

CHICAGO, Illinois (AP) – New research highlights a frustrating fact about science: What was good for you yesterday frequently will turn out to be not so great tomorrow.

The sobering conclusion came in a review of major studies published in three influential medical journals between 1990 and 2003, including 45 highly publicized studies that initially claimed a drug or other treatment worked.

Subsequent research contradicted results of seven studies – 16 percent – and reported weaker results for seven others, an additional 16 percent.

That means nearly one-third of the original results did not hold up, according to the report in Wednesday’s Journal of the American Medical Association.

“Contradicted and potentially exaggerated findings are not uncommon in the most visible and most influential original clinical research,” said study author Dr. John Ioannidis, a researcher at the University of Ioannina in Greece.

Ioannidis examined research in the New England Journal of Medicine, JAMA and Lancet – prominent journals whose weekly studies help feed a growing public appetite for medical news.

Experts say the report is a reminder to doctors and patients that they should not put too much stock in a single study and understand that treatments often become obsolete with medical advances.

“The crazy part about science and yet the exciting part about science is you almost never have something that’s black and white,” said Dr. Catherine DeAngelis, JAMA’s editor-in-chief.

Editors at the New England Journal of Medicine added in a statement: "A single study is not the final word, and that is an important message."

The refuted studies dealt with a wide range of drugs and treatments. Hormone pills were once thought to protect menopausal women from heart disease but later were shown to do the opposite, and Vitamin E pills have not been shown to prevent heart attacks, contrary to initial results.

Contradictions also included a study that found nitric oxide does not improve survival in patients with respiratory failure, despite earlier claims. And a study suggested an antibody treatment did not improve survival in certain sepsis patients; a smaller previous study found the opposite.

Ioannidis acknowledged an important but not very reassuring caveat: “There’s no proof that the subsequent studies … were necessarily correct.” But he noted that in all 14 cases in which results were contradicted or softened, the subsequent studies were either larger or better designed. Also, none of the contradicted treatments is currently recommended by medical guidelines.

Not by accident, this week’s JAMA also includes a study contradicting previous thinking that stomach-lying helped improve breathing in children hospitalized with acute lung injuries. The new study found they did no better than patients lying on their backs.

DeAngelis said she included the study with Ioannidis’ report to highlight the issue. She said the media can complicate matters with misleading or exaggerated headlines about studies.

Ioannidis said scientists and editors should avoid “giving selective attention only to the most promising or exciting results” and should make the public more aware of the limitations of science.

“The general public should not panic” about refuted studies, he said. “We all need to start thinking more critically.”

Drugs, I was put on Zantac 75 in '94, at $90 a bottle. After they became over the conter medicine at $7 a bottle I was told they weren’t doing what the doctor thought in the first place, wanting to change me over to another $100 bottle of meds. I refused and went natural healing. Using Acidopholis now. Medical research is a farce in the US now. It’s all about the bucks.

One other thing, putting faith in someone’s research is kind of stupid in the first place if you strip it down for what it’s worth. How do you know the researcher didn’t cheat in school? Food for thought. :)

Yes, research driven by money is often bad research. This is not a blow to science, however, but to the way in which it has been institutionalized.

Additionally, while the hsitory of science has been a litany of errors, scientific claims are fallible, and at best they are no more than justified given the evidence of the time. Really, I am surprised that the number they found in error was only 1/3. This in no way discredits science, it is the way science works.

Quote (TomS @ July 14 2005,09:14)
This in no way discredits science, it is the way science works.

I guess descriptions like “frustrating,” “results don’t hold up” and “sobering conclusion” are compliments to science??

Since when did you trust a liberal biased news agency? Science is doing just fine.

Quote (ksdb @ July 14 2005,10:00)
I guess descriptions like “frustrating,” “results don’t hold up” and “sobering conclusion” are compliments to science??

A huge part of science is PEER REVIEW as well as REPRODUCABILITY. Here science worked exactly as it should have. Someone ran a study adn had conclusions, others ran similar studies and debunked the original study. I mean, that’s how it works. It’s called the scientific method. If your findings don’t hold up in other experiments, then they get tossed int the trash, the study is reviewed to see where the possibily of contamination was, and things move on. As TomS said, this isn’t a jab at science at all. Perhaps a jab at dopes prematurely making a big deal out of a study of 6 people in Iowa that didn’t get sick after drinking green tea for a week. There is such a thing as bad science and unfortuantely many times when profit is involved, sciece, like statistics and eveything else, gets massaged in the process to show a biased outcome.

Desciptions like frustrating, results dont’ hold up et al. are a compliment to science. Though they are quite humilating and degrading for the bozo researchers who did skewed studies.

The big prolem I see is people do not understand science and how to sort good science from bad science. Aspirin and its effects… good science. We can reproduce the results over and over and over. Blood indeed thins, headaches do indeed go away. It works here, in India, on the moon, everywhere. That’s pretty solid science.

Then we have bad science. Three people hung quartz crystals over their bed and didn’t get AIDS… err, to then say quartz crystals prevents/cures AIDS is not a valid conclusion. I see no elephants, but that doesn’t mean my belt buckle is an elephant repellant. Small sample sets, flawed methodologies, and improper logic to draw conclusions are all where folks fail in the scientific method. The method always works. (Or so it has thus far in the universe) The practitioners are the ones that screw it up. You’re blaming the fire on the alarm rather than the arsonist.

What do we know about the accuracy of studies that dispute the original findings?? Are one-third of those wrong too??

Quote (Mr Soul @ July 14 2005,12:06)
Since when did you trust a liberal biased news agency? Science is doing just fine.

I posted sources like this so you can’t dismiss them as … what do you call it … right-wing smear??? Obviously it’s a poorly written story because it extrapolates medical research in a way that seems to apply to the whole of science. But then, again, you guys are saying this is how science works, by being wrong a third of the time, so I guess I can trust your words.

Perhaps. Sorry man, science does not work in absolutes. You are confusing math and science. That is why there are scientific theories and mathmatical laws. I think you would hardly dispute the existence of gravity. In science it is a theory, math it is a law. Now to get to the theory stage, something has to be tested many many times. The burden of proof is very high. From the way you are talking you are being very skeptical… a scientific way of thinking. If you don’t believe in science… then by all means see how you do with out antibiotics, eat some arsenic, try to fly off a building, and maybe bend some silverware with your mind. Science has very clear conclusions to the outcome of all of these activities and I choose to believe science as to what will result. Though, not believing these and testing them is scientific. However, you may never live long enough to publish your findings.

As soon as you stop trying to disprove science, science has failed. Now, having faulty methodology 1/3 of the time, that is not the fault of science but of the humans doing the science. Do you understand the difference? (Either you are being difficult or you don’t know the difference.) Good science tells me gravity is there. I don’t expect to drop a bowling ball off a building 100 times and have it float in the air 33 times. If it does, you must continue to test and challenge gravity until you find the reason why it is floating 33% of the time.

Quote (Bubbagump @ July 14 2005,12:47)
Perhaps. Sorry man, science does not work in absolutes. You are confusing math and science. That is why there are scientific theories and mathmatical laws. I think you would hardly dispute the existence of gravity. In science it is a theory, math it is a law. Now to get to the theory stage, something has to be tested many many times. The burden of proof is very high. From the way you are talking you are being very skeptical… a scientific way of thinking. If you don’t believe in science… then by all means see how you do with out antibiotics, eat some arsenic, try to fly off a building, and maybe bend some silverware with your mind. Science has very clear conclusions to the outcome of all of these activities and I choose to believe science as to what will result. Though, not believing these and testing them is scientific. However, you may never live long enough to publish your findings.

As soon as you stop trying to disprove science, science has failed. Now, having faulty methodology 1/3 of the time, that is not the fault of science but of the humans doing the science. Do you understand the difference?

No one said or suggested anything about absolutes or math or living without science … such suggestions are way off the mark.

The media presents many a research project in ways that do sound like absolute facts (global warming, for example). What we don’t necessarily see are the subsequent studies that counter original claims. On occasion some secondary studies are published, but it doesn’t always undo the earlier impressions and/or misunderstandings. Finally here’s a story where the media says, “Hey, reader beware!”

Nevertheless, there still seems to be a general impression that science is flawless and untouchable; I’m just trying to put back the human touch.

In that case, I agree with you 100%. Every night on the news there is some new health claim that is a study of 10 people over 5 minutes or some silliness and it gets big news time. Poor representative sample, poor methodology, etc. As I said, folks don’t know good science from bad and are unwilling to hit the primary sources themselves. I have 100% faith in science, though not necessarily so much in the practitioners. Luckily, science allows for such skepticism and encourages baloney detection.

global warming is george bush’s fault!! it’s a scientific fact!!

I think the problem here stems from the FDA saying to the drug companies “prove your product is safe and effective” - The drug companies run the studies, publish the results and have almost complete control over the data which is presented to the FDA. They can edit the information to spin it in their own favour. The FDA is not an independent testing agency, they only evaluate the information they are given. If you have a multi-million dollar inventment in a new product, you are hardly likely to publish a report that says your product causes genital whithering, warts and frequent uncontrollable flatulance. (Although such a report would be a great laugh!)


<!–QuoteBegin>

Quote
global warming is george bush’s fault!! it’s a scientific fact!!


How is that possible the Shrubmeister says it doesn’t exist - and he’s incapable of being wrong (or lying), am I right?

.-=gp=-.

<!–QuoteBegin>

Quote
How is that possible…


now, where’s the sarcasm emoticon again?

:D

isaac

Anyone who doesn’t believe in gloabl warming… I woul dencourge them to understand the operations of a green house and the carbon cycle. Most folks know about the greenshouse effect, most don’t know how the carbon cycle effects such things…

<!–QuoteBegin>

Quote
most don’t know how the carbon cycle effects such things…


I think it was back in the seventies, in the Olympic games, that Lotus first built a carbon fibre bicycle for the British cyclist, and he won a whole stack of medals.

So, carbon fibre bike = going faster = more energy output = more global warming.

See… Not only is that an example of “good” science, but you now also understand the role of the carbon cycle in global energy economics. :cool:

My washing machine has a carbon cycle, but I have been afraid to use it.

Seriously, I have no idea why so many people don’t get the global warming thing. When Michigan is like Arkansas they will, I suppose…but then where will southern Florida be?

<!–QuoteBegin>

Quote
Seriously, I have no idea why so many people don’t get the global warming thing.


Ever tried to tell a mother her baby is ugly Tom? Talk CO2 emissions to a NASCAR fan? Discuss child slavery in the third world with someone who’s shares have just gone up by 30% 'cos Brazil is now allowing 7 year olds to be forced to work a 94 hour week?

“Where do you live?” I’m sometimes asked online.

Well, for many of us that’s easy to answer; we live totally inside our own heads, and the rest of the universe is “foreign parts”! Nothing to do with me! Not my problem!

Anyway, after those poor unfortunate foreign bastards die, God/Allah/Karma/The Fed.Reserve, will give them Paradise/Lots of Shagging/A bigger shit-shovelling spade in the next incarnation/or whatever; so that makes it all Ok. :)

:(

EDIT: Sorry guys, I’m not having a go at religion here, just having a go at those who use religion to justify inhumanity; and there’s a lot of them.

My point being, the carbon cycle is saving our butts at the moment. There will be a point where new growth in North America slows down. The sea is pretty much to capacity, so there ain’t no place left but the air. As soon as we hit capacity in terrestrial stores in 15-20 years, the real fun begins. Again, the warming thing, there is so much over whelming evidence… species falling off the top of mountains.