With thanks to the One Click Group 3.4.12

Cancer Research False Claims 29 March 2012

100 Amgen scientists were astonished to find that they were able to replicate the results of only 6 of 53 widely cited landmark cancer research papers. A committee of the National Academy of Sciences heard testimony about the tenfold increase during the last decade, in the number of scientific journal articles that had to be retracted.

But that number may obscure the far greater number of unsupportable published reports in so-called peer reviewed academic journals that are never retracted. A report in NATURE, by Glenn Begley (former head of cancer research at Amgen) and Lee Ellis (surgical oncologist at MD Anderson), Drug development: Raise standards for preclinical cancer research describes the effort of 100 Amgen scientists to replicate the results claimed by the authors of 53 widely cited landmark cancer research papers.

The Amgen scientists were astonished to find that they were able to replicate only 6 (11%) of the published conclusions.

“It was shocking,” said Begley, now senior vice president of privately held biotechnology company TetraLogic, which develops cancer drugs. “These are the studies the pharmaceutical industry relies on to identify new targets for drug development. But if you’re going to place a $1 million or $2 million or $5 million bet on an observation, you need to be sure it’s true. As we tried to reproduce these papers we became convinced you can’t take anything at face value.”

So, it is cancer research academics–not industry scientists–who are responsible for polluting the scientific literature with false “positive” claimed findings. Indeed, even when the academic got the result he published only once–after attempting, but failing, six times to replicate his own finding–he nevertheless submitted his unsupportable finding for publication.

Some authors required the Amgen scientists sign a confidentiality agreement barring them from disclosing data at odds with the original findings. “The world will never know” which 47 studies — many of them highly cited — are apparently wrong, Begley said. Such a legal shield protects fraud and fraudsters.

Last year, Bayer scientists reported similar findings…

A scientist at the University of Nova Scotia who had worked at Merck, is quoted stating:

“It drives people in industry crazy. Why are we seeing a collapse of the pharma and biotech industries? One possibility is that academia is not providing accurate findings.”

Ferric Fang of the University of Washington, who addressed the NAS panel, said he blamed a hypercompetitive academic environment that fosters poor science and even fraud, as too many researchers compete for diminishing funding.

“The surest ticket to getting a grant or job is getting published in a high-profile journal,” said Fang. “This is an unhealthy belief that can lead a scientist to engage in sensationalism and sometimes even dishonest behavior.”

The Amgen authors of the Nature article who could replicate the findings of only 6 o the 53 published reports indicate that some authors of the dubious published reports required them to sign a confidentiality agreement barring them from disclosing data at odds with the original published findings. Thus, Glenn Begley stated: “The world will never know” which 47 studies — many of them highly cited — are apparently wrong.

How can one justify a public expenditure of more than $4.9 billion annually for cancer research whose integrity cannot be relied upon?

An excellent review by Sharon Begley (Reuters) : In Cancer Science, Many “Discoveries” Don’t Hold Up

Vera Sharav Reuters

In cancer science, many “discoveries” don’t hold up By Sharon Begley NEW YORK | Wed Mar 28, 2012

(Reuters) – A former researcher at Amgen Inc has found that many basic studies on cancer — a high proportion of them from university labs — are unreliable, with grim consequences for producing new medicines in the future.

During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 “landmark” publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development.

Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature.

“It was shocking,” said Begley, now senior vice president of privately held biotechnology company TetraLogic, which develops cancer drugs. “These are the studies the pharmaceutical industry relies on to identify new targets for drug development. But if you’re going to place a $1 million or $2 million or $5 million bet on an observation, you need to be sure it’s true. As we tried to reproduce these papers we became convinced you can’t take anything at face value.”

The failure to win “the war on cancer” has been blamed on many factors, from the use of mouse models that are irrelevant to human cancers to risk-averse funding agencies. But recently a new culprit has emerged: too many basic scientific discoveries, done in animals or cells growing in lab dishes and meant to show the way to a new drug, are wrong.

Begley’s experience echoes a report from scientists at Bayer AG last year. Neither group of researchers alleges fraud, nor would they identify the research they had tried to replicate.

But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers.

George Robertson of Dalhousie University in Nova Scotia previously worked at Merck on neurodegenerative diseases such as Parkinson’s. While at Merck, he also found many academic studies that did not hold up.

“It drives people in industry crazy. Why are we seeing a collapse of the pharma and biotech industries? One possibility is that academia is not providing accurate findings,” he said.

BELIEVE IT OR NOT

Over the last two decades, the most promising route to new cancer drugs has been one pioneered by the discoverers of Gleevec, the Novartis drug that targets a form of leukemia, and Herceptin, Genentech’s breast-cancer drug. In each case, scientists discovered a genetic change that turned a normal cell into a malignant one. Those findings allowed them to develop a molecule that blocks the cancer-producing process.

This approach led to an explosion of claims of other potential “druggable” targets. Amgen tried to replicate the new papers before launching its own drug-discovery projects.

Scientists at Bayer did not have much more success. In a 2011 paper titled, “Believe it or not,” they analyzed in-house projects that built on “exciting published data” from basic science studies. “Often, key data could not be reproduced,” wrote Khusru Asadullah, vice president and head of target discovery at Bayer HealthCare in Berlin, and colleagues.

Of 47 cancer projects at Bayer during 2011, less than one-quarter could reproduce previously reported findings, despite the efforts of three or four scientists working full time for up to a year. Bayer dropped the projects.

Bayer and Amgen found that the prestige of a journal was no guarantee a paper would be solid. “The scientific community assumes that the claims in a preclinical study can be taken at face value,” Begley and Lee Ellis of MD Anderson Cancer Center wrote in Nature. It assumes, too, that “the main message of the paper can be relied on … Unfortunately, this is not always the case.”

When the Amgen replication team of about 100 scientists could not confirm reported results, they contacted the authors. Those who cooperated discussed what might account for the inability of Amgen to confirm the results. Some let Amgen borrow antibodies and other materials used in the original study or even repeat experiments under the original authors’ direction.

Some authors required the Amgen scientists sign a confidentiality agreement barring them from disclosing data at odds with the original findings. “The world will never know” which 47 studies — many of them highly cited — are apparently wrong, Begley said.

The most common response by the challenged scientists was: “you didn’t do it right.” Indeed, cancer biology is fiendishly complex, noted Phil Sharp, a cancer biologist and Nobel laureate at the Massachusetts Institute of Technology.

Even in the most rigorous studies, the results might be reproducible only in very specific conditions, Sharp explained: “A cancer cell might respond one way in one set of conditions and another way in different conditions. I think a lot of the variability can come from that.”

THE BEST STORY

Other scientists worry that something less innocuous explains the lack of reproducibility.

Part way through his project to reproduce promising studies, Begley met for breakfast at a cancer conference with the lead scientist of one of the problematic studies.

“We went through the paper line by line, figure by figure,” said Begley. “I explained that we re-did their experiment 50 times and never got their result. He said they’d done it six times and got this result once, but put it in the paper because it made the best story. It’s very disillusioning.”

Such selective publication is just one reason the scientific literature is peppered with incorrect results.

For one thing, basic science studies are rarely “blinded” the way clinical trials are. That is, researchers know which cell line or mouse got a treatment or had cancer. That can be a problem when data are subject to interpretation, as a researcher who is intellectually invested in a theory is more likely to interpret ambiguous evidence in its favor.

The problem goes beyond cancer.

On Tuesday, a committee of the National Academy of Sciences heard testimony that the number of scientific papers that had to be retracted increased more than tenfold over the last decade; the number of journal articles published rose only 44 percent.

Ferric Fang of the University of Washington, speaking to the panel, said he blamed a hypercompetitive academic environment that fosters poor science and even fraud, as too many researchers compete for diminishing funding.

“The surest ticket to getting a grant or job is getting published in a high-profile journal,” said Fang. “This is an unhealthy belief that can lead a scientist to engage in sensationalism and sometimes even dishonest behavior.”

The academic reward system discourages efforts to ensure a finding was not a fluke. Nor is there an incentive to verify someone else’s discovery. As recently as the late 1990s, most potential cancer-drug targets were backed by 100 to 200 publications. Now each may have fewer than half a dozen.

“If you can write it up and get it published you’re not even thinking of reproducibility,” said Ken Kaitin, director of the Tufts Center for the Study of Drug Development. “You make an observation and move on. There is no incentive to find out it was wrong.”

(Note: Amgen researcher C. Glenn Begley is not related to the author of this story, Sharon Begley)

(Reporting By Sharon Begley; Editing by Michele Gershberg and Maureen Bavdek)

See http://avilian.co.uk/2012/03/scandalous-scientists-and-doctors-falsifying-research-data/

See http://avilian.co.uk/2012/02/pharma-fraud-withheld-clinical-trial-data-shows-antidepressants-no-better-than-dummy-pills/

See http://avilian.co.uk/2012/02/misconduct-pervades-uk-research-according-to-financial-times/

See http://avilian.co.uk/2012/01/reporting-of-research-ghosts-in-the-machine/

See Why Most Published Research Findings Are False Ioannidis JPA (2005) Why Most Published Research Findings Are False. PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124 ‘… There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research…’

See http://avilian.co.uk/2012/01/one-third-of-scientific-researchers-could-not-find-the-original-data-to-back-up-figures-in-scientific-papers-when-these-were-questioned/

See http://avilian.co.uk/2011/10/retractions-in-the-medical-literature-how-many-patients-are-put-at-risk-by-flawed-research/

See A key player in stoking this Swine Flu hysteria was Dr Iain Stephenson, a Consultant at University Hospital of Leicester NHS Trust who has been found guilty of vaccine research fraud on a grand scale by the General Medical Council (GMC).

See Thirteen per cent of scientists or doctors have said they know of colleagues who have fabricated data in order to get research published. More than one in ten (13%) scientists or doctors have witnessed colleagues intentionally altering or fabricating data to get published or during their research, a poll suggests. The survey of almost 2,800 experts in the UK also found 6% knew of possible research misconduct at their institution that has not been properly investigated. The poll, for the British Medical Journal (BMJ), is being presented at a meeting aimed at tackling research misconduct in the UK. 13 January 2012

See Mental scores decline precipitously at 30 months after anthrax vaccine, but CDC spins study to say vaccine safe Thursday, January 12, 2012

See Research misconduct is widespread and harms patients, BMJ 2012; 344 doi: 10.1136/bmj.e14 (Published 5 January 2012), Cite this as: BMJ 2012;344:e14

See http://www.theoneclickgroup.co.uk/news.php?id=6732#newspost  Patient No. 28665 was helping a Chesterfield company test a drug dubbed the “female Viagra” but stopped showing up for exams in December 2007. That didn’t stop a staffer of the company, PPS Clinical Research STL, from claiming that six months later the patient came in for an exam, was diagnosed with medical conditions, had undergone tests and even had given a urine sample. That admission was made in the company’s guilty plea to a federal felony charge of obstructing a proceeding of a federal agency. The drug the company helped test, aimed at women who suffered from an unexplained lack of sexual desire, never made it to market. As part of the plea, PPS admitted that it “corruptly influenced, obstructed, and impeded” an inspection by the Food and Drug Administration by providing false patient records in May 2010. During the trials, the drug’s manufacturer faulted PPS for failing to have a trained investigator conduct some exams, for back dating records and for failing to properly oversee one patient’s selection for the study, according to prosecutors. Boehringer did not respond to emails seeking comment.

See http://www.theoneclickgroup.co.uk/news.php?id=6610#newspost A well-known psychologist in the Netherlands whose work has been published widely in professional journals falsified data and made up entire experiments, an investigating committee has found. Experts say the case exposes deep flaws in the way science is done in a field, psychology, that has only recently earned a fragile respectability.

See ‘…it is simply no longer possible to believe much of the clinical research that is published…

See The Whole Truth? Meta-Analysis of Reboxetine Trials Calls Into Question Veracity of All Industry-Sponsored Research

See Lies, Damned Lies, and Medical Science

See Science for Sale: Protect Yourself From Medical Research Deception

See British Medical Journal – now only 11% of NHS Treatments Backed by Evidence (was 13%)

See Big Pharma researcher admits to faking dozens of research studies

See CAM deaths 0 Iatrogenic deaths 999,936 annually

See Big Pharma Bias at Harvard Medical School

See Andrew Wakefield demands retraction from BMJ after documents prove innocence from allegations of vaccine autism data fraud

See Reporting of Conflicts of Interest in Meta-analyses of Trials of Pharmacological Treatments. Michelle Roseman, BA; Katherine Milette, BSc; Lisa A. Bero, PhD; James C. Coyne, PhD; Joel Lexchin, MD; Erick H. Turner, MD; Brett D. Thombs, PhD. JAMA. 2011;305(10):1008-1017. doi: 10.1001/jama.2011.257