|
The views expressed herein are
those of the author and do not necessarily reflect the position of ORI, the
Department of Health and Human Services, or any component thereof.
 |
|
Sandra L. Titus
|
We know from major scandals in the recent past that many professions have
been severely and adversely affected when our society recognized large
lapses in ethical standards. Unfortunately, each profession has suffered
from members who knew about fraudulent or other highly problematic
behaviors, but who contributed to cover-ups such that the public remained
unaware of the problem. They hoped that a pretense of normality and denial
could be successfully sustained. When the dam has burst on various
occasions, the public has learned not only about an ethical lapse in
question at the time, but also that the professions’ leaders had
foreknowledge and awareness, yet had not taken appropriate and needed
action. It is small wonder that the public in these instances is outraged in
the face of what appears to be a total and complete failure in
self-regulation.
The purpose of this paper is first to examine available data on research
misconduct, and second to explore whether there is reason to suspect that
scientists might be operating with pretense and denial. Do these indicators
reflect an early harbinger as we have seen in other professions? Thus,
throughout this abbreviated review of social science research on research
misconduct, I will raise questions about the effectiveness of self
regulation. I also will discuss two essential strategies that can be applied
to prevent misconduct, promote integrity and build professional
self-regulation, supervision, and whistleblower protection.
The Office of Research Integrity (ORI) has collected data for the last 18
years on research misconduct cases that are reported to the Department of
Health and Human Services (DHHS) by universities. ORI does not seek and
discover cases, but rather provides oversight and guidance to institutions
that conduct the required investigation when someone reports a possible case
of misconduct involving DHHS supported research. Over a 10-year period, ORI
made an average 13 findings of misconduct per year (approximately the same
number of investigations result in no finding of misconduct) (9). This does
not seem to be a very large number of cases of misconduct per year. Thus,
many people question whether this number reflects the actual rate of
misconduct among scientists, or more accurately represents the tip of an
iceberg of pervasively more serious patterns and incidents.
Since there are over 325,000 National Institutes of Health (NIH) supported
scientists, one could argue that 13 findings per year indicate that
scientists do not resort to fabricating and falsifying research, because
this is a very small proportion (1 per 25,000 researchers). However, there
are many unanswered questions. Does everyone who sees misconduct report it?
How many incidents go undetected? If an allegation is made at the
institution, what proportion has a thorough investigation?
There are three compelling recent reports that provide evidence that
under-reporting is pervasive and that the scientific culture has many
behaviors that are eroding our confidence in the findings that researchers
report.
One source of information on
how scientists are behaving comes from Michael Rossner, Editor, Journal of
Cell Biology (JCB). In a report by Garath Cook, who interviewed Rossner, we
learn that for the past three years JCB has had to reject 1/100 of the
accepted papers because of serious improper digital image manipulations that
were misleading, images that either added or subtracted elements that
changed the experimental results (1). JCB has been vigilant. The editors
have taken steps at the production level to detect fraudulent data. But as
we have seen in 2005 and 2006, few journals attempt to catch fraudulent data
at either the peer review stage or during production. The news reports of
the scandals of Hwang, Sudbø, Poehlman, and others have aptly demonstrated
this problem. This analysis by JCB clearly has indicated that journal
reviews cannot easily review source data and they cannot be expected to be
the primary gate keeper. The old mantra that journal peer review is a
mainstay of self regulation appears to be unreliable. We need to identify
other ways to assure the quality and validity of published articles. One
such way would be for institutions to develop a robust peer review process
that would be required before an article could be submitted.
The second source that
describes researchers’ behaviors comes from the 2005 study on research
misconduct reported by Martinson, Anderson, and DeVries, who described
behaviors of 3,247 NIH scientists who self-reported how they handled
research issues (3). They found that 3/1,000 principal investigators
self-reported in the past three years that they were falsifying or engaging
in that practice known more colloquially as “cooking” data. Hence, we then
have 1 per 1,000 scientists per year who have confessed to research
behaviors that would be judged to be misconduct. A quick extrapolation to
the 325,000 investigators who are supported by NIH funds would suggest that
there would be approximately 325 incidents of falsified or “cooked” research
data each year. This is 12-fold more than the ORI case load and 25-fold
higher than ORI’s determination of misconduct. This suggests that possible
research misconduct is being ignored and underreported.
Perhaps even more worrisome is the finding by Martinson that 15% of the
researchers report that they drop observation points and 27.5% report they
have inadequate record keeping. Without accurate data records, a researcher
can appear to have fabricated or falsified data. How can we trust scientific
findings when the data supporting them are not carefully collected? What
message does this send to those who are being trained by the principal
investigator (PI)? Lack of sound record keeping practices for one in every
four researchers means everyone must start to be much more suspicious about
the data and findings that are presented and published.
A third source of information on how scientists are faring in avoiding the
dark side comes from inspections of the Food and Drug Administration (FDA)
clinical data. In 2004, Bramstedt reviewed the warning letters that the
Centers for Drugs and for Devices issued in a two-year period. In the 58
warning letters, she found that 8% were for fabrication and falsification
(n=3) (2). In this same time period FDA made 1,738 field inspections; thus,
1.7/1,000 investigators or their subordinates had committed research
misconduct in the two-year period (annualized to .9/1000) (6). This finding
is noteworthy because it has occurred in a field that requires following
Good Clinical Practice Standards (GCP) and investigators are aware of the
possibility that their data will be inspected. When FDA discovers falsified
data, the entire trial may be rejected; the sponsor will not obtain FDA
approval to market the drug if their data are found to have been fabricated
or falsified. Thus, most sponsors closely monitor their data sets to ensure
compliance with FDA requirements for data integrity. In non-FDA-regulated
science, we rely almost exclusively on the belief that researchers are
honest. Very few studies are validated by others before publication.
Research misconduct is occurring. It is not a myth. If we ignore this fact
we are burying our heads in sand. While we do not know the exact rate, it
appears that using 1:1,000 is a reasonable assumption to make at this time.
It could be substantially higher than this because the two data sources
(Martinson and FDA analysis) are conservative estimates. It is not
unreasonable to assume the self report data is conservative because people
will under-report negative behaviors about themselves. The FDA data is a
conservative finding because the regulated drug industry knows that FDA can
inspect their data, and, therefore, they do not want missing source data and
flawed data sets. Most other research does not have this level of scrutiny
on source data. In short, this means we need major interventions that force
us to develop new ways to assure data integrity. In addition to educational
approaches, termed responsible conduct of research (RCR), there are two
critical and essential strategies that scientists can use to thwart research
misconduct: effective supervision and professional transparency through
whistleblower protections.
| Table 1.
Adjusting Bramstedt Annual Analysis with FDA Fiscal Analysis to
Determine Denominator Related to Finding Three Misconduct Cases. |
Inspections
Drugs
Devices
Total |
FY
2002
288
153 |
Adj%
.66
.66
|
2002
Cases
192
102
294
|
FY
2003
674
356
|
Adj %
0
0
|
2003
Cases
674
356
1,030 |
FY
2004
655
333
|
Adj. %
.42
.42
|
2004
Cases
275
140
415 |
Supervisory Role of Mentor: Attention to Data and to Opportunities for
Misconduct
The first way that research misconduct can be thwarted is to have much
clearer faculty understanding of what they are expected to do if they have a
research group. The PI, who is responsible for advising and mentoring PhD
trainees and others, needs to understand the scope of this responsibility.
Specifically, the PI must be willing to spend time supervising their
trainees and to focus on reviewing and interpreting the source research data
with the trainee. Failure to examine trainee source data and have clear
research standards is present in a large proportion of the trainee
misconduct cases reviewed by ORI (8). Adequate supervision prevents trainees
from having the “opportunity” to fabricate or falsify data. Adams and Pimple
point out that when there is the absence of capable supervision and also a
lack of informal social interaction, there is a much greater likelihood that
one can and will engage in problematic and irresponsible behaviors, such as
research misconduct (7). They point out that all people have the ability and
capacity to carry out inappropriate, unethical, and even criminal acts.
However, research has shown that when “opportunities” to engage in
inappropriate behavior are minimized, there is always a reduction in crime.
A scientist that works alone might easily be able to hide unacceptable
practices. If one knows how to use sophisticated equipment that few others
use, then one can quite easily falsify data. A supervisor who attends
carefully to the appropriate acquisition of data will reduce the opportunity
for engaging in questionable research behaviors, as well as serious research
misconduct. This is not to suggest that the PI become a policeman, but
rather to identify how attention to “opportunity” can be easily incorporated
as a component of supervision. Howard Hughes Medical Institute’s educational
offering, “Making the Right Moves,” discusses many other practical methods
that faculty can learn to manage and supervise the work of others (5).
However, one of the most critical parts of supervision in preventing
research misconduct is setting clear standards and being physically present
and observing how trainees are performing. Mentors can define and enforce
standards of integrity or they can be lax and not be involved. We must alter
the position that “absentee mentoring,” is acceptable. The justification
that a PI must focus primarily on securing money rather than on supervising
their group is very detrimental to science. Both faculty and students need
to change their behaviors in order for the public and fellow scientists to
trust the research findings.
Promoting Professional Transparency: Whistleblower Protections
Supervision alone does not address the observation that many conscientious
and trustworthy scientists know about misconduct but do not want to report
misconduct (4). There is a widely held belief and an experienced fear among
any number of professionals that a whistleblower, termed a complainant in
misconduct cases, will suffer retribution. People who want to report are
often caught in a proverbial Catch 22. If one reports, one may well be
considered as breaking rank or being disloyal. Retribution can follow. But
in the wider scheme of things, failure to report is itself a collusion,
condones the wrong already done, and contributes significantly to the view
that science has no mechanism to prevent dishonest behavior. Having such
blinders on contributes greatly to the corruption of any profession.
We need to think seriously about what must be done to protect those who do
seek to report misconduct honestly, forthrightly, and for the right
motivations. A lab chief must make it clear that s/he has zero tolerance for
research misbehaviors and research misconduct. Those in the lab should be
told that if they observe behaviors by others that appear suspicious or not
quite right, then they should and must report it to the lab director, who
will take responsibility to look into it. Institutional polices need to be
strengthened so that they define the obligation to report misconduct. In a
study done in 2000, only 29% of institutional misconduct policies had an
explicit statement about the obligation to report (10). Institutions need to
specify this obligation to report misconduct. And institutions need to spell
out possible mechanisms that they will use to protect whistleblowers.
Institutional commitment via policy changes would be a first step in
creating the necessary institutional dialogue that needs to take place.
When someone makes a good faith allegation of possible misconduct, there
must be better ways to protect them. In research done on ORI cases, Lublin
found that 70% of whistleblowers involved in a research misconduct case
reported one or more negative outcomes (11). Nineteen percent were fired,
25% reported being ostracized, 43% were urged to drop their concerns, 51%
reported mental heath problem, 25% reported marital problems, 33% reported
negative impact on finances, and so on. This should not be happening!
Attitudes and behaviors by research administrators and faculty members that
lead to ostracizing and marginalizing those who report possible research
misconduct are unacceptable behaviors and need to be treated as such. Would
you be willing to report a case of research misconduct? If you answer no,
then look at whether you too feel a high degree of vulnerability that seems
to go hand in hand with being a whistleblower. And then ask yourself what
your institution would need to do so that you would feel safer.
Conclusion: Changing How We Self Regulate
This paper has not been about “Science” but rather about the “Behavior of
Scientists.” Supervising trainees and protecting whistleblowers are not the
only issues that scientists need to address to assure data integrity and
demonstrate that they are self monitoring their own behavior, but these are
two areas where the payoffs can be large. We need to focus our attention on
the ethical conduct of science and how research misconduct is monitored,
reported, and corrected. There is sound evidence in all fields of science of
careless record keeping, poor standards for data interpretation,
questionable research practices, and some level of data fabrication and
falsification. People fear reporting it because the bearer of bad news can
often be considered the bad news itself. However, the price of not reporting
research misconduct is extremely high. In the words of the National Academy,
“Misconduct can harm individuals outside of science (as when falsified
results become the basis of a medical treatment), it squanders public funds,
and it attracts the attention of those who would seek to criticize science.”
(12)
If we do not attend to faulty data keeping and careless analysis, then we
inevitably promote at the very least a climate that breeds cutting corners,
something closely approaching actually committing misconduct. If falsified
data is used in grant applications, then we fund unworthy proposals and
obstruct scientific proposals with real integrity from being sponsored. If
we do not attend to behaviors that lack integrity, then we as a scientific
community promote and allow a waste of time and resources trying to
replicate and repeat experiments. No person of professional integrity wants
to learn that her or his name is associated with a publication discredited
because of data fraudulence. Who wants to discover that a relative is
receiving treatment based on fraudulent research? The stakes are very
high-for investigators, for their institutions, and most of all for the
public whose benefit is the purpose of research itself.
We need to attend to this hidden crisis before it leads to further
undesirable outcomes or further erosion of the good name of academic and
scientific inquiry. Institutions need to help bring about this change by
setting and enforcing standards, such as directing PIs to discuss and
publicly post the rules that will be used for record keeping in their lab.
It is well recognized in organizational research that individual behavior is
more likely to be modified and changed when the entire culture adopts and
adheres to the new standards. Thus, we must focus energy on the system, as
well as on the individual. But individuals change systems! We need change
agents to step forward who want to protect their institution’s name and
reputation and are capable of insisting that scientists change their
behavior. How can we say we are self-regulating if institutions and
scientists do not develop better ways to promote research integrity? Shame
on us if we fail to face the dark side of science and respond to the many
indicators that point out that research integrity is being compromised.
References
1. Adams, D., Pimple, K. “Research Misconduct and Crime Lessons From
Criminal Science on Preventing Misconduct and Promoting Integrity,”
Accountability in Research, 12: July 2005, 225-240.
2. Bramstedt, K.A., “A Study of Warning Letters Issued to Clinical
Investigators by the United States Food and Drug Administration,” Clinical
Invest Med 27: (3) June 2004, 129-134.
3. Cook, G., “Technology Seen Abetting Manipulation of Research” Boston
Globe.
http://www.boston.com/news/science/articles/2006/01/11/technology_seen_abetting_manipulation_of_research/,
January 11, 2006.
4. Gallup Study, “Reporting Suspected Research Misconduct in Biomedical and
Behavioral Research” Description posted on ORI web site,
http://ori.dhhs.gov/publications/reporting_suspected_research_misconduct.shtml
This study, conducted by The Gallup Organization, provides a description of
the frequency and types of suspected misconduct which 2212 scientists
observed in three academic years (2003-2005). The study indicates that a
substantial amount of suspected research misconduct is not being reported.
Twenty percent of the scientists wrote that the most important way to
promote reporting research misconduct is the degree of protection offered to
whistleblowers. An article has been submitted to a peer reviewed journal. “
5. Howard Hughes Medical Institute and Burroughs Wellcome Fund. “Making the
Right Moves;” course available at http://www.hhmi.org/labmanagment, 2004.
6. Lublin, James. Consequence of Whistleblowing for the Whistleblower in
Misconduct in Science Cases, October 30,1995.
http://ori.hhs.gov/documents/consequences.pdf
7. Martinson B., Anderson M., DeVries R., “Scientists Behaving Badly” Nature
435: June 9, 2005, 737-738.
8. National Academy of Science, On Being a Scientist, National Academy
Press, Washington DC, 1995
9. Report submitted to ORI, “Analysis of Institutional Policies for
Responding to Allegations of Scientific Misconduct,” September 29, 2000
http://ori.dhhs.gov/documents/institutional_policies.pdf
10. Rhoades, L. “ORI Closed Investigations into Misconduct Allegations
Involving Research Supported by the Public Health Service: 1994-2003”
http://ori.hhs.gov/research/intra/studies_completed.shtml, 2005.
11. Titus, Sandra, Personal communication with C. Hommel, OS, FDA, on April
23, 2007, who provided information which led to the calculation in Table 1.
12. Wright, D., Titus, S.L., Cornelison, J., “Mentoring and Research
Misconduct: An Analysis of Research Mentoring in Closed ORI Cases,” An
article has been submitted to a peer reviewed journal.
Acknowledgment:
I thank Kay Fields, Edward Gabriele, Robert K. Leedham Jr., Lawrence Rhoades
and Cynthia Ricard, for their thoughtful and helpful critiques on early
drafts of this paper. |
