Research Integrity: A Dean of Research’s Perspective – Professor Graeme Milligan, University of Glasgow, Scotland.
Professor Milligan, Dean of Research at the College of Medical, Veterinary, and Life sciences at the University of Glasgow, began by pointing out that it is important to monitor and police research processes.
Current views on research integrity are shaped around The Singapore Statement, 2010. The four key features of this statement comprise principles around research integrity. These are honesty in all aspects of research; accountability in the conduct of research; professional courtesy and fairness in working with others; and good stewardship of research on behalf of others.
“If we fail to match up to these aspirations then there is a danger of committing an aspect of research misconduct, which by definition is a significant departure from accepted practice that was knowingly and recklessly committed,” emphasised Professor Milligan. “There should be a preponderance of evidence to establish deliberate falsification of data.”
He highlighted that an unintentional and honest mistake is not research misconduct, nor are the diverse views and interpretations of the same piece of data, because everyone can differ in this respect.
Fabrication, falsification and plagiarism
There are three accepted types of misconduct: fabrication of data, falsification of data and plagiarism. Fabrication denotes the making up of data or results and their subsequent recording or reporting. Professor Milligan recounted a well-known example from dermatology that involved studies of immunological rejection of transplanted tissues. In 1974, the New York-based immunologist William Summerlin was investigating the transplantation of dermis from a darkly pigmented mouse to the back of a light-coloured mouse without melanocytes. Summerlin showed his supervisor a single experimental animal that was the best evidence of transplant success, drawing attention to an apparent greying of the patch. His early experiments looked exciting and suggested that his work was en route to overcoming transplant rejection. However, in time, it proved increasingly difficult to replicate these data, and it became evident that Summerlin had deliberately falsified results by taking a black marker pen and colouring in a patch on the mouse where the transplanted black dermis had started to fade to grey. “It seems unbelievable, but it was an issue of falsification with a marker pen,” said Professor Milligan.
Peer review also has a role to play, Professor Milligan pointed out. In another example of fabrication, he told the tale of a scientist from Korea who said she could convert a dermal cell into pluripotent stem cells just by reducing the pH of the growth environment. “This was published in Nature. But nobody could replicate the work, so it was withdrawn and the head of her university had to explain what happened. Her career was wrecked,” he stressed.
Turning to falsification, Professor Milligan highlighted that one challenge in moving towards experimental medicine is the large variability of results. “This is the truth of working with genetically non-pure organisms and the danger is that it is too easy to manipulate or change or omit results such that the research record is inadequately represented,” he explained. He gave examples of this in practice, including the removing of outlying data points without appropriate statistical analysis, the selecting of subsets of data, and in particular the subjective and biased scoring of images. Image manipulation has been a related major challenge in cell biology. “Omitting a single data point can transform an image from showing no effect to something that is highly statistically significant. Sometimes there is good reason to leave out an outlier, but unless you have carried out the correct statistical test then it probably isn’t reasonable to do so,” said the professor.
Plagiarism, which is the appropriating of other people’s ideas, results or words without giving appropriate credit is challenging because we all re-use ideas, Professor Milligan observed. “But I always aim to give credit for so doing.”
The final problem discussed lies in the lack of reporting of negative or contradictory data. Professor Milligan commented that it is widely understood that it is much easier to publish positive data than data that do not show an effect, although he also noted that journals are making increasing efforts to publish negative data.”
Professor Milligan then posed the question of whether such potential pitfalls were really a major problem, or whether, in reality, they occur very infrequently. Surveys have been conducted where scientists have been asked what they know about scientific misconduct. “In fact, when these surveys have been done, we find that one in 50 admit to fabricating or modifying data as part of their career,” he said, with some surprise. “If you ask scientists if they know of someone who has done this then the number increases remarkably. There is a seven-fold increase in the number who know of someone else who has been involved in misconduct, and 34% admitted that they’d been involved in questionable practices and nearly everyone believes that everyone else does it.”
What are the reasons for breaches of research integrity?
All researchers are under various pressures to publish, as well as facing pressures relating to aspects of career retention and advancement. “There are also a number of cultural issues around carrying out work and giving credit to others, and clearly sometimes there may be other problems associated with an individual,” Professor Milligan remarked.
Turning to the consequences of scientific misconduct, one of the key issues is pollution of the literature. For example many pharmaceutical companies fail to reproduce the work of academic labs and vice versa. “There are capital costs, brand costs and human costs associated with scientific misconduct,” he stressed.
The cost associated with a brand is illustrated by the example of Alirio Melendez, who gained his PhD at the University of Glasgow, worked in Singapore, and then returned to Glasgow as a senior lecturer and then professor. The misconduct was based on the publication of high profile articles in high impact journals, which were all made up, reported Professor Milligan. A total of 50 of his papers were withdrawn by the University of Glasgow after an investigation revealed that he had manipulated many of the data. These data were fraudulent, he explained. This represented a brand cost for the associated universities.
“Furthermore, there are human costs including those to clinical trials, uptake of medication, and the distress to individuals.” Autism and MMR (measles, mumps and rubella) vaccination is a classic example where we see parents moving away from vaccinating their children, said professor Milligan, further illustrating the point about human costs. “This was driven by research that was, at best, questionable. There are also questions around the use of statins and whether their use should be further extended.”
Finally, he highlighted issues around authorship, in particular, the issue of honorary authorship that remains a challenge in many areas.
“Here at the University of Glasgow, we try to avoid situations where we coerce researchers into obtaining a certain result. Generally, in science we’ve moved away from the idea of disproving a hypothesis,” said the professor. “Mostly we try to prove it, so we have a better grant application to submit. However, we should praise hypothesis-killing research and we also need to tolerate errors. I’ve have certainly not got every experiment right and we need to nurture a non-judgemental forum for research discussion so we do not become trapped in thinking that this is the correct answer. We should also encourage early career researchers to challenge those at more senior levels.”
Publication Ethics in the UK –Sally Marshall, Editor-in-chief of Diabetologia, and Professor of Diabetes, Newcastle University, England.
With a room full of aspiring researchers keen to publish before her, Professor Sally Marshall talked on publication ethics in the UK from an editor-in-chief’s perspective. She noted first that Diabetologia is the official journal of the European Association for the Study of Diabetes (EASD), and that, as editor, she “more or less has free rein to direct what is published in the journal”.
She explained that part of her job is to maintain the reputation of the journal as solid and trustworthy. “I don’t want the journal damaged by publishing incorrect data.” She discussed the efforts they make to spot errors and inconsistencies via quality checks – deliberate or not. “We double check as much as we can.”
At submission, an author has to complete a checklist to ensure that the studies have been conducted according to best practice, she pointed out. “The questionnaire is an aide memoire to ensure readers, editors, and peer-reviewers are convinced that the study has been done properly. It should be relatively quick for the author to complete.”
Following this, a review process is initiated. Once provisionally accepted, the copy is checked for plagiarism using the iThenticate plagiarism checker, while imaging is checked with image forensics. The copy-editing phase also incorporates a quality check. “The iThenticate plagiarism checker gives a percentage duplication with another paper. A small percentage is fine, because there’s probably only one way of writing a concise method section, and that is legitimate if you have used the same method,” she acknowledged. “However, 58% duplication, for example, rings alarm bells, perhaps related to a study by the same team of researchers described in a different paper. “This is a red flag,” she pointed out, adding that people understandably want to produce a good number of papers from one study, given that many studies consume a lot of resource of both time and money. “There’s an acceptable level of dividing up of data, but you need to divide up carefully between papers. Better two papers in high ranking journals than ten in less high-ranking ones.”
Duplicate publications can happen, if rarely, she noted. In this case, an explanation would be sought from the authors. “Without a good explanation we would just stop publication. I have a scientific integrity panel of experts behind me and they would advise here,” said Professor Marshall.
Image forensics will allow a researcher to alter the whole image, for example, contrast or brightness, but it will not accept alteration to one part, for example, enhancing or obscuring certain features. “A Western blot image of lanes should all be run from one gel, unless it is made very clear that two different gels have been used for a good reason.” She showed some examples of errors in blot images and said that if there were questions the journal would ask for original images, which usually pass the forensic tests. “We now ask for originals at submission.”
The last example shown by Professor Marshall involved an error picked up at copy editing stage. “Copy editors ensure that the paper is consistent, clear, unambiguous, and in the house style of the journal. Consistency is key,” she highlighted.
She showed a recent example of how copy editing had picked up an error with numbers. The text said 498 subjects had been screened, 112 excluded, and 386 included; while the CONSORT flow chart said 498 screened, 102 excluded, and 384 included. “We went back to the authors, thinking it was simple error but when the authors checked they realised they had made a major error. The way individuals were included and excluded was complicated in this study and somewhere along the line, the numbers had become scrambled,” she explained. “They went back through their data, reanalysed and re-submitted. It was a genuine mistake and the authors were open with us. This paper has now been published. Annoyingly it was missed by peer-reviewers, an associate editor and me, prior to the copy editors spotting it.”
“Genuine errors happen – we don’t get it right all the time,” Professor Marshall reinforced. “These errors would be approached honestly and transparently. If you realise the error before it is accepted, go straight to the editor and explain fully and work with them to solve it.”
Once a paper is published, it is a little different. “You still need to acknowledge and explain that you’d like to make the correction. In this case we would usually publish an erratum. If you have to publish one erratum then your peers will not think any less of you.” Only with a big error do we need to retract a paper. “Retraction often happens if we have not had a satisfactory answer from the author in response to a query.”
However, she explained, if there is real concern then the Scientific Integrity Panel is consulted. This comprises a group of very experienced, highly respected scientists.
The Scientific Integrity Panel inspects all the data and provides advice to the editor, and if evidence of misconduct is found, letters are composed to the authors and to the head of their institution, requesting an institutional investigation. If work is already published, this may lead to publication of an ‘Expression of Concern’ and ultimately retraction of paper, if no satisfactory explanation is given. “This is very bad news for author and the journal,” stressed Professor Marshall.
“When you do make a mistake, be open, honest and work with the editorial team to resolve it as speedily and effectively as possibly,” she concluded.
Publication Ethics in an American Heart Association Journal, – Professor Dame Anna Dominiczak, President of the Association of Physicians of Great Britain & Ireland, 2019-2020, and Regius Professor of Medicine, Vice Principal and Head of the College of Medical, Veterinary and Life Sciences at the University of Glasgow, Scotland.
Professor Dominiczak is editor-in-chief of Hypertension, a journal of the American Heart Association (AHA). Hypertension is leading scientific debate at scientific conferences, both in the traditional print journal, and by publishing all content online ahead of print. The journal also encourages clinical discussion at sponsored Clinical Pathological Conference sessions.
“In addition to publishing in the best hypertension journal in the world, publishing in Hypertension offers additional benefits. “We like to reward our authors, with the selection of three articles in each issue, 36 articles per year, that are further highlighted through inclusion in our popular Clinical Implications section,” Professor Dominiczak pointed out.
Nearly 20% of all original articles published have additional attention drawn to them through an invited editorial commentary. “Another four or five articles are selected each month to be featured on the journal cover,” she added.
The journal has undergone significant transformation to ensure an equal balance between clinical and basic science papers. “Coming to this job as a clinician-scientist, I wanted to change the journal to 50% clinical and 50% basic science and that is what we have today,” noted Professor Dominiczak.
AHA Journals are committed to publishing high-quality research and upholding accepted standards of methodological rigour, reproducibility, and transparency. Ongoing initiatives include methodological checklists for preclinical research, reporting guidelines for clinical studies and trials, Transparency and Openness Promotion (TOP) guidelines, and statistical & methodological review, when appropriate, remarked the professor.
These checklists are aimed at ensuring that appropriate details are provided in the following areas: study design, inclusion and exclusion criteria, randomisation, blinding, sample size and power calculations, data reporting, statistical methods, and experimental details, ethics, and funding statements.
Transparency and Openness Promotion (TOP) guidelines for authors publishing in an AHA Journal were adopted to help ensure that articles include enough information and access to original data, when appropriate, to allow others to replicate and build on work published in AHA journals.
“These guidelines include the requirements including that all data, program code, and other methods are appropriately cited; that authors must disclose the availability of all data and materials and provide accessibility information; and that authors should follow standards for disclosing key aspects of the research design and data analysis, including adherence to appropriate reporting guidelines,” Professor Dominiczak remarked.
“In a world of big data and even small data, we won’t allow you to hide them from the world and we will not publish your paper in such a case,” she added.
Turning to the blight of Western blot images in publications, she said that they seem to find quite a few of these errors. “We look for these because we’ve been burnt before.” She showed an example where manipulation of data had been carried out. “We have a salaried expert on Western blots to look at these. It is sad that the AHA has to do this, but some of these manipulations are so bad that I think people want to be discovered.”
Reflecting the comments of Professor Marshall from Diabetologia, Professor Dominiczak showed an example of a retraction notice.
Against this background of manipulation, fabrication and retraction, Professor Dominczak said she was naturally an optimist and wanted to finish on an upbeat note. “Altmetrics, the attention score, for our papers in Hypertension, shows amazing results. In July 2018 nearly half a million people accessed the full text of papers.”
The Modern Clinician Scientist – Professor David Newby, British Heart Foundation Duke of Edinburgh Chair of Cardiology, University of Edinburgh, Scotland.
Professor Newby took to the floor to discuss why clinician scientists should focus on experimental medicine. “Mice are okay, but I want to know what’s going on in people – humans are what I want to study,” he asserted.
Showing a photo of a paper he was recently involved in writing on air pollution and heart failure, Professor Newby pointed out the value of epidemiology and health records in understanding these associations. “But these associations are completely confounded. As a statistician once said to me, did you know that the stork population correlates perfectly with the birth rate in the Scottish population!” he joked, while making a serious point about the limitations of epidemiology. “If we want to prove causal association, we need to go back to the person. My interest in air pollution and cardiac problems, I want to know the mechanism, what causes this.”
He described some experimental medicine he is currently involved with together with a team of asthma physicians working in Northern Sweden. “The air is very clean there but the researchers have an experimental room into which passes diluted diesel exhaust from a tractor engine into an exposure chamber where volunteers cycle. Although this may seem extreme, the levels of pollution inside this chamber are similar to roadside levels found on a busy street in central London. We found that these acute exposures caused major abnormalities in blood vessel function including impaired relaxivity and release of tissue-plasminogen activator.”
Another clinician scientist is working on thrombus formation in Mount Sinai Healthcare System, New York, US. He found that when people are exposed to exhaust particles there was a lot more clot formation. “But if you add a particle trap to the air supply then you can get rid of the clot formation. It seems that it is the particles in the air that drive this,” reported Professor Newby.
Finally, Professor Newby referred to how the British Heart Foundation has conducted research into how fine and ultrafine particulate matter makes existing heart conditions worse and raises the chance of heart attack or stroke in vulnerable groups. Dr Mark Miller, from the University of Edinburgh, Scotland, used harmless gold nanoparticles to mimic the ultrafine particulate matter found in air pollution, similar to that found in diesel exhaust. After breathing in these nanoparticles for two hours, volunteers were tracked as the nanoparticles they progressed through the body, moving from lung to bloodstream. They were still identifiable after three months. The researchers also found that the nanoparticles accumulated in fatty plaques in the arteries. If similar sized particles from pollution behave in the same way then it is likely that they might contribute to heart disease, Professor Newby pointed out.
Funding of Clinician Scientists in the UK – a Talk by Dr Marjolein Schaap, Senior Funding Manager, Research Careers, from Cancer Research UK (CRUK)
Dr Schaap discussed how the charity, CRUK, supported clinician scientists. CRUK is the largest fundraising medical research charity in the world and has played a role in eight of the world’s top ten cancer drugs.
In 2017/2018, total annual spend at CRUK was £652 million. “We spend the majority of this money on research, which is why our researchers are hugely important to us,” Dr Schaap pointed out. “Of £652 million, £423 million went towards research including support for our CRUK Centres & Institutes, personal fellowships, and grants. This includes research into specific types of cancer as well as research into cancer biology, which underpins all types of cancer.”
She went on to explain that they spend £43 million on policy and information activities, including prevention and early diagnosis work; campaigning; communicating health messages; and engaging patients, the public and health professionals. A total of £97 million went on fundraising, including marketing to engage new supporters and innovation to develop new ways to raise money, and to make sure CRUK can continue to research in the future.
“Clinician scientists are critical to delivery of the CRUK strategy,” said Dr Schaap. “They provide a valuable bridge between research and clinical practice but unfortunately there is a substantial deficit of clinician scientists.
In May 2014, CRUK launched its research strategy which highlighted six priorities: early diagnosis research, precision medicine, therapeutic innovation, cancers of substantial unmet need, cancer prevention and basic understanding of cancer. To help deliver on our research priorities, CRUK is committed to attract, develop and retain the best clinical academics in research”
CRUK funding to support the careers of clinician scientists includes a pre-doctoral research bursary, a UK-wide clinical academic training (CAT) programme for clinical trainees, a post-doctoral research bursary for early stage post-doctoral researchers, personal (clinician scientist) fellowships for experienced post-doctoral researchers and junior group leaders, through to university salary supported positions with a career establishment award and a programme foundation award for lecturers and readers respectively.
In April this year, CRUK introduced the CAT programme that is focused on attracting and retaining clinical trainees in academia post-PhD. It aims to incorporate a mix of attractive support options including, flexible training options, mentorship, integrated support, and a UK-wide clinical academic network. The programme will support Clinical Research Training Fellowships, MB-PhDs and programme related activities to provide holistic support before, during after training to attract and retain clinical trainees in academia post-PhD.
As part of CRUK’s commitment to promote equality, diversity and inclusion in research, they have removed the post-PhD time restrictions on all their response-mode fellowship schemes and introduced a new competency framework that outlines the skills and experience researchers will need to demonstrate at different stages in their career. With the framework, CRUK aims to provide greater flexibility for applicants to apply at the right time in their career; clarity and transparency in terms of the skills and experience that are needed at different stages in their career; a holistic approach towards the assessment of candidates; and being consistent with other major funders, which will allow applicants to compare funding options more easily.
These contributions were only selected features of a well attended session that stimulated many questions and lively discussion. As well as providing a forum to air key issues, debates and evolving best practice in the arena of research integrity, the session emphasised the appetite of early career clinical researchers to engage in this more informal setting with senior researchers. It is hoped not only that attendees this year will draw on the workshop to help expedite their publication of high quality research in good journals, but also that they will return to future workshops, which have become a fixture of the AoP annual meeting. Indeed, with the newly diversified membership structure meaning that ECRs are now welcomed as members, it is hoped and expected that ECR members themselves will increasingly play a role in shaping future workshop programmes. Onwards to Oxford 2020!