18 Sep 2011 6 Comments
Author: Tracey Brown
Fact-finding Mission to China Provides Key Insights
"In China, the research community is gaining year on year in resources and ability. That is very exciting to be around.” — Tracey Brown, Managing Director, Sense About Science
Tracey Brown believes peer review is vital to good science and the society that uses it.
And it’s a conviction the Managing Director of Sense About Science shares with members of the Chinese Academy of Sciences, as she discovered during a trip to the research-rich country in March this year.
Brown embarked on the fact-finding mission with two key aims in mind; she was keen to test out views advanced about the integration of Chinese authors and reviewers into international STM publishing, and to explore future collaborations to help researchers, policy makers and journalists identify the best science.
During the two-week visit, which was supported by Elsevier, she met not only the Chinese Academy of Sciences (CAS), but Science.net, journalists, post docs and publishers.
Brown says: “It was clear that CAS is keen to discuss the best ways to evaluate research and to explore their concerns about what peer-reviewed publishing can - and can't - deliver. In an effort to avoid cronyism and subjective assessment in China, there has been a shift towards using flatter measurements; for example, the Impact Factor. There is a feeling, however, that these do not reveal enough about individual papers or the research output of an institution. Most people, including CAS, are coming to the conclusion that what we really need is a mix of the two.”
Gaining new understanding
Asked to highlight some of her key learnings during the trip, Brown says:
“People raised many interesting points and some quite contradictory ones. The early career researchers I spoke with viewed international journals as motivated by quality and fairness, and in some cases compared them favourably with Chinese journals, which can be seen as wedded to the relationships and prestige of individuals and institutions.
“On the other hand, some of the more editorially-experienced people had stories of less than positive attitudes among international editors to Chinese papers. They were concerned about a head-in-the-sand approach to such a major research base and that valuable new insights could be missed.”
- General agreement that reviewing is an important part of the role of a researcher. However, involvement in it varies enormously.
- As in many other countries, a researcher's day is structured in a way that makes it difficult to find time to review and their career progresses in response to grants and publications, not time spent reviewing. Views differed widely about the problems this posed and whether it inhibited the reviewing effort.
- A strong interest in training, both for authors and reviewers.
- Interest in other metrics for evaluating research output, the respective contributions of regions/countries and the performance of individual institutions.
There were also a few eye-opening moments for Brown.
She explains: “I had not expected people’s personal experiences to differ so widely. For example, I was speaking to two post doc students at Shanghai Jiao Tong University. Both had published very successfully early in their careers in some of the top journals - the elite of the elite. One was receiving almost weekly requests to review while the other had received only one request in a year. That may reflect the different nature of their papers but I heard their stories repeated elsewhere. It is perhaps to be expected that peer-review requests from international journals are still a bit of a hit and miss process in China.”
She adds: “Each time I was about to draw a conclusion about anything I would meet someone who took me in a different direction – a symptom, I imagine, of things being a work in progress there.
“Another surprising thing for me was the high level of confidence in the research community in contrast to the UK, and perhaps the US, where universities face straitened circumstances. In China, the research community is gaining year on year in resources and ability. That is very exciting to be around.”
The pressure to publish
Commenting on the quality – and quantity – of papers submitted by Chinese researchers, Brown says: “There is some concern, internationally, about filtering the sheer weight of papers produced by China. A big sea of papers makes it difficult to pick out the best.
“The thing is, there is a large pressure to publish in China and doing so in international journals brings career breaks and prestige. While lead institutions no longer pay incentives for this, some second-tier universities still appear to, which may contribute to journals being overwhelmed by unsuitable papers.
“We discovered that inappropriate submissions also stem from a lack of local knowledge about international journals, with younger researchers copying where their supervisors have published. Library services can play a very important role in widening the pool of journals considered.”
She adds: “Since returning I have been in touch with members of the Publishing Research Consortium to discuss the prospect of looking at how these new regions, such as China and India, are being integrated. Do editors now need something different from publishers with regard to support and advice? These are questions I know publishers are asking too. There is clearly some opportunity for international publishers to improve the availability of information about how to publish and where to publish, probably via librarians in those institutions where library services are developing and pro-active.”
Looking to the future
And what does Brown think the next five years will hold for the Chinese research community?
“Because of the volume of research and population size, even minority behaviors in China are likely to have a significant effect. If just a proportion of the new generation of researchers are trained and engaged with reviewing, it could have a big impact on sharing the reviewing burden. I know that there are already programs underway, such as Elsevier’s Reviewer Workshops and Reviewer Mentorship Program. The value of their contribution to the research output cannot be overstated – just like so many other things in China at the moment!”
What is Sense About Science?
Sense About Science is a UK charitable trust that equips people to make sense of science and evidence on issues that matter to society. With a network of more than 4,000 scientists, the organization works with scientific bodies, research publishers, policy makers, the public and the media, to lead public discussions about science and evidence. Through award-winning public campaigns, it shares the tools of scientific thinking and the peer-review process. Sense About Science’s growing Voice of Young Science network engages hundreds of early career researchers in public debates about science. Sense About Science will be publishing a Chinese edition of its public guide to peer review I Don’t Know What to Believe early in 2012 in collaboration with learned societies, patient groups and journalists.
MANAGING DIRECTOR OF SENSE ABOUT SCIENCE
Tracey has been the Director of Sense About Science since shortly after it was established in 2002. Tracey is a trustee of Centre of the Cell and MATTER. In 2009 she became a commissioner for the UK Drugs Policy Commission. She sits on the Outreach Committee of the Royal College of Pathologists and in 2009 was made a Friend of the College.