With the increasingly interconnected internet and developments in ‘big data’ analysis, there are now many ways available to measure research impact.
Traditional bibliometrics may be supplemented by usage data (pageviews and downloads), while the success of online communities and tools have led to more widespread visibility for learned commentary, readership statistics, and other measures of online attention. Altmetrics encompass social activity in the form of mentions on social media sites, scholarly activity in online libraries and reference managers and scholarly commentary, for instance, through scientific blogs and mass media references.
Elsevier has long been an advocate for robust informetrics and we are particularly interested in understanding how these new measures can be used in conjunction with usage and citation data, to provide new meaningful indicators to the research community. Mendeley statistics, for instance, appear to provide more insights into the academic use of a paper than Twitter.
Altmetrics, (the measurement of scholarly activity on social networks and tools), is a major buzzword at the moment, and although it is a very new discipline, interest in it is growing fast, as demonstrated by the relative search volumes in the graph below.
The results of a search on Google.com for “altmetrics” (red line) against “bibliometrics” (blue line).
It’s not only general interest that is growing – scholarly interest appears to show high growth too, as demonstrated by a simple keyword search on Scopus for altmetric* or alt-metric* in the title, abstract, or keywords of any paper, the results of which are visible in the graph below.
The first two papers were Proceedings of the ASIST Annual Meeting in 2011; in 2012 the number of papers published jumped to eight, and in 2013 to 28 (data for 2013 may still be incomplete due to publication and indexation delays). Even though in absolute terms these are still low numbers, this is quite tremendous growth.
Altmetrics offer an alternative insight into the use and readership of scholarly articles, and this information has driven authors, researchers, editors, and publishers to try to understand the data. To this effect, Elsevier employees are engaged on the NISO Altmetrics project, have spoken at conferences around the world (e.g. altmetrics12, ALM workshop 2013), have written academic papers, and have conducted webinars in this multi-faceted field.
The last year has seen us launch several initiatives - three of which are explored in further detail below: these are pilots on Elsevier.com journal homepages and ScienceDirect and an existing Scopus project. Joining these is an article usage alert program informing authors in participating journals how their article is being viewed. Mendeley data continues to provide an invaluable and free source of data on the discipline, location, and status of researchers.
Elsevier has also formed partnerships with altmetrics start-ups:
- We use the Altmetric.com explorer product to understand and analyse trends as well as inform some of our marketing campaigns.
- We have joined a pilot to investigate how interested Elsevier authors are in showcasing their research via Kudos (further details of this pilot will be available in a ‘promoting your journal’ special issue of Editors’ Update due to be published in May).
This year will see even more metrics activity. With a special altmetrics issue of Research Trends planned, more products in the pipeline, and our partnerships beginning to produce solid results, we are continuing to actively support research in this fascinating field.
Altmetric.com pilot on Elsevier.com journal homepages
At the end of last year, we began displaying the Altmetric.com colorful donuts for a journal's top three rated articles on the Elsevier.com homepages of 33 Elsevier titles.
This rating is based on a social media traffic score given by Altmetric.com and an article must have received at least one social media mention within the last six months to qualify. By clicking on the "view all" option beneath the top three list, visitors can review the donuts for the top 10 articles. In both lists, the article name links to the full-text article on ScienceDirect, while the donut links to a breakdown of the news and social media mentions.
The pilot is led by Hans Zijlstra, Project Manager for Elsevier's STM Journals Project Management department. He worked closely with Elsevier's e-marketing team in cooperation with Altmetric.com — a company founded by Euan Adie (@Stew), who won Elsevier's Apps for Science Challenge in 2011.
Although it is still early days, Zijlstra will be closely monitoring how much traffic the donuts receive over the coming months. Based on those results and the feedback he receives, the aim is to make this available to all Elsevier journals.
He said: "These additional article metrics are intended to provide authors with extra insight into the various flavors of impact their research may achieve. We believe altmetrics will help them select a journal for article submission by giving a clearer indication of where a journal's strengths and weaknesses lie."
The donuts have also provided useful insights to publishers and editors. He explained: “They help publishers and marketeers determine which media they should engage with more often and publishers and editors can identify hot topics that might merit a special issue.”
Zijlstra and his colleagues are still working on adding to the journal homepage the names of the authors for the top ranked articles. In addition, they plan to include the donuts for participating health and medical titles on their homepages on the Health Advance platform.
Altmetric.com’s colorful donut explained
The Altmetric.com algorithm computes an overall score taking into account the volume, source and author of the mentions a paper receives. This includes mentions of academic papers on social media sites (e.g. Twitter, Facebook, Pinterest, Google+), science blogs, many mainstream media outlets (including The New York Times, The Guardian, non-English language publications like Die Zeit and Le Monde and special interest publications like Scientific American and New Scientist), peer-review site Publons, and reference managers.
News items are weighted more than blogs, and blogs are weighted more than tweets. The algorithm also factors in the authoritativeness of the authors, so a mention by an expert in the field is worth more than a mention by a lay person. The visual representation – the Altmetric.com donut – shows the proportional distribution of mentions by source type. Each source type displays a different color – blue for Twitter, yellow for blogs, and red for mainstream media sources. Links to the source data are also available. Altmetric.com tracks around a hundred thousand mentions a week, with some 3,000 new articles seen each day.
Altmetric.com pilot on ScienceDirect
Elsevier’s ScienceDirect platform, home to one-quarter of the world’s STM journal and book content, launched a six-month altmetrics pilot in December 2013.
Until June this year, 26 journals – including The Lancet – will display alternating altmetrics images on an article level. Visitors landing on the relevant pages have a 50 percent chance of seeing either the traditional Altmetrics.com donut or the information presented in a bar chart form. This reflects ScienceDirect’s AB testing approach – the results will be monitored to discover which design is the most engaging and clear for users. The pilot also includes sharing buttons to promote social media mentions of the covered articles and will provide access to the individual article detail pages, which enables users to explore the actual mentions of the paper.
During the pilot we will be assessing the popularity of the altmetrics score with users. We will also be trying to determine how far the scores promote use of the article sharing buttons.
Todd Vaccaro, ScienceDirect Product Manager, commented: “The journals chosen for the pilot represent a good mix. We’ve included titles with a range of Impact Factors, types of attention on the social web, average altmetrics scores and subjects. We’ve also included some society journals and a recent OA title.”
Example of the pilot bar chart altmetrics image on the same ScienceDirect article from Journal of Adolescent Health.
Altmetric.com donuts in Scopus
Since June 2012, Elsevier’s Scopus – the largest abstract and citation database of peer-reviewed literature – has offered the Altmetric.com donut in the sidebar of document and abstract pages. It can be found on the right hand side of the screen when data is available for the article being viewed. Visitors can click through to scan the content mentioning the article and click on any entry to navigate to the original site. A “demographics” tab will also show a breakdown of where in the world the attention paid to the article is coming from.
Michael Habib, Senior Product Manager for Scopus, said: “Customers and users alike have found this a useful supplement to traditional citations. The primary point of interest hasn’t necessarily been the metrics themselves, but the underlying content. Discovering that a respected science blogger has given a positive review of your article is much more important than knowing how many people have blogged about it. This pilot has proven a powerful tool for uncovering these previously hidden citations from non-scholarly articles.”
Example of an Altmetric.com donut on a Scopus article from the journal Computers in Human Behavior.
Useful Altmetric.com tools
- For further details on the social media reports, and to see the score for any article containing a DOI, download the Altmetric.com Bookmarklet.
- The newly released Altmetric.com widget generator, built for bloggers, will pull in relevant articles and their related altmetrics score based on your chosen criteria.
SENIOR PUBLISHING INFORMATION MANAGER
As part of the Scientometrics & Market Analysis team in Research & Academic Relations at Elsevier, Sarah Huggett provides strategic and tactical insights to colleagues and publishing partners, and strives to inform the research evaluation debate through various internal and external discussions. Her specific interests are in communication and the use of alternative metrics for evaluating impact. After completing an M.Phil in English Literature at the University of Grenoble (France), including one year at the University of Reading (UK) through the Erasmus programme, she moved to the UK to teach French at University of Oxford. She joined Elsevier in 2006 and the Research Trends editorial board in 2009.
TECHNOLOGY RESEARCH SPECIALIST
Based in Oxford, Mike Taylor has worked at Elsevier for 18 years, the past four as a technology research specialist for the Elsevier Labs group. In that role, he has been involved with the ORCID Registry. His other research interests include altmetrics, contributorship and author networks. Details of his research work can be found on the Elsevier Labs website. He is currently producing a series of three plays about scientists for Oxford-based theater company www.111theatre.co.uk.
Taylor is also one of the organizers of Altmetris 14, a workshop at the ACM Web Science conference in Indiana on June 23, 2014. Submissions are welcome.