Article Level Metrics
The most commonly used metrics in measuring article impact are citation metrics such as Times Cited and other field normalization metrics. In recent years, Altmetrics are also frequently used to measure the immediate impact of the scholarly publication received in areas other than the scholarly publishing community like social media.
In this guide, the below common article-level metrics will be introduced:
1. Times Cited & Statistical Analysis
2. Field-Weighted Citation Impact (FWCI)
3. Category Normalized Citation Impact (CNCI)
4. Altmetrics
Service Counter
WhatsApp: 5441-5441
Research Consultation (HKU academic staff and postgraduate students only)
Times Cited
Times Cited refers to citation counts or number of citations an article received from other sources available within the database. It may vary across different databases due to the difference in collection coverage between databases. This metric could be seen on various platforms and the most popular ones are the Web of Science and Scopus.
Hot & Highly Cited Papers
Hot Papers and Highly Cited Papers are the two major indicators in the Clarivate Essential Science Indicators (ESI) indicating papers with significant impacts. Hot Papers refer to the number of hot papers for an entity (author, institution, country, and journal) divided by the total number of documents produced by the given research entity times 100. % Hot Papers is the percentage of publications regarded as Hot Papers, meaning the top 0.1% by citations for field and age.
Highly cited papers are the top one percent in each of the 22 ESI subject areas per year, based on the publications in the most recent 10 years. % Highly Cited Papers is a percentage of the number of ESI highly cited papers for an entity (paper, author, institution, country, journal and field) divided by the total number of documents produced by the given entity. This measure is commonly used to show the percentage of an institution's output is among the most impactful papers in the world.
Both the Hot Papers and Highly Cited Papers will be updated in ESI on a bi-monthly basis.
Source: Clarivate InCites Help, Essential Science Indicators (Sep 2021)
Retrieving Hot & Highly Cited Papers on Essential Science Indicators (ESI)
Step 1:
Access Incites Essential Science indicator via Find@HKUL.
Step 2:
Select the options and apply filters for your search results. You can select categories like research fields, authors, institutions, journals, countries/regions or research front in the list and filters. In this example, we are looking for the hot papers in the social science field.
Step 3:
By clicking the “hot papers” column, you will be able to view a list of results which are marked “ESI hot”. You can also select “Highly Cited Papers” in the filter panel to search for highly cited papers.
% of Document Cited / in Top 1% & 10% / in Top Journal Percentiles
Further to Times Cited, more statistical analyses are available for measuring article-level impact in terms of other aspects:
% of Document Cited: the percentage of publications in a set that have received at least one citation.
% of Document in Top 1% & 10%: the percentage of publication in the top 1% and 10% based on citations by category, year, and document type. This indicator shows the top 1 or 10 percent most cited documents in a given field, year and publication type divided by the total number of documents in a given set of documents.
It aims to show the extent of publications which have attained notable distinction and impact, based on the notion that only those most highly cited papers could be ranked in the top 1 or 10 percent in their respective field. It is useful for indicating research excellence and the indicator could also be applied to any level of aggregation, such as author and institution.
% of Document in Top Journal Percentiles: In InCites, it is shown as % Documents in Q1 - Q4 Journals. It is counted on the basis of the Journal Impact Factor quartile, which is the quotient of a journal’s rank in category (X) and the total number of journals in the category (Y), so that (X / Y) = Percentile Rank Z. Please refer to the JIF Quartile section for how a journal is ranked in which quartile is determined on InCites.
% Documents in Q1 - Q4 Journals refers to the percentage of documents that appear in a journal in a particular Journal Impact Factor Quartile in a given year. For example, if a value displays 10%, it indicates that 10% of the documents in the set were published in journals of the specified Journal Impact Factor Quartile in that year.
The calculation is as follows:
% of documents in Q1 Journals = (Count of Documents in Q1 Journals) / (Count of Documents in JIF Journals)
Source: Clarivate InCites Help, Impact Indicators (Sep 2021); Clarivate InCites Help, Journal Citation Reports Data (Sep 2021)
Retrieving % of Document Cited / in Top 1% & 10% / in Top Journal Percentiles on InCites
Sorted by Research Areas/Fields
Step 1:
Access Incites via HKUL. You are required to apply for a personal account within the HKU campus IP.
Step 2:
After logging in, click on “Analyze” to open a drop-down list and select “Research Areas”.
Step 3:
On the right of the screen, click on “Add indicator”. Tick the box “% Documents cited” and click “Apply”. You will be able to see a new column “% Documents cited” appear in the result table.
Similarly, for finding the “%Documents in Top 1% or 10%, or Top Journal Percentiles”, you may apply relevant indicators as follows:
Field-Weighted Citation Impact (FWCI)
Field-Weighted Citation Impact is a field-normalized metric measuring the performance of an article or publication using the Scopus data, which considers the differences in research behaviour across disciplines, enabling comparison between different fields. FWCI is the ratio of the total citations actually received by an article and the total citations that would be expected based on the average of the subject field.
A Field-Weighted Citation Impact of:
Exactly 1 (FWCI = 1) means that the output performs just as expected for the global average.
More than 1 (FWCI > 1) means that the output is more cited than expected according to the global average. For example, 1.48 means 48% more cited than the global average.
Less than 1 (FWCI < 1) means that the output is cited less than expected according to the global average.
Source: What is Field weighted Citation Impact (FWCI)? (Sep 2021)
Retrieving FWCI on Scopus
Step 1:
Access Scopus via HKUL.
Step 2:
Click on “Search”, under “Documents”, select “Article title”, then enter the full article title or keywords and click “Search”.
Step 3:
Click into the title of the article. You will be able to see the “Times Cited” under “Citations in Scopus” and the FWCI score on the right panel.
Click on "View all metrics".
Category Normalized Citation Impact (CNCI)
Similar to the FWCI used on Scopus, on InCites, it offers a field-normalized metric named the Category Normalized Citation Impact (CNCI) benchmarking the impact of an article in a particular subject area.
The Category Normalized Citation Impact (CNCI) of a document is calculated by dividing the actual count of citing items by the expected citation rate for documents with the same document type, year of publication and subject area. The CNCI of a set of documents, for example the collected works of an individual, institution or country/region, is the average of the CNCI values for all the documents in the set.
A CNCI value of:
1 represents performance at par with world average.
above 1 is considered above average.
below 1 is considered below average.
For instance, a CNCI value of 2 represents the article is considered twice world average. Details of the calculation formulas could be referred to: https://incites.help.clarivate.com/Content/Indicators-Handbook/ih-normalized-indicators.htm
CNCIs of articles published in a journal is also used in calculating the Journal Citation Indicator (JCI) which indicates the performance of a journal.
Source: Clarivate InCites Help, Normalized Indicators (Sep 2021)
Retrieving CNCI on InCites
Step 1:
Access Incites via HKUL. You are required to apply for a personal account within the HKU campus IP.
Step 2:
After logging in, click on “Analyze” to open a drop-down list and select “Publication Sources”.
Step 3:
On the right of the screen, click on “Add indicator”, tick the box “Category Normalized Citation Impact” and click on “Apply”.
To check the documents published by the publication sources, you can click the bold numbers in the result list.
A list of articles will be displayed as below. Scroll to the right and you will see the CNCI of the article.
Altmetrics
Different from the other citation metrics that are calculated within the scholarly publication context, Altmetrics (Alternative metrics) is a measure to indicate scholarly impact based on activities in online tools and environments. It has become commonly used as online platforms and tools such as social media, online reference managers, scholarly blogs, and online repositories are deeply embedded into the system of scholarly communication. It is often used to indicate the immediate impact of a piece of work and serve as a reference for possible intentions of citations. The most popular used tools for Altmetrics are PlumX and Altmetric.com.
Concerns
There are concerns on the accuracy and reliance of this newly emerged indicator of research impact:
As Altmetrics takes the number of tweets and shares on social media platforms into account, it could be gamed by dishonest use of the system such as “buying” likes or tweets.
There is a lack of theory on how and why Altmetrics are generated.
The Altmetrics score has to be interpreted cautiously as a high score may not necessarily mean that the paper is especially good, but just on a controversial or fashionable topic.
It may under-represent olders papers published in the past as social media is relatively new.
Source: Mingers , J., & Leydesdorff , L. (2015). A review of theory and practice in scientometrics . European Journal of Operational Research, 246(1), 1 19. https://doi.org/10.1016/j.ejor.2015.04.002
PlumX
PlumX Metrics provide insights into the ways people interact with individual pieces of research output (articles, conference proceedings, book chapters, and many more) in the online environment. Metrics are categorized into 5 separate categories: Citations, Usage, Captures, Mentions, and Social Media.
Citations - includes both traditional citation indexes and citations indicating societal impact.
Usage - includes activities of reading and otherwise using the research, such as counts of clicks, downloads, views, video plays, etc.
Captures - includes activities indicating someone might want to come back to the work, including counts of being bookmarked, added into favourites, etc.
Mentions - includes activities such as news articles or blog posts about research. It could be in comments, blog posts, reviews, news media, etc.
Social media - includes all activities on social media platforms like tweets on Twitter, likes and shares on Facebook, etc.
Source: About PlumX Metrics (Sep 2021)
PlumX on Scopus
Step 1:
Access the record of the article on Scopus.
Step 2:
Click on "View all metrics" and select the “View PlumX details” and you are able to see more details in each category:
Altmetric.com
Altmetric.com provides an indicator of the volume and type of attention a research output has received. An Altmetric Attention Score is presented in the centre of the “Altmetric donut”. The colors of the Altmetric donut each represent a different source of attention.
The Altmetric Attention Score is an automatically calculated, weighted count of all of the attention a research output has received. It is based on 3 main factors: Volume, Sources, and Authors.
Volume: It counts when the paper or publication is being mentioned. Only 1 mention from each person per source is counted.
Sources: Standard weightings are applied to the source of mentions. For example, a newspaper article weighs more than a blog post which is weighted higher than a tweet.
Authors: Weightings also apply to the author of each mention. The weighting aims to avoid bias towards a particular journal or publisher, and to consider who the audience is.
Source: The donut and Altmetric Attention Score (2021)
Altmetric.com on HKU Scholars Hub and DataHub
HKU Scholars Hub
Step 1:
Access HKU Scholar Hub and click “Publication” on the top menu bar. Fill in the details of the article and click “Search”.
Step 2:
Click on the title of the article.
Step 3:
On the article record, under the “Supplementary” section on the left panel, you will be able to find the Altmetrics information.
Step 4:
Click on the altmetrics information and you will be directed to the record page of the article on Altmetric.com for more details.
DataHub
Step 1:
Go to DataHub (https://datahub.hku.hk/).
Step 2:
Click into a dataset record and on the right panel under “usage metrics”, you will be able to see the Altmetrics details. Clicking on “see more details” and you will be directed to the record page for this dataset record on Altmetric.com.
Scite_
Scite is a tool that offers a quantitative and qualitative insight into how scientific publications cite each other. Beyond merely the number of citations a paper has received, Scite uses access to full-text articles and its deep learning model to tell if the citation is supporting, contrasting or simply mentioning the paper. It could also show the textual context of each citation and the location within the citing paper. For example, you may refer to the below images for meanings of the symbols:
For more details on the features and use of scite, please refer to our Scite LibGuide.
How to use Scite_
Step 1:
Access Scite (http://find.lib.hku.hk/record=HKU_IZ61623882910003414) via HKUL. You are required to create a personal account on scite_ for viewing the citation statements.
Step 2:
Select “Paper”, enter the title of an article in the search bar to perform a search. Click on the title of the article.
Step 3:
On the article record, you will find the counts of citation types on the left menu and read the citation statements of each citation.
