Essential Publication Metrics

Hi Reader, we commonly communicate our science by publishing it.

But to know where to publish, we often look at the impact factor of a journal.

Also, whether you get a good postdoc position or become a professor partially depends on your publication record or h-index.

Still, what are these metrics, and what do they measure?

How Do We Measure Reach?

In the past few years, publication pressure has significantly shaped science.

Just consider that publishers such as Wiley, Elsevier, and Nature have estimated annual revenues of more than a billion dollars each.

But how do we measure the success of our publications? Let’s review some of the most common metrics and understand how they come together:

Metrics to Assess Journals

Despite all its shortcomings, we most commonly use citations as the main indicator to assess the importance or impact of scientific work.

And the following numbers are there to give you an idea of the impact, prestige, or importance of a journal.

Impact Factor (IF)

The total number of citations, divided by the total number of papers published in the previous two years.

• Widely recognized
• Can be skewed by a few highly cited papers (that is not infrequently the case)
• Doesn’t provide information about the citations of a specific paper

Indeed, many scientists estimate the impact of their work based on the IF of the journal in which it's published in.

However, a journal impact factor (JIF) provides information about the average success of a journal, not any single piece of work published there, as citation counts often vary widely.

CiteScore

The average number of citations received by a journal for all documents published over the previous four years.

Includes several document types: Papers, Reviews, Conference papers, Book chapters, Editorials, Letters, Errata/corrections

• Published by Elsevier (Scopus)
• Potentially punishes outlets with many “non-paper” publications that are less commonly cited (such as Nature or Lancet)

SCImago Journal Rank (SJR)

The SCImago Journal Rank (SJR) indicator is calculated by dividing the total weighted citations a journal receives by the number of citable publications during the previous three years.

Weighted means that influence is quantified through the prestige of the citing journals, as determined by the SJR algorithm; citations from highly ranked journals contribute more.

• Less sensitive to citation manipulation (e.g., self-citation)
• Technically comparatively complex to calculate

Eigenfactor Score

Eigenfactor estimates how important a journal is within the whole citation network by weighting citations according to the prestige of the citing sources based on the previous 5 years.

> Eigen factor is therefore similar to the SJR but the latter divides by the number of articles and is therefore size-independent.

Eigen factor is size-dependent (big journals score higher because they publish many papers, so they accumulate more total citations even if their average paper is only moderately cited.)

• Derived from Eigenfactor is the Article Influence Score measuring the average influence per article (similar to SJR)
• Excludes journal self-citations
• Technically complex to calculate

Article-Level Metrics (Altmetrics)

Online attention to individual papers.

Includes: Social media mentions, News coverage, Policy citations, Downloads

• Common indicator: Altmetric Attention Score

Important to Note

We couldn’t dive into all the details and there are still more metrics, such as the Source Normalized Impact per Paper (SNIP), which indicates the journal citation impact normalized for field citation practices, allowing for cross-field journal comparisons.

However, those metrics are mostly used by librarians and institutions to some extent because metrics are strongly field-dependent.

Indeed, many "low scoring" journals have a quality and visibility problem. However, please remember: high metrics don’t necessarily mean the best choice.

Publishing in renowned journals often goes along with much longer editorial processes, strict selection criteria, and higher APCs for open access pieces.

Metrics to Assess Individuals

The following metrics should summarize the impact of a single scientist’s work.

Although they are not as widely discussed, they are, for example, crucial for securing more senior positions in academia:

h-Index (H factor)

Wikipedia states: “The h-index is defined as the maximum value of h such that the given author (or journal) has published at least h papers that have each been cited at least h times.”

Honestly, h-indices are intuitive but hard to put into words.

In essence, a researcher has an index h if they have h papers cited at least h times each.

• Resistant to outliers but ignores very highly cited papers.
• Favors senior researchers with more publications.
• Does not account for author position; first author, last author, and middle author receive the same credit.

g-index

Largest number g such that the top g papers (if one rank-orders papers according to number of citations) have at least g² citations combined.

Hence, a g-index of 10 indicates that the top 10 publications of an author have been cited at least 100 times.

A g-index of 20 indicates that the top 20 publications of an author have been cited 400 times. One basically searches for the highest g based on all publications of an author.

Field-Weighted Citation Impact (FWCI)

FWCI compares how often a researcher’s publications are cited relative to the world average for similar publications.

> FWCI = Actual citations received divided by expected citations for similar publications.

“Similar publications” means same field/subject area, publication year, document type (article, review, etc.)

You do this for each paper and then average all individual FWCIs into one number.

• Can be skewed by one blockbuster paper if publication numbers are small
• Computing the benchmark (the average citations for all comparable papers in the database) is technically laborious

m-index (also called m-quotient)

Equals h-index divided by the years since first publication.

As the h-index favors senior researchers who have had more time to accumulate citations, the m-index corrects for this by measuring average annual impact

i10-index

Number of publications with ≥ 10 citations.

• Provider: Google Scholar.

Citation Percentiles / Top-X% Papers

Essentially provides the % of papers in the top 10% most cited

• Many systems (especially Clarivate and Elsevier analytics) report this metric.

Total Citations

Sum of all citations to a researcher’s work

Important to Note

Once again, we could also talk about Mean Normalized Citation Scores (MNCS), v-indices, or the Relative Citation Ratio (RCR), which is used in the U.S. NIH ecosystem and compares a paper’s citation rate to its co-citation network (papers cited alongside it).

With all of these metrics, we have to consider that we often get a summary of an author’s work.

Especially for junior scientists with fewer than 10 publications, many metrics such as FWCI are not stable.

That means year-to-year changes can be dramatic, also because their work is often published more recently. One normally needs a body of 30–50 publications for a more robust assessment.

I intentionally did not include average or expected values for these metrics.

On the one hand, they differ by field, decade, and expectations. On the other hand, they can be misleading, since publications and citations often come unevenly.

Key Take-Aways

Several metrics are used to evaluate journals and scientists, but some are cited disproportionately often.

Each serves a different purpose and is based on specific assumptions about what matters to assess (e.g., timeframes).

There is also not a single established metric I know of that would try to estimate translatability or innovativeness of your work.

However, it is important to have a general understanding of how these metrics function in order to prepare yourself if you plan to pursue an academic career.

How We Feel Today