↓ Skip to main content

Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology

Overview of attention for article published in Scientometrics, August 2012
Altmetric Badge

About this Attention Score

  • In the top 25% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (82nd percentile)
  • Above-average Attention Score compared to outputs of the same age and source (63rd percentile)

Mentioned by

twitter
12 tweeters
facebook
1 Facebook page

Citations

dimensions_citation
13 Dimensions

Readers on

mendeley
70 Mendeley
Title
Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology
Published in
Scientometrics, August 2012
DOI 10.1007/s11192-012-0822-6
Pubmed ID
Authors

Christopher J. Lortie, Lonnie W. Aarssen, Amber E. Budden, Roosa Leimu

Abstract

Metrics of success or impact in academia may do more harm than good. To explore the value of citations, the reported efficacy of treatments in ecology and evolution from close to 1,500 publications was examined. If citation behavior is rationale, i.e. studies that successfully applied a treatment and detected greater biological effects are cited more frequently, then we predict that larger effect sizes increases study relative citation rates. This prediction was not supported. Citations are likely thus a poor proxy for the quantitative merit of a given treatment in ecology and evolutionary biology-unlike evidence-based medicine wherein the success of a drug or treatment on human health is one of the critical attributes. Impact factor of the journal is a broader metric, as one would expect, but it also unrelated to the mean effect sizes for the respective populations of publications. The interpretation by the authors of the treatment effects within each study differed depending on whether the hypothesis was supported or rejected. Significantly larger effect sizes were associated with rejection of a hypothesis. This suggests that only the most rigorous studies reporting negative results are published or that authors set a higher burden of proof in rejecting a hypothesis. The former is likely true to a major extent since only 29 % of the studies rejected the hypotheses tested. These findings indicate that the use of citations to identify important papers in this specific discipline-at least in terms of designing a new experiment or contrasting treatments-is of limited value.

Twitter Demographics

The data shown below were collected from the profiles of 12 tweeters who shared this research output. Click here to find out more about how the information was compiled.

Mendeley readers

The data shown below were compiled from readership statistics for 70 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Brazil 6 9%
United States 5 7%
Spain 2 3%
Canada 2 3%
Russia 2 3%
Malaysia 1 1%
Hungary 1 1%
Sweden 1 1%
United Kingdom 1 1%
Other 0 0%
Unknown 49 70%

Demographic breakdown

Readers by professional status Count As %
Researcher 13 19%
Student > Master 12 17%
Professor 8 11%
Student > Ph. D. Student 7 10%
Professor > Associate Professor 6 9%
Other 24 34%
Readers by discipline Count As %
Agricultural and Biological Sciences 30 43%
Environmental Science 9 13%
Social Sciences 9 13%
Computer Science 8 11%
Medicine and Dentistry 3 4%
Other 11 16%

Attention Score in Context

This research output has an Altmetric Attention Score of 7. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 27 September 2013.
All research outputs
#2,257,176
of 12,972,771 outputs
Outputs from Scientometrics
#474
of 1,630 outputs
Outputs of similar age
#21,875
of 128,961 outputs
Outputs of similar age from Scientometrics
#4
of 11 outputs
Altmetric has tracked 12,972,771 research outputs across all sources so far. Compared to these this one has done well and is in the 82nd percentile: it's in the top 25% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 1,630 research outputs from this source. They typically receive more attention than average, with a mean Attention Score of 9.1. This one has gotten more attention than average, scoring higher than 70% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 128,961 tracked outputs that were published within six weeks on either side of this one in any source. This one has done well, scoring higher than 82% of its contemporaries.
We're also able to compare this research output to 11 others from the same source and published within six weeks on either side of this one. This one has gotten more attention than average, scoring higher than 63% of its contemporaries.