Diversity, entropy and the effective number of species

note

This was originally a short detour in something longer I am working on. I decided to turn it into a standalone short explainer I can point to.

One of the metrics that we use to quantify the diversity of an ecological community is Shannon’s entropy. In contrast to “richness”, i.e. the number of species that comprise the community, entropy is determined not only by how many species are there but also their relative abundance. It is calculated as

\[ H' = -\sum_i p_i log(p_i) \]

where \(p_i\) is the relative abundance of species \(i\)¹. So, if we have an “even” community of four species [🦑, 🐟, 🦀, 🐚] equally abundant, its entropy is:

-((1/4)*log(1/4))*4

[1] 1.386294

There are two ways for entropy to go down. If all 🦑 individuals suddenly vanished², the new entropy would be:

-((1/3)*log(1/3))*3

[1] 1.098612

The other way for entropy to drop is if the community became less even, changing for example from \([.25, .25, .25, .25]\) to \([.4, .2, .2, .2]\). Here’s the new entropy:

-((.4)*log(.4)) - ((.2)*log(.2))*3

[1] 1.332179

The number of species determines an upper limit to diversity; how uneven a community is determines how far below that the actual diversity is. The exponent of entropy is sometimes called the effective number of species (for entropy). What does that mean? If we exponentiate the entropy of the initial four-species even community, we get back 4, the number of species. If we exponentiate the uneven community, we get back 3.79. This way, we match our uneven community to an even community of 3.79 species. An even community of 3.79 species has the same entropy as our uneven community, 1.33. So, weird as it may sound, we use \(species\) as a continuous unit of measurement to measure diversity in a way that, in fact, incorporates information about both the number of species and the community’s evenness.

# the entropy of an even 4-species community
-((1/4)*log(1/4))*4

[1] 1.386294

# the exponent of the entropy of an even 4-species community
exp(-((1/4)*log(1/4))*4)

[1] 4

# the entropy of an uneven 4-species community with [.4, .2, .2, .2]
-((.4)*log(.4)) - ((.2)*log(.2))*3

[1] 1.332179

# the exponent of the entropy of an uneven 4-species community with [.4, .2, .2, .2]
exp(-((.4)*log(.4)) - ((.2)*log(.2))*3)

[1] 3.789291

# the entropy of an even community of 3.79 species
-((1/3.789291)*log(1/3.789291))*3.789291

[1] 1.332179

You can see this from the perspective of a generalist consumer that feeds indiscriminately on these four species. In an environment where the four prey species are equally “available”, the consumer’s diet will consist of four species. In an environment where 97% of prey individuals belong to one species, with the other three species taking up 1% each, the effective number of prey species will be 1.18. The effective diversity of the consumer’s diet is slightly above one species.

exp(-((.97)*log(.97)) - ((.01)*log(.01))*3)

[1] 1.182582

Working with the effective number of species (for Shannon’s entropy), we can examine changes in diversity across an additive scale of even communities. Working with entropy, the natural logarithm of the effective number of species, we examine these changes in a multiplicative scale. One unit change is \(e\) times up or down in diversity.

Well, this is because we have been calculating entropy using the natural logarithm. However, there is no particular reason to use the natural logarithm. Switching to base 2 would make things a bit more intuitive, I think³. When it comes to changes in magnitude, whether it’s time, distance or size, half of something or double of something is more graspable than \(e\) times of the thing, right? I would argue that half/double is also more graspable than ten times something or one tenth of the thing⁴. It’s good to make things as graspable as we can. In fact, one argument in favor of using the effective number of species instead of raw Shannon entropies when communicating changes in diversity, is that we might misrepresent the magnitude of change. I am not sure this is a real issue, as long as we keep in mind that we are dealing with multiplicative changes⁵. And especially if we work with base 2 logarithms to calculate Shannon entropy, things are fairly straightforward: one unit of change in entropy represents a doubling (or halving) of diversity in terms of the effective number of species.

Footnotes

Based on the number of individuals or, less commonly, biomass. Either makes sense depending on the context, so it is a good idea to be explicit.↩︎
👽↩︎
{vegan}’s diversity() has a base argument 😉.↩︎
If you were considering the option of base 10. What are you, an astronomer?↩︎
Never mind that you may exponentiate your entropies because you were encouraged to work with effective numbers of species instead and then you find out you have to log them again for the purposes of your analysis.↩︎