Physical Sciences
March 18, 2019

Generalising the entropy formula through master equations

Entropy is one of the most important and most widely studied quantities in physics, and for centuries, its value has been robustly described using simple mathematical relationships. Yet however elegant, Tamás Biró at the Hungarian Academy of Science believes the formula is hiding a more complex array of relationships. Through constructing ‘master equations’ to describe these relationships, Biró and his colleagues are using statistics to study problems as diverse as the formation of hadrons, changes in biodiversity, and patterns in popularity on Facebook.
Every system around us has a different degree of chaos associated with it. While atoms within metals are arranged in neat, highly ordered lattices, the movements of gas molecules in the air are far more chaotic and unpredictable. Since the 19th century, physicists have described this chaos using a quantity known as ‘entropy’. The changes in entropy which take place during physical processes form the basis of one of the most fundamental laws of physics: The Second Law of Thermodynamics. The law states that in any system closed off from the outside, entropy must not decrease over time; as such, the universe becomes more chaotic as a whole.<=”image”>

As Dr Biró describes, this inevitable increase in entropy represents a decrease in the number of possible leftover arrangements of particles, or ‘states’, that a system can find itself in as the one-way passage of time progresses. “Macroscopically, entropy has proved never to decrease spontaneously; an idea related to the ‘arrow of time’, or irreversibility,” he explains. “Many processes in nature end up only in some selected states while they may begin in any of a vastly larger amount of possible states. All microscopic changes in a complicated closed system conspire in a way that the total entropy never decreases.”

To quantify entropy, physicists ultimately need to use statistics to study how particles move around into different arrangements over time. “Kinetic theory and its follower, statistical physics, depend on our knowledge about entropy,” Dr Biró continues. “They can be viewed as given mathematical formulae based on the probability of being in a given state. All random, or ‘stochastic’ models are now checked for satisfying such a theorem, accounting for random forces in their calculations.” Even after centuries of scrutiny, these mathematical descriptions have proved time and again to robustly describe real physical systems. However, in their research, Dr Biró and his colleagues now want to take the theory a step further.

Generalising Boltzmann’s formula
In the 1870s, Austrian physicist Ludwig Boltzmann was able to reduce the quantity of entropy down to a remarkably simple, elegant formula. He proposed that its value simply has a logarithmic relationship with the number of possible states a system can have under its current conditions. Yet although the formula is so effective in mathematically predicting changes in entropy, Dr Biró and colleagues believe it cannot completely account for the more complex physical processes which underlay it. In their research, the researchers aim to generalize Boltzmann’s simple equation; identifying mathematical relationships which build up to the overall logarithmic relationship.

All microscopic changes in a complicated closed system conspire in a way that the total entropy never decreases.

“Our research couples to the novel efforts to generalize Boltzmann’s original formula, containing the logarithm – a neat function mapping products to sums,” says Dr Biró. “What are the more general dynamical processes which make it unavoidable to use Boltzmann’s logarithm?” Through several studies, the researchers have used real-world examples of entropy changes to study how Boltzmann’s logarithm can be generalised in this way. The problems they have analysed so far range from fundamental questions of particle physics to patterns of interaction on social media.

Solving the Hadronization problem
In the dynamic first moments of the universe, fundamental particles known as quarks and gluons existed as freely-flowing elements of a gas-like soup, or plasma. However, these particles ultimately cannot exist by themselves; within octillionths of a second after the Big Bang, they grouped together to form stable structures named hadrons – made from three quarks bound together by gluons. Yet the process of ‘hadronization’ throws up a problem for physicists: since hadrons appear to be far more ordered than a quark-gluon plasma, how could the overall entropy of the universe have increased?

In their research, Dr Biró and colleagues have proposed a new way of looking at the problem, based on the earlier work of Brazilian physicist Constantino Tsallis. In the 1980s, Tsallis drew up a new set of statistical parameters to generalise Boltzmann’s formula, resulting in a new branch of statistics for the specific case of entropy. In particular, his equations resulted in new sets of probability distributions – statistical formulae which describe how likely it is for individual particles in a system to be in a certain place. The probability distributions derived by Tsallis could describe the entropy of physical systems far more realistically than previous models.


Schematic representation of the coarse-grained random growth model. Previously published in Zoltán Néda et al. and is under the Creative Commons Licence CC BY 4.0.

Based on Tsallis’ work, Dr Biró’s team could construct a set of ‘master equations’ – mathematical relationships which build up to Boltzmann’s formula collectively but better describe the behaviour of physical systems individually.

This allowed the researchers to approach the hadronization problem from a new angle; providing new insights into how entropy could have increased as a whole in the earliest moments of the universe. Dr Biró and colleagues also applied their mathematics to other situations; testing the effectiveness of their model on systems including patterns of popularity on social media, distributions in income and wealth, along with size distributions of settlements and ecosystems.


Previously published in Zoltán Néda et al. and is under the Creative Commons Licence CC BY 4.0.

A set of master equations
Through these studies, Dr Biró and colleagues have made strides towards generalizing Boltzmann’s original formula, using mathematics adapted from the principles of statistical physics which Tsallis first generalized for the case of entropy. As Dr Biró explains, “We found that nonlinear master equations: dynamical equations treating the probabilities of systems being in given states not linearly, but via some more complicated function, lead to the use of non-logarithmic formulas for the entropy.” Using such sophisticated mathematics, the researchers could construct entropic probability distributions which more closely reflect those which can be observed in nature.

Ultimately, these master equations allowed Dr Biró and colleagues to model how the probability distributions of systems evolve as time progresses, and entropy increases. Over time, an increase in entropy will mean the probability distribution associated with a system can be expected to shift. “Our way of showing this relied on the use of an ‘entropic divergence’: a non-symmetric distance formula between two probability distributions,” Dr Biró continues. “Stable nonlinear master equation models describe a change of probabilities always towards a no-more-changing, stationary distribution.” As such, the team’s master equations could model the changes in entropy of real systems over time.

Dr Biró and colleagues have made strides towards generalizing Boltzmann’s original formula.

Through this work, Dr Biró and colleagues could construct a ’non-extensive’ entropy formula, which accounted for how the probability distribution of a system after a process has played out will be independent of the initial probability distribution for most materials. Since this entropic distance relationship more closely reflected reality, the master equations could be used to construct a highly generalized formula for calculating entropy. “A number of physical systems are described by stochastic master equations,” explains Dr Biró. “Their corresponding stationary distributions belong to maximal entropy; therefore, entropy can also be defined as the entropic divergence from the uniform distribution.”


Left: scheme for processes with local changes (e.g. diffusion). Mid: scheme for locally one-way processes with resets to the ground state (e.g. popularity). Right: scheme for general processes with long jumps (e.g. stress in earthquakes).

Applying the model
The mathematics described by Dr Biró and colleagues can already be used to model the entropy changes which unfold in a wide variety of different systems. “We apply our mathematical model to various physical systems, including complex networks of popularity like citations or Facebook likes, city size distributions, produced hadron energies in big collider experiments, student exchange networks between universities, among others,” Dr Biró concludes. Yet the team’s mathematical description is still far from complete. In future work, the researchers will study more complex nonlinear master equations; enabling for an even greater understanding of how entropy works.

Personal Response

What future research studies do you have planned?

<=”acceptedAnswer”>So far, we have discovered that whenever a dynamical master equation uses non-linear functions of probabilities, then the general way of constructing entropic divergence and through this the entropy – probability relation leads to new formulae. On the other hand, more trivially, transition rates depending on the state properties (including linear preference rates) also determine the stationary distribution having a non-exponential, non-Boltzmannian form.

Our plans include to study systems where both the initial and the target state influences the micro-transition rate between them. Here the increase of entropy is highly a nontrivial question, and no definite answer has been found so far – except for cases restricted by the detailed balance condition. Another future development is to study processes where no detailed balance is possible, due to asymmetric big jumps occurring as easily as small transitions. Such physical systems can show avalanches or earthquakes. For these questions, we concentrate not on the final distribution of stress (which describes a state), but on the distribution of the size of the jumps in the dynamics.

This feature article was created with the approval of the research team featured. This is a collaborative production, supported by those featured to aid free of charge, global distribution.

Want to read more articles like this?

Sign up to our mailing list and read about the topics that matter to you the most.
Sign Up!

Leave a Reply

Your email address will not be published. Required fields are marked *