Joint entropy

The Verbundentropie is a measure of the average information content of a plurality of news sources.

Definition

The Verbundentropie for two sources is defined as follows:

Here, x and y are in each case individual symbols of the source alphabets X and Y.

Applies in the special case of statistical independence of the sources

And thus

For more than two sources yields: