LaTeX
H(X)=-\sum_i p_i\log_2 p_iVariables
- H(X): entropy
- p_i: probability of outcome i
How to use this formula
Measures uncertainty in a discrete probability distribution in bits.
Important notes
- Terms with p_i=0 contribute zero by continuity.
- Changing the logarithm base changes the unit.
Quick example
A fair binary variable has entropy 1 bit.
Applicability, worked calculation, and verification
Assumptions and domain checks
- Terms with p_i=0 contribute zero by continuity.
- Logarithm arguments must be positive in the real domain, and the base must be stated when it is not implied.
- For the Shannon Entropy, the index variable, lower bound, upper bound, and any empty-sum or empty-product convention must be clear.
- For the Shannon Entropy, identify whether each quantity is a sample statistic, population parameter, estimator, or model value, and check the method assumptions.
Worked example
A fair binary variable has entropy 1 bit.
Common mistakes
- Do not omit the index or bounds in Shannon Entropy; changing either one changes which terms are included.
- For the Shannon Entropy, do not interpret a descriptive statistic as a causal or population conclusion without the sampling and model assumptions.
Continue the workflow
Use Shannon Entropy in your own work
- Check the domainMatch the variables and assumptions to the problem before substituting values.
- Copy the exact notationPreserve grouping, signs, and exponents in
H(X)=-\sum_i p_i\log_2 p_i. - Edit or convertOpen the expression in the LaTeX editor, then export it for your document or web page.
Review and verification
Last reviewed: 2026-07-23
Automated quality check: Kept noindex until the missing evidence is supplied.
Formula references
- Introductory Statistics 2eOpenStax, Rice University — Reviewed probability and statistics definitions, notation, and formulas.
Frequently asked questions
What is the Shannon Entropy used for?
Measures uncertainty in a discrete probability distribution in bits.
Can I copy this formula as LaTeX?
Yes. Copy H(X)=-\sum_i p_i\log_2 p_i or open it in the LaTeX editor.
What should I check before using it?
Confirm that each variable, unit, domain restriction, and assumption matches the problem.