Information Theory
Coding Rate and Channel Capacity
Define symbols for code length, rate, entropy, capacity, and error probability.
Read guidePrimary topic hub
Information Theory Guides
Interpret entropy, conditional entropy, mutual information, coding rate, channel capacity, and related probability notation without confusing definitions or log conventions.
Topic owner
This hub is the primary overview
The narrower references below remain available for specific notation questions, while this page owns the broad subject intent and prevents unrelated guides from being consolidated together.
Browse 7 supporting references
These pages answer narrower notation questions and link back to this hub; they are intentionally noindex to reduce cannibalization.
Information Theory
Conditional Entropy and Chain Rules
Write conditional entropy, joint entropy, and information chain rules consistently.
Read guideInformation Theory
Entropy, Cross-Entropy, and KL Divergence
Compare related information measures, logarithm bases, and direction conventions.
Read guideInformation Theory
Information Theory: Common Errors and Verification
Common Errors and Verification for information theory: write, format, verify, and publish clear definitions, assumptions, examples, accessible markup, and portable notation.
Read guideInformation Theory
Information Theory: LaTeX Formatting Guide
LaTeX Formatting Guide for information theory: write, format, verify, and publish clear definitions, assumptions, examples, accessible markup, and portable notation.
Read guideInformation Theory
Information Theory: Notation Reference
Notation Reference for information theory: write, format, verify, and publish clear definitions, assumptions, examples, accessible markup, and portable notation.
Read guideInformation Theory
Mutual Information Notation
Relate joint, marginal, conditional distributions, entropy, and mutual information.
Read guide