Channel capacity: information limits, input distributions and reliable rates

A link can transmit a large number of signaling bits per second while delivering fewer useful information bits reliably. Some transmitted bits provide redundancy or framing. Others may not be distinguishable accurately enough at the receiver under the actual channel conditions.

Channel capacity addresses the information limit of an explicitly defined channel with stated constraints. An advertised bitrate is a counting rate for some chosen signaling or protocol layer. The two quantities can share units without describing the same thing.

A mathematical capacity claim must name its channel model and what inputs are permitted. It also has a specific interpretation: below the limit, suitable coding can approach arbitrarily small error in an asymptotic setting under the model. That is not a promise that a particular modem, short message, or decoder reaches the limit.

Separate symbol rate, assigned bits, code rate, and useful rate

Symbol rate counts how many signaling symbols are sent per second. A binary alphabet offers two labeled choices, often represented as one signaling bit per symbol. An alphabet with more choices can assign several bits to each symbol under a defined mapping. That assignment does not establish that all the choices are equally reliable or equally informative.

Code rate is another quantity. A block code that represents k input information bits with n transmitted coded bits has rate k/n. Even if the signaling system sends one coded bit per symbol, only that fraction corresponds to the code’s input bits under the stated block structure.

Framing, training, identifiers, and other required fields can reduce the application payload further. Useful rate must therefore identify the counted layer. A coded-input rate and an application-payload rate can differ even before errors or retransmissions are considered.

For a hypothetical binary link carrying 1,000 channel uses per second, a rate-one-half code carries 500 code-input bits per second before other overhead. The channel still sees 1,000 transmitted bits. A rate report should identify which of these layers its count represents.

The rate says how many inputs are represented; reliability concerns whether the receiver identifies them correctly. A design can increase the represented information rate while worsening its error probability. Neither rate nor reliability should be inferred solely from the number of physical symbols.

David MacKay's Information Theory, Inference, and Learning Algorithms defines channel capacity by maximizing mutual information over the input ensemble. Its binary symmetric example contrasts skewed and balanced inputs, and its coding theorem gives sufficiently long codes and decoders with arbitrarily small maximal block-error probability at rates below capacity. Its illustrated capacity calculation uses flip probability 0.15. The p = 0.1 values here are independently calculated from the same stated channel form; no book example or product measurement is relabeled as this hypothetical link.

A noisy binary channel leaves uncertainty at the receiver

Consider a stipulated binary symmetric channel. Each use has input X and output Y. The output equals the input with probability 1 − p and is flipped with probability p. Different uses are conditionally independent under this memoryless model, and p remains fixed for the comparison.

With equally likely input bits, the output is also balanced. Its entropy is one bit. Given the input, uncertainty about the output is the binary entropy h2(p) = −p log₂p − (1 − p) log₂(1 − p). That uncertainty reflects the channel’s random flipping rule.

The information shared between input and output is I(X;Y) = H(Y) − H(Y | X) = 1 − h2(p) for this balanced input. At p = 0.1, h2(p) is about 0.4690, so the shared information is about 0.5310 bits per channel use.

The physical output remains a binary value on every use. The reduced information quantity describes uncertainty about how that value relates to the input. Counting received bits alone cannot tell how much input information those bits preserve.

At p = 0.5, the output is independent of the input in this model, and the information is zero. At p = 0, the channel passes the input exactly. At p = 1, it inverts the input deterministically, which can also preserve it if the inversion rule is known. Thus “more flipped bits” is not a complete information argument without the actual rule.

Mutual information depends on the input distribution

The quantity I(X;Y) concerns a joint distribution of inputs and outputs. Changing the input probabilities while leaving the conditional channel rule unchanged can change the information per use. A result for one chosen input distribution is not automatically the channel’s optimized limit.

For the binary symmetric channel at p = 0.1, suppose the input is one with probability 0.9. The output is then one with probability 0.82. Its entropy is about 0.6801 bits, while the conditional output uncertainty remains h2(0.1), about 0.4690. The resulting shared information is about 0.2111 bits per use.

The skewed input conveys less information per use than the balanced input in this example. It offers less uncertainty to resolve at the start, even though the physical channel has the same flipping probability. The distinction is a property of the stipulated joint distribution.

The filter-bank guide supplies a representation comparison for mutual information. Input pairs (3,1) and (1,3) have the same average, so that branch alone cannot distinguish them. Their different difference values restore the distinction when both branches are retained. That is an algebraic dependency example, with no probability distribution assigned. Computing I(X;Y) for it would also require such a distribution. The binary-channel calculation here provides its input and conditional output probabilities explicitly.

For a discrete memoryless channel with stated allowable inputs, capacity is the maximum I(X;Y) over the permitted input distributions. The balanced input achieves the binary symmetric channel’s maximum 1 − h2(p). Optimizing the input is therefore part of the specified capacity calculation.

Constraints and stationarity belong to the capacity claim

A model can restrict average power, allowed symbols, input timing, or another resource. The maximization must respect those restrictions. Removing a constraint in the calculation changes the problem; it does not establish that a real transmitter can use the unconstrained result.

The single-use formula above assumes the stipulated memoryless channel. A channel with memory can require a model of dependencies across uses and a corresponding information-rate analysis. Substituting a measured average error count into the memoryless formula would silently assume away that structure.

Stationarity and knowledge of the channel also matter. If the channel changes over time, the coding strategy and reliability interpretation depend on how it changes and what the transmitter or receiver knows. A limit under one fixed p does not automatically describe a varying environment.

For the hypothetical channel at p = 0.1 and 1,000 uses per second, the optimized limit is about 531 information bits per second under the stated model. This multiplication assumes that the channel-use rate is known and that the counted limit does not already include another layer’s overhead.

Applying this idealized discrete-channel calculation to a physical connection requires evidence for its channel-use rate, conditional output rule, and input constraints. The example supplies these as assumptions, so its 531-bit-per-second limit remains a mathematical model result. Neither listed product provides the error observations needed to establish that model.

Reliable communication below capacity is an asymptotic claim

Under the standard memoryless-channel assumptions, rates below capacity permit suitable sequences of codes whose error probability becomes arbitrarily small as blocklength grows. The statement establishes existence in an asymptotic regime. It does not select an actual code or guarantee a decoder implementation.

At capacity about 0.531 bits per use, rate one-half lies below the model’s limit. The theorem permits asymptotically small block-error probability with suitable codes and decoding algorithms at that rate. Checking a particular finite-length code and its implemented decoder remains a separate calculation or experiment.

A rate of 0.8 information bits per use lies above the same capacity. Under those model assumptions, increasing blocklength cannot drive the error probability arbitrarily close to zero while keeping that rate. A raw stream carrying one transmitted bit per use can still run; the limit concerns reliable information recovery.

The multiple-description guide gives an outcome comparison for reliable communication. Its toy source level eleven has a side reconstruction of 10.5 from one description and exact eleven from both. A defined distortion tolerance may accept the first outcome; an exact-message requirement distinguishes it from the second. The guide supplies no probability bound for the channel here. Choosing the error event is likewise necessary before evaluating a finite code against an asymptotic capacity claim.

Finite blocks, decoder complexity, latency, and model mismatch all affect implementation. A practical result therefore needs a code description, a decoder, a specified error event, and measurements or analysis under relevant channel conditions. A capacity value alone cannot supply those missing facts.

Check what an advertised number actually counts

A rate label may count physical signaling, coded bits, protocol data, or useful payload. Identify that layer before comparing it with an information limit. The denominator also matters: bits per second and bits per channel use require a known relation between elapsed time and channel uses.

Report the symbol rate, bits mapped per symbol, code rate, framing overhead, and measured useful delivery at their defined layers. Compare the relevant information rate with a capacity calculation only after specifying its channel model and constraints.

Test reliability by comparing recovered messages with known originals or another justified reference and counting the defined failures. Observing a fast stream supplies its activity and count rate. It leaves correct delivery untested until the recovered content has been checked.

The distinction remains useful even when no defensible capacity estimate is available. A report can truthfully state the observed signaling and payload rates while retaining uncertainty about the channel’s information limit. Inventing a channel model to fill the gap would make the comparison less reliable.

Defining the relevant physical connection

The Unbranded ZT-703S Electrical Testing instrument, SKU 18623303850, lists waveform-image storage and export among its measurement functions. Those exported images provide a context for counting transferred records. An information limit for that transfer would require the actual signaling channel and its constraints; the listing supplies no binary flip probability for this calculation.

The Focusrite AMZFCS4I4CR3X Red 4i4 Kit, SKU 1894010824, provides four balanced outputs for monitor routing alongside its four-input USB-C interface. The kit also supplies Mackie CR3-X monitors, a condenser microphone and closed-back headphones. These listed components and connection counts describe a recording setup. A transmission-capacity claim would additionally need a communication model, permitted signaling inputs, and error behavior for the relevant connection.

For a finite-code test, keep the original messages and record the decoded outputs. State the number of channel uses per block, the represented message bits, and whether failure means any incorrect bit or an incorrect block. The observed failure count then answers a defined reliability question at a defined rate. The asymptotic theorem supplies a limit for the stipulated channel; the test evaluates the particular code and decoder.