Information Theory & Coding

Information measures, source compression, error-control codes and the representation contracts used to encode and recover data.

Showing 1–3 of 3 guides

Finite-blocklength coding infographic defining the message and error target, information-density variation, averaging with blocklength, framing overhead and transmission time for a verified finite code.

10 min read

Finite-blocklength coding: error targets, variation and delay

Define finite-blocklength coding error targets, calculate information-density variation, and account for framing, message size and transmission time.

Channel capacity infographic comparing signaling and code-input rates, binary-channel input distributions, information limits, asymptotic reliability and the constraints needed to assess a physical connection.

9 min read

Channel capacity: information limits, input distributions and reliable rates

Separate channel capacity from raw bitrate, compare binary-channel input distributions, and check the limits of asymptotic reliability.

$Arithmetic coding shows the probability model, ABA interval subdivision, safe binary-prefix containment, sixteen-A fractional bit cost, and shared model, precision and termination rules for exact recovery.$

9 min read

How arithmetic coding uses intervals and whole-message bit costs

Trace arithmetic coding intervals, binary prefix containment, and fractional payload costs while retaining model, precision, and message-ending rules.