How pipeline backpressure connects admission to retained memory

Three slots restrict how many items remain inside one queue. A worker can remove an item, free its slot and continue retaining its payload. If other stages also retain work, the queue occupancy cannot represent the pipeline's live memory.

To bound retained memory, pipeline backpressure must govern admission according to the resources still held by the design. The account includes executing work, prefetched inputs, expanding outputs and results awaiting ordering or delivery.

The examples use stipulated payload sizes and retention limits. They count live backing objects. Allocator fragmentation, cached free regions and runtime-specific measurements require separate evidence beyond this object-retention account.

Define bounded queue memory in bytes as well as items

Start with a three-slot input queue. If each queued payload is one MiB, its retained payload total can be three MiB. If each is twenty MiB, the same item-count limit can retain sixty MiB.

Both queues obey the count bound, while their retained bytes differ with the payload sizes. Without a maximum item size or another byte admission rule, the three slots do not provide a fixed payload-byte limit.

A byte-weighted queue can instead account for each admitted item's retained weight under a defined budget. The weight must correspond to what is actually held, not merely the length of a small descriptor pointing to a large payload elsewhere.

If the design allows a maximum input payload of four MiB, three queue slots give a twelve-MiB payload bound for that storage boundary. The rule needs to be enforced before accepting a larger item into that bound.

Queue nodes and other bounded metadata require another allowance. The initial twelve-MiB calculation covers payloads only. It is not a total process-memory measurement.

The active-queue-management guide linked through bounded queue memory illustrates a finite buffering boundary whose congestion-signaling rule is separate from exhausting its capacity. A transient burst and a persistent backlog can reach the same occupancy while requiring different interpretations. Here, the queue's storage limit must likewise be kept distinct from the admission policy and the work retained elsewhere.

Trace in-flight work accounting across every retention boundary

Use a pipeline with a source, a producer preparing inputs, the three-slot queue, two workers, an output reorder buffer and a sink. The source can also prefetch two inputs.

A queue item removed by a worker stops occupying a slot, but its input payload remains alive while the worker needs it. If the worker constructs a separate larger output, input and output can coexist.

The producer can retain a completed payload while waiting to enqueue it. A source-prefetch buffer can retain other payloads that have not reached the queue. Neither location is counted by the queue's current occupancy.

The reorder buffer retains completed outputs until their required sequence position is deliverable. The sink can also retain the output it is currently consuming. Completion of worker computation does not imply either object was released.

The TCP congestion-control guide linked through in-flight work accounting tracks sent but unacknowledged data under the sender's congestion and receiver allowances. Its release event belongs to that transport account. The application-memory account here needs a separate release boundary: a queue removal, delivery report or computation completion cannot return credit for a backing object that another stage still retains.

Count unique live backing allocations under the stated model. If two references share one payload, counting its bytes twice would overstate that payload's contribution. If a transformation creates a second buffer, treating the two objects as one would understate the peak.

Construct a conservative peak-retention account

Stipulate maximum input payloads of four MiB and maximum worker outputs of twelve MiB. The producer may hold one input, the source may prefetch two, and the input queue holds at most three.

The source-prefetch allowance is eight MiB. The producer's separate input allowance is four MiB. The queue allowance is twelve MiB. These are distinct simultaneously retained objects in this model.

Each of the two workers may retain its four-MiB input while building a separate twelve-MiB output. That gives sixteen MiB per worker, or thirty-two MiB for both workers together.

Allow three twelve-MiB outputs in the reorder buffer, giving thirty-six MiB. Allow the sink to hold one twelve-MiB output. Finally, stipulate a four-MiB bound for the included task and queue metadata.

The resulting conservative account is eight plus four plus twelve plus thirty-two plus thirty-six plus twelve plus four MiB, or 108 MiB. It sums declared simultaneous retention limits. It does not predict that every stage is always full or claim a measured process footprint.

The arithmetic is useful only if every allowance is enforced. Unlimited output expansion, unbounded prefetch or an unbounded number of blocked producers would invalidate the account without changing the input queue's three-slot limit.

Admission must occur before unbounded retention

Suppose many producers each allocate a full input and then block while trying to enter the full queue. Only three items are queued, but every waiting producer retains another payload. The slot bound has moved the backlog into the waiting tasks.

A resource-admission design must count those waiters or constrain them before they allocate the payload. Acquiring a budget reservation before construction can be part of a specified model, but the reservation needs to cover the actual retained work and later expansion.

Return queue-slot allowance when an item leaves the queue if the allowance represents queue occupancy. Keep a separate whole-pipeline charge while the worker or another owner still retains the payload. In the Python asynchronous queue API, removal by get frees queue capacity, while task_done reports completion for its unfinished-task account. Those are distinct events; neither API label independently proves that every backing object has been released.

For a full-pipeline byte budget, the ownership transfer from queue to worker must preserve the appropriate charge until the object is released or replaced under another counted allowance.

Likewise, knowing an encoded input's small size does not bound a larger decoded or transformed output. The expansion limit and its enforcement belong in the same retention model rather than being left to a separate informal expectation.

Downstream saturation must reach upstream admission

Suppose the sink stops making progress while the reorder buffer reaches its three-output limit. Further completed outputs have no free counted place in that buffer.

The workers can remain holding their allowed results, but the pipeline must then restrict additional work according to the declared budget. If it keeps launching new workers or expanding outputs, the downstream bound has not controlled total retention.

With two worker slots and the other stated allowances enforced, upstream admission eventually stops adding work. The full input queue then prevents the counted producer from advancing, and prefetch must stop at its own limit.

Under this scenario, stopped admission is the specified consequence of unavailable downstream capacity. The relevant question is whether admission responds before another uncounted allocation occurs.

When the sink releases an output and the necessary credits return under their own rules, the pipeline can admit work again. A stage must not return a whole-pipeline allowance while some other owner still retains the object it was meant to cover.

Output ordering can create a hidden backlog

Let the first input take longer to finish than later inputs, while the sink requires original input order. Later completed outputs wait behind that first result even if the workers performed their computations correctly.

A reorder buffer with no enforced limit can accumulate those results as the pipeline keeps admitting new jobs. The input queue can remain bounded or nearly empty throughout. Its occupancy would miss the growing retained-output backlog.

Under the three-output allowance, further admission or completion placement must respect the full buffer. The design also needs to ensure that the needed earlier result has a permitted path to reach the sink without violating the account.

Completion of the ordered stream requires a progress argument alongside this memory bound. The account states where results may be retained and when more work may be admitted. Whether the earliest outstanding result can reach the sink must be established from the implemented dependencies and completion path.

Blocking, rejection, and dropping preserve different service contracts

A blocking policy delays admission until the required capacity is available. It can preserve the obligation to process accepted work, but the waiting party's retained state must still be included in the budget.

A rejection policy refuses new work when the budget is unavailable. The caller needs a truthful refusal rather than a success receipt. The pipeline must also settle any preparation resources already acquired for that rejected request.

A dropping policy intentionally discards specified work under its service contract. A freshness-oriented application might permit old unprocessed data to be replaced, while a completeness-oriented record processor might not.

Dropping cannot be introduced solely to keep a graph's memory line low if the application promised every accepted item would be processed. The policy needs to identify what is discarded, when, and what outcome the caller observes.

Choose blocking, rejection or permitted dropping according to the service obligation. Record what the caller observes and which retained resources remain charged. Reactive Streams likewise defines demand as requested elements not yet delivered and prohibits delivery beyond that demand. An element-count contract needs the separately supported byte weights and ownership boundaries used in this example.

Verify the bound across ownership transfers

Preserve every retention location, maximum payload weight, expansion allowance and admitted owner count. Sum the simultaneous bounds using unique objects, then check transfers and release points against that model.

The check should fail if a fourth reorder output is retained outside any allowance or if unlimited producers allocate before waiting. Those invalid full-pipeline histories can still satisfy the three-slot queue test, which is why both boundaries need separate checks.

The catalogue identifies the Lenovo IdeaPad 15.6-inch FHD touch laptop with an Intel Core i5-1334U, 16GB DDR4 and 512GB SSD. Keep that host capacity distinct from the application's admitted-payload budget. Establishing the 108-MiB account requires the saved stage limits and observed ownership behavior, in addition to the hardware listing.

The ASUS Vivobook 14 FHD laptop listing describes an Intel Core 5 120U and 16GB DDR4. The pipeline record should retain its prefetch, expansion and reorder limits beside that host identity, so another reviewer can reproduce the resource boundary instead of inferring it from installed memory.

A fourth twelve-MiB reorder output retained outside every allowance would increase the example's accounted maximum from 108 to 120 MiB. The input queue could still obey its three-slot rule. Rejecting that history requires the whole-pipeline retention contract, including the point at which each owner transfers or releases its backing objects.