Accumulator processor
The Accumulator processor maintains a running state for groups of events as they flow through a meter. Instead of waiting for a complete time window and then discarding state, the Accumulator continuously updates counters or other metrics for each group and emits the current values when a release condition is met.
Use the Accumulator when you need ongoing counters or totals per account, subscription, product, or other key fields, and when you want to control when those running values are emitted for downstream processing.
The Accumulator processor:
- Keeps a separate state bucket for each key combination you define (for example, one per account or per account and subscription).
- Updates that state as each event arrives, using built‑in operations such as sum, count, min, max, or custom logic.
- Emits the current state when the release trigger condition is met (for example, every hour or every day).
- Continues to accumulate after emitting, so the counter or total carries forward into the next period.
- It uses your chosen grouping fields to define partitions. Each unique combination of those fields has its own state.
- When a new event arrives, the processor locates the correct partition and updates its state using the configured accumulation logic.
- At specific times, determined by the release trigger, the processor emits a snapshot of the current state for each partition.
- After emitting, the processor continues to accept and process new events; it does not need to be reset unless your use case requires it.
You can use advanced accumulation when built‑in field operations are not sufficient. For example, if you need conditional counting, weighted metrics, or transformations that depend on multiple fields and prior state. For straightforward sums or counts, use the standard accumulation options.
How the Accumulator processes events
-
It uses your chosen grouping fields to define partitions. Each unique combination of those fields has its own state.
-
When a new event arrives, the processor locates the correct partition and updates its state using the configured accumulation logic.
-
At specific times, determined by the release trigger, the processor emits a snapshot of the current state for each partition.
-
After emitting, the processor continues to accept and process new events; it does not need to be reset unless your use case requires it.
You can think of it as a continuously updating counter or meter, with controlled points at which you take readings. The Accumulator remembers the state between events. By contrast, an Aggregator is typically used to compute results within a given window and does not maintain continuous state across windows.
Use the Accumulator when:
- You need running totals or counts that carry across multiple windows.
- You want to keep live counters per account, subscription, product, or region.
- You want to emit periodic snapshots of current values without recomputing past windows from scratch.
Use an Aggregator instead when:
- You only need summary values for discrete windows and do not need to preserve state between windows.
- You want to process a well‑defined batch or time window and then discard state.
Use cases
Common scenarios where the accumulator works well include:
- Running usage totals per account or subscription.
- Real‑time counters (events per product, per region, or per device).
- Rolling min/max values per meter or sensor.
- Custom averages or ratios that update as events arrive (using custom logic).
Best practices
- Choose grouping keys carefully: each unique combination creates its own state bucket and uses memory.
- Avoid extremely high‑cardinality groupings (for example, a unique ID per event) unless you have a specific reason and the volume is small.
- Use the built‑in accumulation operations for simple sums, counts, and min/max; only use advanced scripting when your logic truly needs it.
- Design your release triggers to match your business requirements: shorter durations produce more frequent outputs; longer durations produce fewer, larger summaries.
- When using event‑time‑based triggers, ensure that event time fields are populated and formatted consistently, and plan how to handle late‑arriving data.
- Validate that emitted fields and types match the expectations of downstream meters or targets, and test with sample data that spans multiple periods before using in production.