The Dispatch Pipeline¶
Every action in Acteon flows through a well-defined pipeline. Understanding this pipeline is key to predicting how your rules and providers will interact.
Pipeline Overview¶
flowchart TD
A[Action Received] --> B[Acquire Distributed Lock]
B --> C[Evaluate Rules by Priority]
C --> D{First Matching Rule}
D -->|suppress| E[Return Suppressed]
D -->|deduplicate| F{Check State Store}
D -->|throttle| G{Check Counter}
D -->|reroute| H[Change Provider]
D -->|modify| I[Transform Payload]
D -->|group| J[Add to Group]
D -->|state_machine| K[Process State Transition]
D -->|require_approval| L[Create Approval Request]
D -->|chain| M[Start Chain Execution]
D -->|llm_guardrail| N[LLM Evaluation]
D -->|No match| O[Execute via Provider]
F -->|Exists| P[Return Deduplicated]
F -->|New| O
G -->|Under limit| O
G -->|Over limit| Q[Return Throttled]
H --> O
I --> O
O --> CB{Circuit Breaker}
CB -->|Closed/HalfOpen| R[Provider Execution with Retries]
CB -->|Open + Fallback| FB[Walk Fallback Chain]
CB -->|Open + No Fallback| CO[Return CircuitOpen]
FB -->|Healthy provider found| FBE[Execute via Fallback]
FB -->|Chain exhausted| CO
R -->|Success| S[Return Executed]
R -->|All retries failed| T[Return Failed / DLQ]
FBE -->|Success| RR[Return Rerouted]
S --> U[Record Audit Entry]
RR --> U
CO --> U
E --> U
P --> U
Q --> U
T --> U
U --> V[Release Lock]
V --> W[Return Outcome to Caller] Step-by-Step Walkthrough¶
1. Action Intake¶
The action enters the system through the HTTP API (POST /v1/dispatch) or directly via the Rust Gateway::dispatch() method. The action is validated and assigned a UUID if one isn't provided.
2. Distributed Lock Acquisition¶
If any deduplication rules are configured, the gateway acquires a distributed lock keyed by the action's namespace, tenant, and dedup key. This prevents concurrent processing of duplicate actions.
Lock Behavior by Backend
- Memory: Perfect mutex (single process)
- Redis: Strong locking (single instance)
- PostgreSQL: ACID-guaranteed locks
- DynamoDB: Conditional writes
3. Rule Evaluation¶
Rules are evaluated in ascending priority order (lower number = higher priority). The first matching rule determines the action type:
rules:
- name: block-spam # priority: 1 (evaluated first)
- name: dedup-emails # priority: 10 (evaluated second)
- name: throttle-api # priority: 20 (evaluated third)
Each rule's condition is evaluated against the action. If the condition matches, the rule's action is applied. If no rule matches, the action proceeds directly to execution.
4. Rule Action Processing¶
Depending on the matched rule type:
| Rule Action | Pipeline Behavior |
|---|---|
| Suppress | Immediately returns Suppressed — action is blocked |
| Deduplicate | Checks state store for existing entry. If found, returns Deduplicated. If not, marks as seen and continues to execution |
| Throttle | Increments counter in state store. If over limit, returns Throttled with retry_after hint |
| Reroute | Changes the target provider, then continues to execution |
| Modify | Applies payload transformations, then continues to execution |
| Group | Adds action to an event group. If group is ready, flushes. Returns Grouped |
| State Machine | Computes fingerprint, checks/updates state. Returns StateChanged |
| Require Approval | Creates approval record with HMAC-signed URLs. Returns PendingApproval |
| Chain | Initiates multi-step chain. Executes first step. Returns ChainStarted |
| LLM Guardrail | Sends action to LLM for evaluation. Blocks if flagged |
5. Circuit Breaker Check¶
Before executing, the gateway checks the provider's circuit breaker (if enabled). This happens before the executor's retry logic:
| Circuit State | Behavior |
|---|---|
| Closed | Request proceeds to execution |
| HalfOpen (probe available) | Request proceeds as a recovery probe |
| HalfOpen (probe in flight) | Rejected as CircuitOpen |
| Open (fallback configured) | Rerouted to fallback provider (or next healthy provider in chain), returned as Rerouted |
| Open (no fallback / chain exhausted) | Rejected immediately as CircuitOpen |
After execution, the result is recorded in the circuit breaker: successes and retryable failures update the consecutive counters and may trigger state transitions.
See Circuit Breaker for full details.
6. Provider Execution¶
The executor dispatches the action to the target provider with:
- Concurrency limiting via semaphore (
max_concurrent) - Retry logic with configurable backoff strategy
- Timeout per execution attempt (
execution_timeout)
flowchart LR
A[Execute] --> B{Attempt 1}
B -->|Success| C[Return Response]
B -->|Retryable Error| D[Backoff Wait]
D --> E{Attempt 2}
E -->|Success| C
E -->|Retryable Error| F[Backoff Wait]
F --> G{Attempt 3}
G -->|Success| C
G -->|Failed| H[Dead Letter Queue] Retry Strategies¶
| Strategy | Behavior |
|---|---|
| Exponential Backoff | initial_delay * 2^attempt, capped at max_delay |
| Constant | Fixed delay between retries |
| Linear | initial_delay + increment * attempt |
7. Audit Recording¶
After the outcome is determined, an audit record is created containing:
- Action metadata (namespace, tenant, provider, action_type)
- Rule verdict (which rule matched and why)
- Outcome (executed, suppressed, deduplicated, etc.)
- Timing information (dispatch time, completion time, duration)
- Optional payload (if
store_payloadis enabled)
8. Lock Release¶
The distributed lock is released, allowing other instances to process the same action key.
9. Response¶
The ActionOutcome is returned to the caller as an HTTP response or Rust return value.
Background Processing¶
Several features involve asynchronous background work:
flowchart LR
subgraph Background["Background Processor"]
A[Group Flusher]
B[Timeout Checker]
C[Chain Advancer]
D[Cleanup Worker]
end
A -->|"Flush groups past wait time"| GS[(State Store)]
B -->|"Transition timed-out events"| GS
C -->|"Execute next chain step"| EX[Executor]
D -->|"Remove expired records"| AS[(Audit Store)] | Task | Description | Trigger |
|---|---|---|
| Group Flusher | Sends consolidated notifications for ready groups | group_flush_timeout_ms |
| Timeout Checker | Transitions events that have been in a state too long | State machine timeouts config |
| Chain Advancer | Executes the next step in pending task chains | Previous step completion or delay |
| Cleanup Worker | Removes expired audit records | cleanup_interval_seconds |