By removing the constraint of atomicity for a transaction, we can break up a long lived transaction T into several transactions: \(T_{1},T_{2},T_{3},T_{n}\)

In order to maintain consistency, we must also provide compensatory transactions for failure cases: \(C_{1},C_{2},C_{3},C_{n}\)

In the context of Event Sourcing, this is an argument against achieving idempotent events through stateful payloads instead of delta payloads. To make this clear, consider the following two sets of event sequences:

Stateful Payloads

Stateful payloads offer limited idempotency: if I receive the same payload twice, the balance remains the same. However, each event must take a full lock, as each event is dependent on the state of the last event. If two events are written concurrently, they will destructively interfere with each other. Because of this, stateful payloads:

• Constrain write throughput
• Have destructive compensatory events
• Are inherently idempotent if sequentially constrained, provided events are never received out of order (e.g. receiving \(E_{3}\) multiple times is non-destructive, unless we receive it again after receiving \(E_{4}\) ).

The compensatory events are destructive, because if I were to write a compensatory event for \(E_{2}\) at the time \(E_{2}\) occurred it would be {balance: 3}. Applying \(C_{2}\) after \(E_{4}\) results in a state of {balance: 3} instead of {balance: 4}.

Stateful payloads are not a practical solution to idempotency because where there is more than once delivery, there is out of order delivery. Instead, make the consumer idempotent by keeping a log of processed events.

Let’s consider the alternative, events that describe what delta to apply to the state to derive the new state.

Delta Payloads with Non Sequential Events

As we can see the state at each step is the same. Now we’ve lost the limited idempotency of stateful payloads, but gained:

• Improved write throughput
• Non-destructive compensatory events

Lets create a compensatory event for each of these events:

Compensatory Events

If I want to rollback \(E_{2}\) I simply apply \(C_{2}\) to the event stream, resulting in

This only applies to non-sequentially constrained event streams. As soon as a sequentially significant event is applied, the entire event stream becomes sequentially constrained, even if the other events are non-sequential on their own.

Here we can see set \(E_{1},E_{2},E_{3},E_{4},C_{2}\) is equivalent to set \(E_{1},E_{3},E_{4}\)

This is because any sequence of \(E_{n}\) is equivalent to any other sequence of \(E_{n}\), and so is its complimentary set of \(C_{n}\) and thus the set of any combination of the members of the two sets combined also be equivalent.

Delta Payloads with Sequential Events

Consider applying \(C_{2}\) to the above event stream.

However, our objective is to wind back the effects of \(E_{2}\), so the desired outcome is

In this case, we cannot guarantee a compensatory event will be non-destructive unless we can guarantee no sequential events have been inserted between \(E_{2}\) and \(C_{2}\). With sequential events in the mix, we can only make that guarantee if we lock the event stream, constraining write throughput and preventing entries between \(E_{2}\) and \(C_{2}\) to regain atomicity.

This leads us to the following assertions:

• Compensatory events are non-destructive if both set \(E_{n}\) and set \(C_{n}\) are non-sequential.
• \(E_{n}\) is non-sequential if \(E_{1},E_{2},E_{3}\) is equivalent to \(E_{2},E_{1},E_{3}\) and any other combination.
• A single sequential event in an event stream breaks the non-destructive compensation guarantee.

Caveats

• Event stream sequentiality is derived from the combination of reducer and events. This means you could create a second reducer that processes events such that they are no longer non-sequential.
  • If \(R_{1}(E_{n})\) is non-sequential, you cannot guarantee \(R_{2}(E_{n})\) is also sequential.

Practical Applications

When designing event sourced systems with long lived transactions modelled as sagas, you gain non-destructive compensatory events by ensuring your events are non-sequential. You only need to maintain this non-sequentiality constraint for events in the same stream processed by the same reducer. This means the projection derived from \(R_{1}(E_{n})\) is unaffected by sequential events in stream \(E_{b}\), ergo \(R_{2}(E_{n+b})\) cannot non-destructively handle compensatory events in stream \(E_{n}\) while \(R_{1}(E_{n})\).

In a practical sense, reverting a sales order might re-apply deducted funds from a users account — \(R_{1}(E_{n})\) is non-destructive — but it won’t put an item back into inventory if it has already shipped — \(R_{2}(E_{n+b})\) is destructive.

The key takeaway here is to be mindful of your events — are they sequential? If so, is it because there is some stateful information in the event payload that should be refactored into a delta? If it cannot be turned into a delta, you will need to consider the business implications of your compensatory event, i.e. booking a return parcel for a customer, dispatching an email, etc. It often becomes a matter of invoking a compensatory command or saga within the destructively compensated domain.