Demystifying Tracking Event Processors in Axon Framework

TEPs use their own threads and are in full control of how handling happens, although always independent of the transaction that has published the event (this transaction must have been committed). When we want to replay the events that have happened in the past (usually for read model (re)building) this is the mechanism to use. The rest of this blog discusses the features of TEP and how are those implemented.

Each TEP claims its Tracking Token (in order to avoid multiple processing of the same event in different threads/nodes). Claiming the Tracking Token is the procedure of setting the owner of Tracking Token in the Token Store. The owner is not set indefinitely but for a configurable timeout. When this timeout expires and the current owner does not reclaim the token, a different owner (TEP) can claim it. TEP can release the claim telling other TEPs to continue with the processing. A conscious release of the Tracking Token sets the basis for TEP rebalancing — having a load of event processing equally distributed across TEPs.

In Axon, parallel processing is achieved by segmenting an event stream. For a certain TEP, we would start several threads which would work on their own segment of event stream in parallel. The number of segments per TEP is configurable. Let’s define what a segment actually is.

A segment is a fraction of the total stream of events (see Figure 2). In other words, Tracking Token is segmented into several portions which means that Token Store contains an entry per Tracking Token and Segment. A segment contains segment identifier and a mask. Mask is used to determine whether a certain event belongs to the given segment. Segment plays a significant role in parallel and distributed event processing. Each TEP can have several segments assigned to it. For each one of them, it starts a separate thread for event processing enabling us with parallel processing. Segments can be distributed among TEPs on several nodes giving us the possibility to process events in a distributed fashion (do note that TEPs on different nodes still have to claim the Tracking Token so they don’t process the same event).

When TEP is started it starts a worker for each segment in a separate thread. Each one of them tries to claim a Tracking Token for a certain amount of time. If claiming is successful, a processing loop is started. The happy flow of processing loop contains the steps shown in Figure 3.

After event processing has stopped (either explicitly or by an error) the TEP releases the segment so another processor can claim it and continue processing.

In Tracking Event Processing, events are processed in different threads which makes the error handling more complicated. To solve this problem, Axon provides Error Handlers that may be configured on the TEP and act when the exception occurs in the event handling component. By default, exceptions are propagated ultimately causing the TEP to release its claims and start retrying. If necessary, custom Error Handlers may be provided to the TEP. The recommendation is to have error cases clearly defined and represented by corresponding exceptions. Having this, Error Handlers can act on them with different strategies in order to resolve the unexpected behavior.

Tracking event processing is a really powerful mechanism in the Axon Framework which gives us

In CQRS architectures query models are updated separately from command models which give us the possibility to scale our query models differently. In such cases (and many other) Tracking Processing is a preferred way of event processing. Hence, Tracking Processing is the default in when using an Event Store.

Related posts:

Os dois amigos estavam sentados na sala, um olhando o celular e outro tomando um copo d’água. Finalmente tinha dado certo de um visitar a cidade com outro e, após um almoço caro (porém saboroso)…

A plethora of mobile developers are looking to add video chat in mobile apps to drive engagement and interactions between users. Whether we look at dating, telemedicine, gaming, and even e-commerce…

Are you facing sleep deprivation and insomnia? Here are some ways to improve your sleep quality and durarion.