3 // Streams

Since we like the idea of letting our applications be based on event sourcing, as we get an automatic audit log and can easily rebuild the state of our application, we decided to build Freym on top of it. So we created a service that manages the event streams. This page will give you an idea of the Streams service.

Features

The Streams service provides several APIs related to events:

Tenancy

The Streams service supports multi-tenancy. Each event is associated with a tenant. Each tenant's data is stored in a separate database. Any action associated with events must specify the tenant being used.

Publishing Events

The Streams service distinguishes between two types of events: Persistent Events and Messages.

• Persistent events are stored in the event store and can be retrieved later.
• Messages are not stored in the event store and are only used for communication between services. We also refer to messages as broadcast events.

Use messages for real-time communication between services where the event history is not important. Use persistent events to store business-critical data where you want to keep track of the event history.

The Streams service allows you to publish multiple events in a transactional manner. If an error occurs while publishing one of the events, none of the events will be published.

While an event must be associated with a topic, it can also be associated with a stream. This enables event streaming functionality for the event.

Subscribing to Events

The Streams service allows subscriptions to both persistent events and messages. It is possible to create a subscription that listens to multiple topics at once.

If the subscription handler fails to process a persistent event, the event is retried until it is successfully processed. Messages are not retried.

Querying Events

The Streams service provides several APIs to query events:

• Get an event by its ID
• Get the latest event within a topic
• Get the last event within a topic that belongs to a given list of event types
• Paginate through all events in a topic that belong to a given list of event types
• Paginate through all events in a topic that belong to a given list of event types that come after a specific event

Querying Streams

The Streams service provides several APIs to query streams:

• Check if a stream is empty
• Paginate through all events of a stream
• Paginate through all events of a stream that come after a specific event

Snapshotting

Snapshotting can be used to optimize performance when reading event streams. When a snapshot event is added to the event stream, all APIs that paginate the stream will not return any events that occurred before the snapshot. Instead, they return the snapshot event and any events that come after the snapshot.

GDPR

When publishing an event, a payload field can be associated with GDPR logic by specifying a default value for the field. The default will be applied if the field is invalidated.

When GDPR-relevant data is published, an event of type gdpr-data-recorded is published to the gdpr topic. This event contains all the information needed to build the logic to invalidate the data if necessary. While the CRUD service takes care of automatically invalidating GDPR relevant CRUD data, you need to implement GDPR logic using the events in the gdpr topic for any events that your business logic uses.

The invalidation API ensures that the GDPR relevant data is deleted and replaced with the default value. After the invalidation, there is no way to recover the original data as any trace of it is removed.

Further Reading Beyond The Freym Journey

Want to learn more about event sourcing or messaging now? Select one of the following articles to dive deeper into the topic.