Tutorial

EQL is name of the event query language that Esper provides. EQL is very similar to SQL in it's syntax and offers additional capabilities for event stream processing. As part of EQL, Esper also offers a pattern language that provides for stateful (state-machine) event pattern matching. EQL and patterns can be used alone or can be combined into useful, easy to read statements. EQL also has object-oriented event processing capabilities leveraging the Java type system and XML.

EQL offers two principal methods of event processing that can be used alone or in combination:

  1. Event stream analysis
  2. Event pattern matching

Event Stream Analysis

EQL statements are used to derive and aggregate information from one or more streams of events, to join or merge event streams, and to feed results from one event stream to subsequent statements.

EQL is similar to SQL in it's use of the select clause and the where clause. However EQL statements instead of tables use event streams and a concept called views. Similar to tables in an SQL statement, views define the data available for querying and filtering. Views can represent windows over a stream of events. Views can also sort events, derive statistics from event properties, group events or handle unique event property values.

This is a sample EQL statement that computes the average price for the last 30 seconds of stock tick events:

  select avg(price) from StockTickEvent.win:time(30 sec) 

A sample EQL that returns the average price per symbol for the last 100 stock ticks.

  select symbol, avg(price) as averagePrice
    from StockTickEvent.win:length(100)
group by symbol

This example joins 2 event streams. The first event stream consists of fraud warning events for which we keep the last 30 minutes (1800 seconds). The second stream is withdrawal events for which we consider the last 30 seconds. The streams are joined on account number.

  select fraud.accountNumber as accntNum, fraud.warning as warn, withdraw.amount as amount,
         MAX(fraud.timestamp, withdraw.timestamp) as timestamp, 'withdrawlFraud' as desc
    from FraudWarningEvent.win:time(30 min) as fraud,
         WithdrawalEvent.win:time(30 sec) as withdraw
   where fraud.accountNumber = withdraw.accountNumber

Event Pattern Matching

Event patterns match when an event or multiple events occur that match the pattern's definition. Patterns can also be temporal (time-based). Pattern matching is implemented via state machines.

Pattern expressions can consist of filter expressions combined with pattern operators. Expressions can contain further nested pattern expressions by including the nested expression(s) in round brackets.

There are 5 types of operators:

  1. Operators that control pattern finder creation and termination: every
  2. Logical operators: and, or, not
  3. Temporal operators that operate on event order: -> (followed-by)
  4. Guards are where-conditions that filter out events and cause termination of the pattern finder, such as timer:within
  5. Observers observe time events as well as other events, such as timer:interval, timer:at

A sample pattern that alerts on each IBM stock tick with a price greater then 80 and within the next 60 seconds:

  every StockTickEvent(symbol="IBM", price>80) where timer:within(60 seconds)

A sample pattern that alerts every 5 minutes past the hour:

  every timer:at(5, *, *, *, *)

A sample pattern that alerts when event A occurs, followed by either event B or event C:

  A -> ( B or C )

An event pattern where a property of a following event must match a property from the first event:

  every a=EventX -> every b=EventY(objectID=a.objectID)

Combining Patterns Matching with Event Stream Analysis

Patterns match when a sequence (or absence) of events is detected. Pattern match results are available for further analysis and processing.

The pattern below detects a sitation where a Status event is not followed by another Status event with the same id within 10 seconds. The statement further counts all such occurances grouped per id.

select a.id, count(*) from pattern [ every a=Status -> (timer:interval(10 sec) and not Status(id=a.id)]
group by id

Event Representation in Java

Events are represented as JavaBean instances. Esper can interrogate any plain-old Java object representing an event as long as the object exposes the required getter methods following the JavaBean conventions.

Esper can also query object graphs consisting of arrays, mapped or nested event properties, or any combination. The following are examples of valid event properties for EQL and pattern statements:

  OrderEvent.item[1].description                                
  RFIDEvent.attribute('shelfcode')