Esper engine configuration is entirely optional. Esper has a very small number of configuration parameters that can be used to simplify event pattern and EQL statements, and to tune the engine behavior to specific requirements. The Esper engine works out-of-the-box without configuration.
An instance of net.esper.client.Configuration represents all configuration parameters. The Configuration is used to build an (immutable) EPServiceProvider, which provides the administrative and runtime interfaces for an Esper engine instance.
You may obtain a Configuration instance by instantiating it directly and adding or setting values on it. The Configuration instance is then passed to EPServiceProviderManager to obtain a configured Esper engine.
Configuration configuration = new Configuration(); configuration.addEventTypeAlias("PriceLimit", PriceLimit.class.getName()); configuration.addEventTypeAlias("StockTick", StockTick.class.getName()); configuration.addImport("org.mycompany.mypackage.MyUtility"); configuration.addImport("org.mycompany.util.*"); EPServiceProvider epService = EPServiceProviderManager.getProvider("sample", configuration);
Note that Configuration is meant only as an initialization-time object. The Esper engine represented by an EPServiceProvider is immutable and does not retain any association back to the Configuration.
An alternative approach to configuration is to specify a configuration in an XML file.
The default name for the XML configuration file is esper.cfg.xml. Esper reads this file from the root of the CLASSPATH as an application resource via the configure method.
Configuration configuration = new Configuration(); configuration.configure();
The Configuration class can read the XML configuration file from other sources as well. The configure method accepts URL, File and String filename parameters.
Configuration configuration = new Configuration(); configuration.configure("myengine.esper.cfg.xml");
Here is an example configuration file. The schema for the configuration file can be found in the etc folder and is named esper-configuration-1-0.
<?xml version="1.0" encoding="UTF-8"?> <esper-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="esper-configuration-1-0.xsd"> <event-type alias="StockTick" class="net.esper.example.stockticker.event.StockTick"/> <event-type alias="PriceLimit" class="net.esper.example.stockticker.event.PriceLimit"/> <auto-import import-name="org.mycompany.mypackage.MyUtility"/> <auto-import import-name="org.mycompany.util.*"/> </esper-configuration>
The example above is only a subset of the configuration items available. The next chapters outline the available configuration in greater detail.
This configuration item can be used to allow event pattern statements and EQL statements to use an event type alias rather then the fully qualified Java class name. Note that Java Interface classes and abstract classes are also supported as event types via the fully qualified Java class name, and an event type alias can also be defined for such classes.
The example pattern statement below first shows a pattern that uses the alias StockTick. The second pattern statement is equivalent but specifies the fully-qualified Java class name.
every StockTick(symbol='IBM')"
every net.esper.example.stockticker.event.StockTick(symbol='IBM')
The event type alias can be listed in the XML configuration file as shown below. The Configuration API can also be used to programatically specify an event type alias, as shown in an earlier code snippet.
<event-type alias="StockTick" class="net.esper.example.stockticker.event.StockTick"/>
Esper can process Java classes that provide event properties through other means then through JavaBean-style getter methods. It is not necessary that the method and member variable names in your Java class adhere to the JavaBean convention - any public methods and public member variables can be exposed as event properties via the below configuration.
A Java class can optionally be configured with an accessor style attribute. This attribute instructs the engine how it should expose methods and fields for use as event properties in statements.
Table 2.1. Accessor Styles
Style Name | Description |
---|---|
javabean | As the default setting, the engine exposes an event property for each public method following the JavaBean getter-method conventions |
public | The engine exposes an event property for each public method and public member variable of the given class |
explicit | The engine exposes an event property only for the explicitly configured public methods and public member variables |
Using the public setting for the accessor-style attribute instructs the engine to expose an event property for each public method and public member variable of a Java class. The engine assigns event property names of the same name as the name of the method or member variable in the Java class.
For example, assuming the class MyLegacyEvent exposes a method named readValue and a member variable named myField, we can then use properties as shown.
select readValue, myField from MyLegacyEvent
Using the explicit setting for the accessor-style attribute requires that event properties are declared via configuration. This is outlined in the next chapter.
When configuring an engine instance from an XML configuration file, the XML snippet below demonstrates the use of the legacy-type element and the accessor-style attribute.
<event-type alias="MyLegacyEvent" class="com.mycompany.mypackage.MyLegacyEventClass"> <legacy-type accessor-style="public"/> </event-type>
When configuring an engine instance via Configuration API, the sample code below shows how to set the accessor style.
Configuration configuration = new Configuration(); ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy(); legacyDef.setAccessorStyle(ConfigurationEventTypeLegacy.AccessorStyle.PUBLIC); config.addEventTypeAlias("MyLegacyEvent", MyLegacyEventClass.class.getName(), legacyDef); EPServiceProvider epService = EPServiceProviderManager.getProvider("sample", configuration);
Sometimes it may be convenient to use event property names in pattern and EQL statements that are backed up by a given public method or member variable (field) in a Java class. And it can be useful to declare multiple event properties that each map to the same method or member variable.
We can configure properties of events via method-property and field-property elements, as the next example shows.
<event-type alias="StockTick" class="net.esper.example.stockticker.event.StockTickEvent"> <legacy-type accessor-style="javabean" code-generation="enabled"> <method-property name="price" accessor-method="getCurrentPrice" /> <field-property name="volume" accessor-field="volumeField" /> </legacy-type> </event-type>
The XML configuration snippet above declared an event property named price backed by a getter-method named getCurrentPrice, and a second event property named volume that is backed by a public member variable named volumeField. Thus the price and volume properties can be used in a statement:
select avg(price * volume) from StockTick
As with all configuration options, the API can also be used:
Configuration configuration = new Configuration(); ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy(); legacyDef.addMethodProperty("price", "getCurrentPrice"); legacyDef.addFieldProperty("volume", "volumeField"); config.addEventTypeAlias("StockTick", StockTickEvent.class.getName(), legacyDef);
Esper employes the CGLIB library for very fast read access to event property values. For certain legacy Java classes it may be desirable to disable the use of this library and instead use Java reflection to obtain event property values from event objects.
In the XML configuration, the optional code-generation attribute in the legacy-type section can be set to disabled as shown next.
<event-type alias="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass"> <legacy-type accessor-style="javabean" code-generation="disabled" /> </event-type>
The sample below shows how to configure this option via the API.
Configuration configuration = new Configuration(); ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy(); legacyDef.setCodeGeneration(ConfigurationEventTypeLegacy.CodeGeneration.DISABLED); config.addEventTypeAlias("MyLegacyEvent", MyLegacyEventClass.class.getName(), legacyDef);
By default the engine resolves Java event properties case sensitive. That is, property names in statements must match JavaBean-convention property names in name and case. This option controls case sensitivity per Java class.
In the configuration XML, the optional property-resolution-style attribute in the legacy-type element can be set to any of these values:
Table 2.2. Property Resolution Case Sensitivity Styles
Style Name | Description |
---|---|
case_sensitive (default) | As the default setting, the engine matches property names for the exact name and case only. |
case_insensitive | Properties are matched if the names are identical. A case insensitive search is used and will choose the first property that matches the name exactly or the first property that matches case insensitively should no match be found. |
distinct_case_insensitive | Properties are matched if the names are identical. A case insensitive search is used and will choose the first property that matches the name exactly case insensitively. If more than one 'name' can be mapped to the property an exception is thrown. |
The sample below shows this option in XML configuration, however the setting can also be changed via API:
<event-type alias="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass"> <legacy-type property-resolution-style="case_insensitive"/> </event-type>
The engine can process java.util.Map events via the sendEvent(Map map, String eventTypeAlias) method on the EPRuntime interface. Entries in the Map represent event properties. Keys must be of type java.util.String for the engine to be able to look up event property names in pattern or EQL statements. Values can be of any type. JavaBean-style objects as values in a Map can also be processed by the engine. Please see the Chapter 5, Event Representations section for details on how to use Map events with the engine.
Via configuration we provide an event type alias name for Map events for use in statements, and the event property names and types enabling the engine to validate properties in statements.
The below snippet of XML configuration configures an event named MyMapEvent.
<event-type alias="MyMapEvent"> <java-util-map> <map-property name="carId" class="int"/> <map-property name="carType" class="string"/> <map-property name="assembly" class="com.mycompany.Assembly"/> </java-util-map> </event-type>
This configuration defines the carId property of MyMapEvent events to be of type int, and the carType property to be of type java.util.String. The assembly property of the Map event will contain instances of com.mycompany.Assembly for the engine to query.
The valid list of values for the type definition via the class attribute is:
string or java.lang.String
char or java.lang.Character
byte or java.lang.Byte
short or java.lang.Short
int or java.lang.Integer
long or java.lang.Long
float or java.lang.Float
double or java.lang.Double
boolean or java.lang.Boolean
Any fully-qualified Java class name that can be resolved by the engine via Class.forName
You can also use the configuration API to configure Map event types, as the short code snippet below demonstrates.
Properties properties = new Properties(); properties.put("carId", "int"); properties.put("carType", "string"); properties.put("assembly", Assembly.class.getName()); Configuration configuration = new Configuration(); configuration.addEventTypeAlias("MyMapEvent", properties);
Finally, here is a sample EQL statement that uses the configured MyMapEvent map event. This statement uses the chassisTag and numParts properties of Assembly objects in each map.
select carType, assembly.chassisTag, count(assembly.numParts) from MyMapEvent.win:time(60 sec)
Via this configuration item the Esper engine can natively process org.w3c.dom.Node instances, i.e. XML document object model (DOM) nodes. Please see the Chapter 5, Event Representations section for details on how to use Node events with the engine.
Esper allows configuring XPath expressions as event properties. You can specify arbitrary XPath functions or expressions and provide a property name by which their result values will be available for use in expressions.
For XML documents that follow an XML schema, Esper can load and interrogate your schema and validate event property names and types against the schema information.
Nested, mapped and indexed event properties are also supported in expressions against org.w3c.dom.Node events. Thus XML trees can conveniently be interrogated using the existing event property syntax for querying JavaBean objects, JavaBean object graphs or java.util.Map events.
In the simplest form, the Esper engine only requires a configuration entry containing the root element name and the event type alias in order to process org.w3c.dom.Node events:
<event-type alias="MyXMLNodeEvent"> <xml-dom root-element-name="myevent" /> </event-type>
You can also use the configuration API to configure XML event types, as the short example below demonstrates. In fact, all configuration options available through XML configuration can also be provided via setter methods on the ConfigurationEventTypeXMLDOM class.
Configuration configuration = new Configuration(); ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM(); desc.setRootElementName("myevent"); desc.addXPathProperty("name1", "/element/@attribute", XPathConstants.STRING); desc.addXPathProperty("name2", "/element/subelement", XPathConstants.NUMBER); configuration.addEventTypeAlias("MyXMLNodeEvent", desc);
The next example presents all relevant configuration options in a sample configuration entry.
<event-type alias="AutoIdRFIDEvent"> <xml-dom root-element-name="Sensor" schema-resource="data/AutoIdPmlCore.xsd" default-namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1"> <namespace-prefix prefix="pmlcore" namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1"/> <xpath-property property-name="countTags" xpath="count(/pmlcore:Sensor/pmlcore:Observation/pmlcore:Tag)" type="number"/> </xml-dom> </event-type>
This example configures an event property named countTags whose value is computed by an XPath expression. The namespace prefixes and default namespace are for use with XPath expressions and must also be made known to the engine in order for the engine to compile XPath expressions. Via the schema-resource attribute we instruct the engine to load a schema file.
Here is an example EQL statement using the configured event type named AutoIdRFIDEvent.
select ID, countTags from AutoIdRFIDEvent.win:time(30 sec)
The schema-resource attribute takes a schema resource URL or classpath-relative filename. The engine attempts to resolve the schema resource as an URL. If the schema resource name is not a valid URL, the engine attempts to resolve the resource from classpath via the ClassLoader.getResource method using the thread context class loader. If the name could not be resolved, the engine uses the Configuration class classloader.
By configuring a schema file for the engine to load, the engine performs these additional services:
Validates the event properties in a statement, ensuring the event property name matches an attribute or element in the XML
Determines the type of the event property allowing event properties to be used in type-sensitive expressions such as expressions involving arithmatic (Note: XPath properties are also typed)
Matches event property names to either element names or attributes
If no schema resource is specified, none of the event properties specified in statements are validated at statement creation time and their type defaults to java.lang.String. Also, attributes are not supported if no schema resource is specified and must thus be declared via XPath expression.
The xpath-property element adds event properties to the event type that are computed via an XPath expression. In order for the XPath expression to compile, be sure to specify the default-namespace attribute and use the namespace-prefix to declare namespace prefixes.
XPath expression properties are strongly typed. The type attribute allows the following values. These values correspond to those declared by javax.xml.xpath.XPathConstants.
number (Note: resolves to a double)
string
boolean
Esper allows invocations of static Java library functions as outlined in Section 6.14, “Single-row Function Reference”. This configuration item can be set to allow a partial rather than a fully qualified class name in such invocations. The imports work in the same way as in Java files, so both packages and classes can be imported.
select Math.max(priceOne, PriceTwo) // via configuration equivalent to select java.lang.Math.max(priceOne, priceTwo)
Esper auto-imports the following Java library packages if no other configuration is supplied. This list is replaced with any configuration specified in a configuration file or through the API.
java.lang.*
java.math.*
java.text.*
java.util.*
In an XML configuration file the auto-import configuration may look as below. Note that all configuration options are available through the Configuration API as well.
<auto-import import-name="com.mycompany.mypackage.*"/> <auto-import import-name="com.mycompany.myapp.MyUtilityClass"/>
Esper has the capability to join event streams against historical data sources, such as a relational database. This section describes the configuration entries that the engine requires to access data stored in your database. Please see Section 6.13, “Joining Relational Data via SQL” for information on the use of EQL queries that include historical data sources.
EQL queries that poll data from a relational database specify the name of the database as part of the EQL statement. The engine uses the configuration information described here to resolve the database name in the statement to database settings. The required and optional database settings are summarized below.
Database connections can be obtained via JDBC javax.xml.DataSource or alternatively via java.sql.DriverManager. Either one of these methods to obtain new database connections is a required configuration.
Optionally, JDBC connection-level settings such as auto-commit, transaction isolation level, read-only and the catalog name can be defined.
Optionally, a connection lifecycle can be set to indicate to the engine whether the engine must retain connections or must obtain a new connection for each lookup.
Optionally, define a cache policy to allow the engine to retrieve data from a query cache, reducing the number of query executions.
Some of the settings can have important performance implications that need to be carefully considered in relationship to your database software, JDBC driver and runtime environment. This section attempts to outline such implications where appropriate.
The sample XML configuration file in the "etc" folder can be used as a template for configuring database settings. All settings are also available by means of the configuration API through the classes Configuration and ConfigurationDBRef.
The snippet of XML below configures a database named mydb1 to obtain connections via a javax.sql.DataSource. The datasource-connection element instructs the engine to obtain new connections to the database mydb1 by performing a lookup via javax.naming.InitialContext for the given object lookup name. Optional environment properties for the InitialContext are also shown in the example.
<database-reference name="mydb1"> <datasource-connection context-lookup-name="java:comp/env/jdbc/mydb"> <env-property name="java.naming.factory.initial" value ="com.myclass.CtxFactory"/> <env-property name="java.naming.provider.url" value ="iiop://localhost:1050"/> </datasource-connection> </database-reference>
To help you better understand how the engine uses this information to obtain connections, we have included the logic below.
if (envProperties.size() > 0) { initialContext = new InitialContext(envProperties); } else { initialContext = new InitialContext(); } DataSource dataSource = (DataSource) initialContext.lookup(lookupName); Connection connection = dataSource.getConnection();
The next snippet of XML configures a database named mydb2 to obtain connections via java.sql.DriverManager. The drivermanager-connection element instructs the engine to obtain new connections to the database mydb2 by means of Class.forName and DriverManager.getConnection using the class name, URL and optional username, password and connection arguments.
<database-reference name="mydb2"> <drivermanager-connection class-name="my.sql.Driver" url="jdbc:mysql://localhost/test?user=root&password=mypassword" user="myuser" password="mypassword"> <connection-arg name="user" value ="myuser"/> <connection-arg name="password" value ="mypassword"/> <connection-arg name="somearg" value ="someargvalue"/> </drivermanager-connection> </database-reference>
The username and password are shown in multiple places in the XML only as an example. Please check with your database software on the required information in URL and connection arguments.
Additional connection-level settings can optionally be provided to the engine which the engine will apply to new connections. When the engine obtains a new connection, it applies only those settings to the connection that are explicitly configured. The engine leaves all other connection settings at default values.
The below XML is a sample of all available configuration settings. Please refer to the Java API JavaDocs for java.sql.Connection for more information to each option or check the documentation of your JDBC driver and database software.
<database-reference name="mydb2"> ... configure data source or driver manager settings... <connection-settings auto-commit="true" catalog="mycatalog" read-only="true" transaction-isolation="1" /> </database-reference>
The read-only setting can be used to indicate to your database engine that SQL statements are read-only. The transaction-isolation and auto-commit help you database software perform the right level of locking and lock release. Consider setting these values to reduce transactional overhead in your database queries.
By default the engine retains a separate database connection for each started EQL statement. However, it is possible to override this behavior and require the engine to obtain a new database connection for each lookup, and to close that database connection after the lookup is completed. This often makes sense when you have a large number of EQL statements and require pooling of connections via a connection pool. If your runtime environment includes an application server, the connection pool may be exposed as a DataSource.
The XML for this option is below. The connection lifecycle allows the following values: pooled and retain.
<database-reference name="mydb2"> ... configure data source or driver manager settings... <connection-lifecycle value="pooled"/> </database-reference>
Cache settings can dramatically reduce the number of database queries that the engine executes for EQL statements. If no cache setting is specified, the engine does not cache query results and executes a separate database query for every event.
Caches store the results of database queries and make these results available to subsequent queries using the exact same query parameters as the query for which the result was stored. If your query returns one or more rows, the cache keep the result rows of the query keyed to the parameters of the query. If your query returns no rows, the cache also keeps the empty result. Query results are held by a cache until the cache entry is evicted. The strategies available for evicting cached query results are listed next.
The least-recently-used (LRU) cache is configured by a maximum size. The cache discards the least recently used query results first once the cache reaches the maximum size.
The XML configuration entry for a LRU cache is as below. This entry configures an LRU cache holding up to 1000 query results.
<database-reference name="mydb"> ... configure data source or driver manager settings... <lru-cache size="1000"/> </database-reference>
The expiry time cache is configured by a maximum age in seconds and a purge interval. The cache discards (on the get operation) any query results that are older then the maximum age so that stale data is not used. If the cache is not empty, then every purge interval number of seconds the engine purges any expired entries from the cache.
The XML configuration entry for an expiry-time cache is as follows. The example configures an expiry time cache in which prior query results are valid for 60 seconds and which the engine inspects every 2 minutes to remove query results older then 60 seconds.
<database-reference name="mydb"> ... configure data source or driver manager settings... <expiry-time-cache max-age-seconds="60" purge-interval-seconds="120"/> </database-reference>
In multithreaded environments, this setting controls whether dispatches of statement result events to listeners preserve the ordering in which a statement processes events. By default the engine guarantees that it delivers a statement's result events to statement listeners in the order in which the result is generated. This behavior can be turned off via configuration as below.
The next code snippet shows how to control this feature:
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setListenerDispatchPreserveOrder(false); engine = EPServiceProviderManager.getDefaultProvider(config);
An the XML configuration file can also control this feature by adding the following elements:
<engine-settings> <defaults> <threading> <listener-dispatch preserve-order="false" timeout-msec="2000"/> <insert-into-dispatch preserve-order="false"/> </threading> </defaults> </engine-settings>
As discussed, by default the engine can temporarily block a thread when delivering result events to listeners in order to preserve the order in which results are generated by a given statement. The maximum time the engine blocks a thread can also be configured, and by default is set to 1 second.
In multithreaded environments, this setting controls whether insert-into streams preserve the order of events inserted into them by one or more statements, allowing statements that consume other statement's events to behave deterministic.
By default, the engine acquires a lock per insert-into stream when a statement makes events available to further statements using the insert into clause. The lock allows generated events to be processed by further statements consuming the insert-into stream in the order the generating statement(s) produce events. This allows statements that require order (such as pattern detection, previous and prior functions) to behave deterministically.
The setting can be changed via the configuration API and XML as shown in the prior section.
This option can be used to disable the internal timer thread and such have the application supply external time events, as well as to set a timer resolution.
The next code snippet shows how to disable the internal timer thread via the configuration API:
Configuration config = new Configuration(); config.getEngineDefaults().getThreading().setInternalTimerEnabled(false);
This snippet of XML configuration leaves the internal timer enabled (the default) and sets a resolution of 200 milliseconds (the default is 100 milliseconds):
<engine-settings> <defaults> <threading> <internal-timer enabled="true" msec-resolution="200"/> </threading> </defaults> </engine-settings>
As discussed in Section 2.4.1.5, “Case Sensitivity and Property Names” this setting controls case sensitivity for Java event class properties of all Java classes as a default, rather then at a class level.
The next code snippet shows how to control this feature via the API:
Configuration config = new Configuration(); config.getEngineDefaults().getEventMeta().setClassPropertyResolutionStyle( Configuration.PropertyResolutionStyle.CASE_INSENSITIVE);
The engine by default attempts to optimize resource usage and thus re-uses or shares views between statements that declare same views. However, in multi-threaded environments, this can lead to reduced concurrency as locking for shared view resources must take place. Via this setting this behavior can be turned off for higher concurrency in multi-threaded processing.
The next code snippet outlines the API to turn off view resource sharing between statements:
Configuration config = new Configuration(); config.getEngineDefaults().getViewResources().setShareViews(false);
By default, the engine does not produce debug output for the event processing execution paths even when Log4j or Logger configurations have been set to output debug level logs. To enable debug level logging, set this option in the configuration as well as in your Log4j configuration file.
The API to use to enable debug logging is shown here:
Configuration config = new Configuration(); config.getEngineDefaults().getLogging().setEnableExecutionDebug(true);
Note: this is a configuration option that applies to all engine instances of a given Java module or VM.