Package org.apache.commons.jexl3

package org.apache.commons.jexl3
Provides a framework for evaluating JEXL expressions and scripts.

Introduction

JEXL is a library intended to facilitate the implementation of dynamic and scripting features in applications and frameworks.

The entry point is always a JexlEngine, created and configured through a JexlBuilder. An engine is thread-safe and meant to be created once and shared; it produces JexlExpression (single expressions) and JexlScript (multi-statement scripts) which are evaluated against a JexlContext carrying the variable bindings. A JxltEngine adds a JSP/JSF-like templating layer on top of the same engine.

Security disclaimer. Neither JexlPermissions.RESTRICTED nor JexlPermissions.SECURE is exhaustive, and neither must be considered completely safe or sufficient on its own for executing untrusted user input. They are hardened baselines, not guarantees. Any application that evaluates untrusted scripts must define its own tailored, strict whitelist of exactly the classes, methods and fields its scripts legitimately need - ideally by composing on top of JexlPermissions.NONE (which denies everything) via JexlPermissions.create(java.lang.String...) - and audit the result with JexlPermissions.logging().

Security note (since 3.7): a bare new JexlBuilder().create() engine is hardened by default. Its permissions are JexlPermissions.SECURE (only a small, safe allow-list of java.lang/java.math/java.util types is reachable) and its default JexlFeatures disable new(...), global side-effects, pragmas and annotations (lexical scoping is on; loops remain available to scripts). Relax these deliberately - see the configuration section - rather than by accident.

Permissions and features can be set programmatically on the JexlBuilder or loaded from a YAML document through JexlConfigLoader, which keeps security policy out of code and in configuration: 

 try (InputStream in = getClass().getResourceAsStream("/jexl.yaml")) {
     JexlEngine jexl = JexlConfigLoader.load(in).create();
 }

A ready-made jexl.yaml that restores the pre-3.7 behavior (RESTRICTED permissions, the full feature set) is bundled as a migration aid; see JexlConfigLoader for the full schema.

A Brief Example

In its simplest form, JEXL merges an JexlExpression with a JexlContext when evaluating expressions. An expression is created using JexlEngine.createExpression(String), passing a String containing valid JEXL syntax. A simple JexlContext can be created using a MapContext instance; a map of variables that will be internally wrapped can be optionally provided through its constructor. The following example binds a 'name' variable, calls a String method on it and concatenates the result. It relies only on a java.lang.String and arithmetic, so it runs as-is under the default (SECURE) permissions - no configuration needed. 

 // Create a JexlEngine (could reuse one instead)
 JexlEngine jexl = new JexlBuilder().create();
 // An expression using a variable, a String method call and concatenation:
 String jexlExp = "'Hello ' + name + ', your name has ' + name.length() + ' letters'";
 JexlExpression e = jexl.createExpression( jexlExp );
 // Create a context and add data
 JexlContext jc = new MapContext();
 jc.set("name", "John");
 // Now evaluate the expression; result is "Hello John, your name has 4 letters"
 Object o = e.evaluate(jc);

To expose your own classes (a Car bean, a service, ...) to scripts, you must permit them explicitly: the default SECURE permissions deny application types, and passing an instance through the JexlContext does not bypass that gate - the introspector still checks the object's class. Grant access by composing your package into the permission set (e.g. new JexlBuilder().permissions(JexlPermissions.RESTRICTED.compose("com.example.app +{}"))) or choose a broader base; see the configuration section on permissions.

Using JEXL

The API is composed of three levels addressing different functional needs:
  • Dynamic invocation of setters, getters, methods and constructors
  • Script expressions known as JEXL expressions
  • JSP/JSF like expression known as JXLT expressions

Important note

The public API classes reside in the 2 packages:
  • org.apache.commons.jexl3
  • org.apache.commons.jexl3.introspection

The following packages follow a "use at your own maintenance cost" policy; these are only intended to be used for extending JEXL. Their classes and methods are not guaranteed to remain compatible in subsequent versions. If you think you need to use directly some of their features or methods, it might be a good idea to check with the community through the mailing list first.

  • org.apache.commons.jexl3.parser
  • org.apache.commons.jexl3.scripting
  • org.apache.commons.jexl3.internal
  • org.apache.commons.jexl3.internal.introspection

Note that the org.apache.commons.jexl3.scripting package implements the JSR-223 (javax.script) integration through JexlScriptEngine.

Dynamic Invocation

These functionalities are close to the core level utilities found in BeanUtils. For basic dynamic property manipulations and method invocation, you can use the following set of methods:

The following example illustrate their usage: 
 // test outer class
 public static class Froboz {
     int value;
     public Froboz(int v) { value = v; }
     public void setValue(int v) { value = v; }
     public int getValue() { return value; }
 }

 // test inner class
 public static class Quux {
     String str;
     Froboz froboz;

     public Quux(String str, int fro) {
       this.str = str;
       froboz = new Froboz(fro);
     }

     public Froboz getFroboz() { return froboz; }
     public void setFroboz(Froboz froboz) { this.froboz = froboz; }
     public String getStr() { return str; }
     public void setStr(String str) { this.str = str; }
 }

 // test API - Quux and Froboz are in your own package; compose it into the permissions.
 JexlEngine jexl = new JexlBuilder()
     .permissions(JexlPermissions.SECURE.compose("com.example +{}"))
     .create();
 Quux quux = jexl.newInstance(Quux.class, "xuuq", 100);
 jexl.setProperty(quux, "froboz.value", Integer.valueOf(100));
 Object o = jexl.getProperty(quux, "froboz.value");
 assertEquals("Result is not 100", Integer.valueOf(100), o);
 jexl.setProperty(quux, "['froboz'].value", Integer.valueOf(1000));
 o = jexl.getProperty(quux, "['froboz']['value']");
 assertEquals("Result is not 1000", Integer.valueOf(1000), o);

Expressions and Scripts

If your needs require simple expression evaluation capabilities, the core JEXL features will most likely fit. The main methods are:

The following example illustrates their usage: 
 // new(...) needs the newInstance feature; Quux/Froboz need explicit permissions.
 JexlEngine jexl = new JexlBuilder()
     .permissions(JexlPermissions.SECURE.compose("com.example +{}"))
     .features(JexlFeatures.createDefault())
     .create();
 JexlContext jc = new MapContext();
 jc.set("quuxClass", quux.class);
 JexlExpression create = jexl.createExpression("quux = new(quuxClass, 'xuuq', 100)");
 JexlExpression assign = jexl.createExpression("quux.froboz.value = 10");
 JexlExpression check = jexl.createExpression("quux[\"froboz\"].value");
 Quux quux = (Quux) create.evaluate(jc);
 Object o = assign.evaluate(jc);
 assertEquals("Result is not 10", Integer.valueOf(10), o);
 o = check.evaluate(jc);
 assertEquals("Result is not 10", Integer.valueOf(10), o);

Unified Expressions and Templates

If you are looking for JSP-EL like and basic templating features, you can use Expression from a JxltEngine.

The main methods are: The following example illustrates their usage: 
 JexlEngine jexl = new JexlBuilder().create();
 JxltEngine jxlt = jexl.createJxltEngine();
 JxltEngine.Expression expr = jxlt.createExpression("Hello ${user}");
 String hello = expr.evaluate(context).toString();

JexlExpression, JexlScript, Expression and Template: summary

JexlExpression

These are the most basic form of JexlEngine expressions and only allow for a single command to be executed and its result returned. If you try to use multiple commands, it ignores everything after the first semi-colon and just returns the result from the first command.

Also note that expressions are not statements (which is what scripts are made of) and do not allow using the flow control (if, while, for), variables or lambdas syntactic elements.

JexlScript

These allow you to use multiple statements and you can use variable assignments, loops, calculations, etc. More or less what can be achieved in Shell or JavaScript at its basic level. The result from the last command is returned from the script.

JxltEngine.Expression

These are ideal to produce "one-liner" text, like a 'toString()' on steroids. To get a calculation you use the EL-like syntax as in ${someVariable}. The expression that goes between the brackets behaves like a JexlScript, not an expression. You can use semi-colons to execute multiple commands and the result from the last command is returned from the script. You also have the ability to use a 2-pass evaluation using the #{someScript} syntax.

JxltEngine.Template

These produce text documents. Each line beginning with '$$' (as a default) is considered JEXL code and all others considered as JxltEngine.Expression. Think of those as simple Velocity templates. A rewritten MudStore initial Velocity sample looks like this: 


 <html>
 <body>
 Hello ${customer.name}!
 <table>
 $$      for(var mud : mudsOnSpecial ) {
 $$          if (customer.hasPurchased(mud) ) {
 <tr>
 <td>
 ${flogger.getPromo( mud )}
 </td>
 </tr>
 $$          }
 $$      }
 </table>
 </body>
 </html>

JEXL Configuration

Almost everything is configured through a JexlBuilder before the engine is created. The builder controls four largely orthogonal concerns:

  • Permissions (JexlPermissions) - what classes, methods and fields scripts may reach through reflection.
  • Features (JexlFeatures) - which syntactic constructs are accepted at parse time.
  • Options (JexlOptions) - how the interpreter behaves at runtime (strict, silent, safe, cancellable, lexical, math).
  • Bindings (namespaces, imports, arithmetic, class loader, caches) - the environment scripts execute against.

For a declarative alternative, JexlConfigLoader builds a pre-configured JexlBuilder from a YAML document covering permissions, features, arithmetic, namespaces and imports - convenient for externalizing configuration or restoring pre-3.7 defaults.

Security: permissions, sandbox and @NoJexl

JexlPermissions is the primary security gate; it controls which packages, classes and members the introspection layer will expose. Four constants are provided: JexlPermissions.UNRESTRICTED (everything), JexlPermissions.RESTRICTED (a broad but curated allow-list), JexlPermissions.SECURE (the minimal safe allow-list and the default since 3.7) and JexlPermissions.NONE (deny everything). A permission set is configured with JexlBuilder.permissions(org.apache.commons.jexl3.introspection.JexlPermissions) and can be composed from a textual DSL through JexlPermissions.parse(java.lang.String...) or RESTRICTED.compose("com.example.api +{}"). To build a closed-world, deny-by-default policy from scratch, start from NONE (or JexlPermissions.create(java.lang.String...)) and compose only what scripts need, e.g. JexlPermissions.create("com.example.api +{}"). To diagnose what a permission set allows or denies under your workload, wrap it with JexlPermissions.logging().

Two finer-grained mechanisms complement permissions: the NoJexl annotation completely shields annotated classes and members from introspection at the source level, while a JexlSandbox - set through JexlBuilder.sandbox(org.apache.commons.jexl3.introspection.JexlSandbox) - gives per-class allow/deny control over properties and methods at configuration time.

Language features (parse time)

JexlFeatures restricts the grammar a script may use; violations are reported as JexlException.Feature when the script is compiled, before any evaluation. Among the toggles are newInstance (new(...)), loops, sideEffect/sideEffectGlobal, lambda, methodCall, structuredLiteral, pragma, annotation and lexical scoping. A feature set is applied with JexlBuilder.features(org.apache.commons.jexl3.JexlFeatures); convenient bases are JexlFeatures.createDefault(), JexlFeatures.createScript(), JexlFeatures.createAll() and JexlFeatures.createNone().

Since expressions are parsed as a single expression, statement-level features (loops, multiple statements, ...) never apply to them regardless of the feature set; they are relevant to scripts only.

Runtime options

JexlOptions carries the behavioral flags evaluated at runtime. They can be set as engine defaults either through the dedicated builder shortcuts or by mutating the builder's option set returned by JexlBuilder.options():

These engine defaults can be overridden per evaluation: a JexlContext that also implements JexlContext.OptionsHandle supplies a fresh JexlOptions for each evaluation. (The legacy JexlEngine.Options interface is deprecated in favor of this mechanism.)

Namespaces, imports and arithmetic

JexlBuilder.namespaces(java.util.Map) registers your own classes or instances as namespaces, exposing their methods as prefix:function(...) calls. 


 public static class MyMath {
   public double cos(double x) {
     return Math.cos(x);
   }
 }
 Map<String, Object> funcs = new HashMap<String, Object>();
 funcs.put("math", new MyMath());
 // MyMath is in your own package; compose it into the permissions.
 JexlEngine jexl = new JexlBuilder()
     .namespaces(funcs)
     .permissions(JexlPermissions.SECURE.compose("com.example +{}"))
     .create();
 JexlContext jc = new MapContext();
 jc.set("pi", Math.PI);
 JexlExpression e = jexl.createExpression("math:cos(pi)");
 Object o = e.evaluate(jc);
 assertEquals(Double.valueOf(-1), o);

If the namespace is a Class and that class declares a constructor that takes a JexlContext (or a class extending JexlContext), one namespace instance is created on first usage in an expression; this instance lifetime is limited to the expression evaluation.

JexlBuilder.imports(java.util.Collection) declares packages whose classes can be referenced by their unqualified name in new(...) and type references, mimicking Java imports. JexlBuilder.arithmetic(org.apache.commons.jexl3.JexlArithmetic) substitutes a custom JexlArithmetic - see customization.

Static & Shared Configuration

Both JexlEngine and JxltEngine are thread-safe, most of their inner fields are final; the same instance can be shared between different threads and proper synchronization is enforced in critical areas (introspection caches).

The library-wide defaults applied by a bare new JexlBuilder() are themselves configurable through static setters: JexlBuilder.setDefaultPermissions(org.apache.commons.jexl3.introspection.JexlPermissions), JexlBuilder.setDefaultFeatures(org.apache.commons.jexl3.JexlFeatures), and JexlBuilder.setDefaultOptions(String...) (runtime evaluation flags such as strict, safe, and cancellable; see JexlOptions.setDefaultFlags(String...)). The pre-3.7 permissive feature set is exposed as JexlBuilder.FULL so an application can restore the legacy behavior process-wide in one place.

Of particular importance is JexlBuilder.loader(java.lang.ClassLoader) which indicates to the JexlEngine being built which class loader to use to solve a class name; this directly affects how JexlEngine.newInstance and the 'new' script method operates.

This can also be very useful in cases where you rely on JEXL to dynamically load and call plugins for your application. To avoid having to restart the server in case of a plugin implementation change, you can call JexlEngine.setClassLoader(java.lang.ClassLoader) and all the scripts created through this engine instance will automatically point to the newly loaded classes.

Caches & debugging

JexlEngine and JxltEngine expression caches can be configured as well. If you intend to use JEXL repeatedly in your application, these are worth configuring since expression parsing is quite heavy. Note that all caches created by JEXL are held through SoftReference; under high memory pressure, the GC will be able to reclaim those caches and JEXL will rebuild them if needed. By default, a JexlEngine does create a cache for "small" expressions and a JxltEngine does create one for Expression.

JexlBuilder.cache(int) sets how many expressions can be simultaneously cached by the JEXL engine, JexlBuilder.cacheThreshold(int) caps the size of cached expressions, and JexlBuilder.cacheFactory(java.util.function.IntFunction) lets you supply an alternative JexlCache implementation. JxltEngine allows defining the cache size through its constructor.

JexlBuilder.debug(boolean) makes stack traces carried by JexlException more meaningful; in particular, these traces will carry the exact caller location the Expression was created from.

Dynamic Configuration

Beyond options, a JexlContext can implement a number of handles to override engine behavior on a per-evaluation basis. The MapContext/ObjectContext implementations cover the common cases; for finer control, implement one or more of:

JEXL Customization

The JexlContext, JexlBuilder and JexlOptions are the most likely interfaces you'll want to implement for customization. Since they expose variables and options, they are the primary targets. Before you do so, have a look at ObjectContext which may already cover some of your needs.

JexlArithmetic is the class to derive if you need to change how operators behave or add types upon which they operate. There are 3 entry points that allow customizing the type of objects created:

You can also overload operator methods; by convention, each operator has a method name associated to it. If you overload some in your JexlArithmetic derived implementation, these methods will be called when the arguments match your method signature. For example, this would be the case if you wanted '+' to operate on arrays; you'd need to derive JexlArithmetic and implement a public Object add(Set<?> x, Set<?> y) method. Note however that you can not change the operator precedence. The list of operator / method matches is described in JexlOperator:

You can also add methods to overload property getters and setters operators behaviors. Public methods of the JexlArithmetic instance named propertyGet/propertySet/arrayGet/arraySet are potential overrides that will be called when appropriate. The following table is an overview of the relation between a syntactic form and the method to call where V is the property value class, O the object class and P the property identifier class (usually String or Integer).

Property Accessors
Expression Method Template
foo.property public V propertyGet(O obj, P property);
foo.property = value public V propertySet(O obj, P property, V value);
foo[property] public V arrayGet(O obj, P property, V value);
foo[property] = value public V arraySet(O obj, P property, V value);

You can also override the base operator methods, those whose arguments are Object which gives you total control.

Extending JEXL

If you need to make JEXL treat some objects in a specialized manner or tweak how it reacts to some settings, you can derive most of its inner-workings. The classes and methods are rarely private or final - only when the inner contract really requires it. However, using the protected methods and internal package classes imply you might have to re-adapt your code when new JEXL versions are released.

Engine can be extended to let you capture your own configuration defaults regarding cache sizes and various flags. Implementing your own JexlCache - instead of the default soft-reference based one - would be another possible extension.

Interpreter is the class to derive if you need to add more features to the evaluation itself; for instance, you want pre- and post- resolvers for variables or nested scopes for for variable contexts.

Uberspect is the class to derive if you need to add introspection or reflection capabilities for some objects, for instance adding factory based support to the 'new' operator. The code already reflects public fields as properties on top of Java-beans conventions.