Kogito, ergo Rules — Part 1: Bringing Drools Further

The Kogito initiative is our pledge to bring our business automation suite to the cloud and the larger Kubernetes ecosystem. But what does this mean for our beloved rule engine, Drools? In this post we introduce modular rule bases using rule units: a feature that has been experimental for a while in Drools 7, but that will be instrumental for Kogito, where it will play a much bigger role. This is the first post of a series where we will give you an overview of this feature (read part 2)

Bringing Drools Further

Droolsis our state-of-the-art, high-performance, feature-rich open source rule engine. People love it because it is a swiss-army knife to the many problems that can be solved using rule-based artificial intelligence. But as the computer programming landscape evolves, we need to think of ways to bring further Drools as well. As you may already know, Kogito is our effort to make Drools and jBPM really cloud-native, and well-suited for serverless deployments: we are embracing the Quarkus framework and GraalVM’s native binary compilation for super-fast startup times and low memory footprint; but we are not stopping there.

The way we want to bring further Drools evolution is twofold: on the one hand, we want to make our programming model easier to reason about, by providing better ways to define boundaries in a rule base with a better concept of module; on the other hand, the concept of modular programming dates back at least to the 1970s and to Parnas’ original seminal paper. Needless to say, if our contribution stopped there, we would be bringing nothing new to the plate. In the last few years, computing has evolved, slowly but steadily embracing the multicore and distributed revolution; yet, to this day, many general-purpose programming languages do not really make it simple to write parallel or distributed programs. rule-based programming system we have the chance to propose something different: a rule engine that is great when stand-alone, but outstanding in the cloud.

Modular Rule Bases. As you already know, Drools provides a convenient way to partition set of rules into knowledge bases. Such knowledge bases can be composed together, yielding larger sets of rules. When a knowledge base is instantiated (the so-called session), rules are put together in the same execution environment (the production memory), and values (the facts) are all inserted together in the same working memory.

This model is very simple and powerful but in some senses it is also very limited. It is very simple, because, as a user of the rule base, you just worry about your data: the values are inserted into the working memory, and the engine does its magic. It is very powerful, because, as a rule author, you can rely upon the rules you have written to realize complex flows of reasoning, without worrying about how and when they will trigger.

At the same time, such an execution model lacks all of the principles, that over the years we have been learning are good programming practice. For instance, there is no proper notion of a module: it is not possible to perfectly isolate one rule from another, or to properly partition the working memory. As the rule base scales up in complexity, it may become harder to understand which rules trigger and why. In some senses, it is as if you were programming in an odd world where proper encapsulation of state does not exist, as if years of programming language evolution had not happened.

Object-Oriented Programming. The term object-oriented programming has been overloaded over the years to mean a lot of different things; it has to do both with inheritance, with encapsulation of state, with code reuse, with polymorphism. All these terms get often confused, but they are not really related: you can reuse code without inheritance, you can encapsulate state without objects, you can write polymorphic code without classes. Very recent, imperative programming languages such as Go and Rust do not come with proper classes, yet they support a form of object-orientation; there is even a beautiful 2015 talk from C++’s dad, Bjarne Stroustrup, showing how his child supports object-orientation without inheritance.

Alan Kay, who fathered the term in his Smalltalk days at Xerox, in his inspiring lecture at OOPSLA 1997 said «I made up the term “object-oriented”, and I can tell you I did not have C++ in mind». In fact, the idea of objects that Alan Kay pioneered was more similar to the concept of actors and microservices. In proper object-oriented programming, objects encapsulate their internal state and expose their behavior by exchanging messages (usually called methods) with the external world.

Today actor systems have seen a renaissance, message buses are very central to what today we call reactive programming, microservices are almost given for granted. So, we wondered, what would it mean for Drools to become a first-class citizen of this new programming landscape?

Kogito, ergo Cloud

In the next post we will see our take on rule-based, modular programming, using rule units. Rule units will provide an alternative to plain knowledge base composition and an extended model of execution. We believe that rule units will make room for a wider spectrum of use cases, including parallel and distributedarchitectures. Stay tuned to read how they fit in the Kogito story, and the exciting possibilities that they may open for the future of our automation platform.




Kogito, ergo Rules — Part 2: An All-Encompassing Execution Model for Rules

This is the second post of a series of updates on the Kogito initiative and our efforts to bring Drools to the cloud. In this post we delve into the details of rule units and show you why we are excited about them.

An All-Encompassing Execution Model for Rules

If you’ve been carefully scrutinising the Drools manual looking for new features at every recent release, you may have noticed that the term rule unit has been sitting there for a while, as an extremely experimental feature. In short, a rule unit is both a module for rules and a unit of execution—the reason why we are not calling them modules is to avoid confusion with JVM modules. In Kogito, we are revisiting and expanding upon our original prototype.
A rule unit collects a set of rules together with the description of the working memory such rules act upon. The description of the working memory is written as a regular Java class, with DataSource fields. Each data source represents a typed partition of the working memory, and different types of data sources exist, with different features. For instance, in the following example we used an append-only data source, called data stream.

Rules of a given rule unit are collected in DRL files with the unit declaration

Each rule in a unit has visibility over all the data sources that have been declared in the corresponding class. In fact, the class and the collection of DRL files of a unit form a whole: you can think of such a whole as of one single class where fields are globals that are scoped to the current unit, and methods are rules. In fact, the use of fields supersedes the use of DRL globals.
A rule unit is submitted for execution to a scheduler. Rule units may decide to yield their execution to other rule units, effectively putting them into execution. For instance:

But rule units may be also put in a long-running state. In this case, other rule units may be run concurrently at the same time; because DataSources can be shared across units, units can be coordinated by exchanging messages.
Consider the following example:

In a certain way, rule units behave as “actors” exchanging messages. However, in a very distinctive way, rule units allow for much more complex chains of executions, that are proper to rule-based reasoning. For instance, consider this example from Akka’s manual:

As you can see, pattern matches in Akka are strictly over single messages. This is unsurprising, because actors process one message at a time. In a rule engine, we are allowed to write several rules, reacting upon the entire state of the working memory at the execution time: this significantly departs from a pure actor model design, but at the same time gives a great deal of flexibility in the way you may write the business logic of your application.

Data Sources

It is worth to spend a few words on data sources as well. The data source construct can be seen as both a partition and an abstraction over the traditional working memory. Different kinds of data sources will be available: full-featured data stores may support to add, remove and update values, allowing for more traditional operations over the working memory; while the more constrained append-only data streams would be easier to integrate with external data sources and data sinks, such as Camel connectors; such constraints would be also valuable to enable more advanced use cases, such as parallel, thread-safe execution and persisted shared channel (e.g.: Kafka) across nodes of an OpenShift cluster, realizing a fully distributed rule engine.
 

Kogito: ergo Cloud

The parallel and distributed use cases are intriguing, but we need to get there with baby steps. However, this does not mean that the first steps won’t be as exciting in their own way.

For Kogito we want to stress the cloud-native, stateless use case, where control flow is externalized using processes and, with the power of Quarkus we can compile this into super-fast native binaries. This is why in the next few weeks we will complete and release rule units for automated REST service implementation.

In this use case, the typed, Java-based declaration of a rule unit is automatically mapped to the signature of a REST endpoint. POSTing to the endpoint implies instantiating the unit, inserting data into the data sources, firing rules, returning the response payload. The response is computed using a user-provided query. For instance, consider this example:

Users may post events using the auto-generated /monitoring-service endpoint.

the reply will be the result of the query. In our case:

Cloudy with a Chance of Rules

We have presented our vision for the next generation of our rule engine in Kogito and beyond. The stateless use case is only the first step towards what we think will be a truly innovative take on rule engines. In the following months we will work on delivering better support for scheduling and deploying units in parallel (local) and distributed (on Openshift), so stay tuned for more. In the meantime, we do want to hear from you about the direction we are taking.

The future of Drools is cloudy… and bright!

drools.js: Towards a Polyglot Drools on GraalVM (with Bonus Tech-Lead Prank)

Image courtesy of Massimiliano Dessì

You can find the full source code for this blog post in the submarine-examples repository.

Different programming languages are better for different purposes. Imagine how hard would it be to query a database using an imperative language: luckily, we use SQL for that. Now, imagine how useless would a rule engine be, if defining rules were not convenient! This is the reason why Drools comes with its own custom language, the DRL. The Drools Rule Language is in a so-called domain-specific language, a special-purpose programming language specifically designed to make interaction with a rule engine easier.

In particular, a rule is made of two main parts, the condition and the consequence.

The condition is a list of logic predicates, usually pattern matches, while the consequence is written using an imperative language, usually Java.

An Abstract Rule Engine

Rules are what really make a rule engine. After all, that’s what a rule engine does: processing rules. Thus, it might sound kind of logical for the engine to be a bit entangled with the language for rule definitions. Our engine is not specially tied to the DRL; but it used to.

In the last year or so, we spent a lot of time unbundling the innards of the DRL from the guts of the Drools core. The result of this effort is what we called the Canonical Model; that is, an abstract representation of the components that make up a rule engine, including rule definitions. Incidentally, this also paved the way for supporting GraalVM and the Quarkus framework; but our goal was also different. We wanted to abstract our engine from the rule language.

Internally, the DRL is now translated into the canonical representation; but, as we said previously, this canonical model is described using Java code. While this representation is not currently intended to be hand-coded, it is very possible to do so. The following is a simple rewriting of the previous DRL rule.

As you can see, although the rule definition is now embedded in a Java “host” language, it still shows the main features of a DRL definition, namely, the logic condition and the imperative consequence (introduced by the on…execute pair) In other words, this is a so-called embedded or internal domain-specific language.

A small disclaimer applies: the code above works, but our translator takes extra steps for best performance, such as introducing indexes. In fact, one of the reasons why we do not intend this API for public consumption is that, currently, a naive rewrite like this may produce inefficient rules.

A Polyglot Automation Platform

As part of our journey experimenting with our programming model, we wanted to see whether it was feasible to interact with our engine using different programming languages. DRL aside, the canonical model rule definition API is pure-Java.

But GraalVM is not only a tool to generate native binaries: in fact, this is only one of the capabilities of this terrific project. GraalVM is, first and foremost, the one VM to rule them all: that is, a polyglot runtime, with first-class support for both JVM languages and many other dynamic programming languages, with a state-of-the-art JIT compiler, that easily compares or exceeds the performance of the industry standards. For instance, there is already support for R, Ruby, JavaScript and Python; and, compared to writing a JIT compiler from scratch, the Truffle framework makes it terribly easy to write your own, and fine-tuning it to perfection.

GraalVM gave us a great occasion to show how easy could it be to make Drools polyglot, and, above all, to play an awful practical joke on our beloved, hard-working, conference-speaking, JavaScript-hating, resident Java Champion and tech lead Mario!

Enter drools.js:

And here’s a picture of Mario screaming in fear at the monster we have created



Jokes aside, this experiment is a window over one of the many possible futures of our platform. The world of application development today is polyglot. We cannot ignore this, and we are trying to understand how to reach a wider audience with our technologies, be it our rule engine, or our workflow orchestration engine; in fact, we are doing the same experiments with other parts of the platform, such as jBPM.

jBPM provides its own DSL for workflow definition. Although this is, again, work in progress, it shows a lot of promise as well. Behold: jbpm.js!

Conclusion

The DRL has served its purpose for a very long time, but we are already providing different ways to interact with our powerful engine, such as DMN and PMML; but power users will always want to reach for finer tuning and write their own rules.

The canonical model API is still a work-in-progress, and, above all, an internal API that is not intended for human consumption; but, if there is enough interest, we do plan to work further to provide a more convenient embedded DSL for rule definition. Through the power of GraalVM, we will be able to realize an embedded DSL that is just as writable in Java as any other language that GraalVM supports.

And this includes JavaScript; sorry Mario!