Update!!!Must see video Land of Lisp
Lisp Language of the Gods
Update!!!Must see video Land of Lisp
KIE
AI Research Overview
For a recent presentation I made an overview diagram of related research areas and ideas, so people can see how things relate. click to enlarge. http://community.jboss.org/wiki/DroolsLanguageEnhancements
For a recent presentation I made an overview diagram of related research areas and ideas, so people can see how things relate.
Machine learning – and Apache Mahout
Isabel Drost recently contributed some enhancements to the Guided Editor (to allow nested facts, very handy) – quite a clever patch. As if that isn’t enough, she is also a contributor to the Apache Mahout project:Mahout is: (in the projects own words): “Mahout’s goal is to build scalable, Apache licensed machine learning libraries.” The projectRead more →
Isabel Drost recently contributed some enhancements to the Guided Editor (to allow nested facts, very handy) – quite a clever patch.
As if that isn’t enough, she is also a contributor to the Apache Mahout project:
Mahout is: (in the projects own words): “Mahout’s goal is to build scalable, Apache licensed machine learning libraries.” The project site is here.
Interestingly one of my #1 books to read on the toilet at the moment is:
This book talks about (amongst many things) using machine learning to “learn” rules – the benefit of learning rules as opposed to some opaque representation is that a human has a fighting chance of understanding the rules, and improving the learning process. It would be interesting to one day see this stuff applied with projects like Mahout.
Anyway, here is Isabel’s writeup on the subject:
The amount of digital data easily available for analysis both in research and
in business has increased tremendously during the last decade. One example of
such data are event logs generated in health care about the patient handling
process. Another example are event logs generated by standard workflow tools.
It is natural to ask whether it is possible to draw conclusions from these
logs, to generalize from what was observed, to learn common process rules
from this data [http://wwwkramer.in.tum.de/ipm08/ ].
in business has increased tremendously during the last decade. One example of
such data are event logs generated in health care about the patient handling
process. Another example are event logs generated by standard workflow tools.
It is natural to ask whether it is possible to draw conclusions from these
logs, to generalize from what was observed, to learn common process rules
from this data [http://wwwkramer.in.tum.de/
In recent years a rather large community of researchers has treated the
problem of learning from example data. The goal of the new Apache project
Mahout [http://lucene.apache.org/
stable, scalable suite of machine learning tools. The framework is designed
for high throughput and will be capable of handling massive datasets both
during training and application – in case this distinction exists. Our focus
is on scalability and we intend to provide parallelized machine learning
algorithm implementations based on the Hadoop framework.
To date several basic algorithms and frameworks have been implemented and
integrated into Mahout: There are implementations for grouping data points
that are similar to each other (clustering). Based on a set of labeled
examples it is possible to learn a classifier that is able to assign new data
points to existing categories(classification). Mahout also integrated Taste,
a framework for learning which items to recommend to users given a log of
user interactions.
Currently the focus is mainly on recommendation mining and learning from
textual data, yet the community is open for new ideas and happily welcomes
contributions from people involved in other topics. So in case you are
interested in machine learning or want to know what one could do with your
data, just drop by on the dev or users mailing list and post your questions
and comments.
Drools and Machine Learning
I’m Gizil. I am doing my master thesis in Drools project. I’m working on decision trees. I have made an ID3, C4.5 implementation with rule generation. I’m investigating bagging and boosting algorithm in order to produce better rules. I am using Annotations on object fields to be able to process extra information on the attributes ofRead more →
I’m Gizil. I am doing my master thesis in Drools project. I’m working on decision trees. I have made an ID3, C4.5 implementation with rule generation. I’m investigating bagging and boosting algorithm in order to produce better rules.
I am using Annotations on object fields to be able to process extra information on the attributes of the objects. I’m generating the rules out of the trees by parsing the trees using depth first search and compiling with PackageBuilder before adding to the RuleBase. In the future I consider using MVEL for templating to generate the rules from the trees.
Target Attribute or Class Label
Since I implement a Supervised learning algorithm the class labels of the target class has to be given by the user. There are two ways to input the labels. The first and easy way is specifying one of the fields related to the target class as the label by the Annotation on that field. The second way is writing a getter function on the object class and specifiying by its Annotation.
Attribute Domains
The most common domain types are categorical and quantitative. Moreover, the decision trees need to deal with complex domains which are not simple primitive object types.
- Categorical (discrete) domain is commonly constructed by a set of String values. There has to be a finite number of discrete values. The attributes are assumed to be categorical by default. Only if there is an annotation saying the opposite then the domain is treated as quantitative. The target attribute has to be categorical since it is not a regression tree implementation.
- Quantitative (continuous) domain: Commonly, subset of real numbers, where there is a measurable difference between the possible values. Integers are usually treated as continuous in practical problems. This type of domain has to be discretized by defining a various number of thresholds (intervals) for each possible class. My implementation can discretize numerical attributes which are a set of real numbers and have quantitative domain.
- Complex domain implements a domain of an attribute that belongs to another object class. This type of domain needs more care because there are many possibilities such as Collections or references to the object class, itself.
For example: age < 15 as child
15 <= age < 20 as teenage
20 <= age as adult
Literal attributes which are set of Strings and have a continuous domain can also be discretized by defining a various number of sets for each possible class.
For example: letter element of {a, e, i, o, u} as vowel
letter not element of {a, e, i, o, u} as consonant
Quinlan’s C4.5 Algorithm
Comparing to the ID3 the C4.5 learning algorithm can tackle with harder domains that contain many number of possible values.
C4.5 deals with the numeric (integer or real) and continuous attributes using a discretization technic based on entropy.
Continuous Attribute Discretization
There are mainly two approaches of discretizing the continuous attributes. One approach is using a global discretization algorithm, which results in a smaller decision tree. A global discretization algorithm would ignore the relation of the continuous attribute with the other attributes. The other approach is at any node of tree dicretizing the continuous attribute on the current set of instances that means applying the global discretization algorithm during the training of the decision tree.
I implemented the Fayyad and Irani’s the Minimum descriptive length method to discretize the numerical domains which is also used by the WEKA project. Fayyad and Irani used information gain approach while evaluating the effectiveness of the discretization. I also tried the Quinlan’s gain ratio approach as an MDL method which is presented as a bit more fair evaluation of the domains due the normalization of the information gain by the information of the current data set based on the domain attribute. Moreover, there are some other approaches such as gini coefficient, or chi-squared test that need to be tested.
For example: Using the 15 Golf instances with 4 attributes (1 boolean, 1 literal and 2 numerical = 2 Categorical and 2 Quantitative) and Boolean target attribute I get a rule saying that the decision should be to play golf outside if the outlook attribute of the Golf object is “overcast”. This rule’s rank is 0.2858 which means that the rule is classifiying 28.58 % of the given Golf objects.
rule “#0 decision= Play classifying 4.0 num of facts with rank:0.2858”
when
$golf_0 : Golf(outlook == “overcast”, $target_label : decision )
then
System.out.println(“[decision] Expected value (” + $target_label + “), Classified as (Play )”);
end
Zementis Drools Case Study
Here at Zementis we have developed a decision engine called ADAPA – Adaptive Decision and Predictive Analytics (http://zementis.com/products.htm) that offers at its core batch and real-time scoring of predictive models as well as fast execution of business rules. As you readers might have guessed, the rules engine is Drools. I will briefly sketch the coreRead more →
Here at Zementis we have developed a decision engine called ADAPA – Adaptive Decision and Predictive Analytics (http://zementis.com/products.htm) that offers at its core batch and real-time scoring of predictive models as well as fast execution of business rules. As you readers might have guessed, the rules engine is Drools.
I will briefly sketch the core philosophy behind the our decision engine and then detail the successful implementation of Drools in ADAPA and how Drools has been an integral part of helping us gain a strong foothold in the analytics market.
The key idea behind our decision engine has always been to integrate two philosophies: Rules-based systems (explicit knowledge) and predictive models (implicit, data-driven algorithms). We believe that in order to have a complete, “intelligent” decision engine, it should not be limited to either rules or predictive models. Hence in ADAPA, rules and predictive models can be combined with great ease to create a complex decision model.
The resulting ADAPA decision engine has been used successfully to deliver intelligent systems to the financial industry. Typically, the financial industry relies heavily on rules-based systems for making decisions on credit applications. We, on the other hand, have implemented systems that not only use conventional business rules but also calculate a custom risk score as part of the decision process. The process in a nutshell: Upon matching of appropriate facts, a rule would fire which would in turn call the appropriate predictive model, and after the predictive model returns a risk score, that number would further be used in other rules to get the correct decision.
The key points about the Drools rule engine that we want to emphasize are:
Drools is extremely reliable: Over the past years, we have successfully used multiple versions of Drools for complex, mission-critical deployments.
Drools is scalable: The rules aspect of our system has supported several thousand rules and hundreds of users while delivering sub-second response times for AJAX-based user interfaces and web services.
Drools is fast: The great thing about Drools that the clients immediately noticed was the speed of execution. The Drools rule engine would return a decision on complex loan applications so fast that we had clients ask: “How can it be so fast?”
Usability: We authored all rules using the Excel based decision table approach. This approach worked better for our user base over writing drl files since it enabled, e.g., a mortgage lender to make changes to rules on a regular basis without needing dedicated IT staff. We would initially create all appropriate Excel files and then let them enter data to reflect their mortgage guidelines. Hence having the ability to generate rules through Excel, versus only being able to write ‘drl’ files gave us a strategic advantage in usability.
Thank you Drools team!