CRISP-Rules Mining (RM)

Published: 7/18/2018 Revised: 6/17/2021
Keywords: rules, machine learning, architecture, delivery
Author: Tim Ackroyd

Cross-industry Process for Rules Mining (elicitation)

CRISP-RM stands for cross-industry process for rules mining. NOTE in an attempt not to plagiarize the great work done by stakeholders in CRISP-DM, I must preface by saying that I am taking from CRISP-DM, as a framework, in this article, and that this is NOT an industry standard but rather an idea to share. The CRISP-RM methodology, like CRSIP-DM (Data Mining) provides a structured approach to planning rules mining projects. It is NOT a well-proven methodology at this time. I do claim any ownership over it at this time but look for feedback on how we can progress in delivery. Recent Machine Learning and Personalization projects have shown major gaps in our abilities to delivery projects faster and more cost-effectively.

CRISP-RM stands for cross-industry process for rules mining. I am however an evangelist of its powerful practicality, its flexibility and its usefulness when using advanced analytics to solve thorny business issues. It is the golden thread that runs through almost every client engagement. The CRISP-RM model is shown on the right.

Stage 1

Stage one – determine business objectives

The first stage of the CRISP-RM process is to understand what you want to accomplish from a business perspective in rules and constraints. Your organization may have competing objectives and constraints that must be properly balanced. The goal of this stage of the process is to uncover important factors that could influence the outcome of the project. Neglecting this step can mean that a great deal of effort is put into producing the right answers to the wrong questions.

What are the desired outputs of the project?

Set objectives - This means describing your primary objective from a business perspective when exercising rules and constraints. There may also be other related questions that you would like to address. For example, your primary goal might be to keep current customers by predicting when they are prone to (churn) move to a competitor based on a Credit Adjustment decision. Related business questions might be “Does the channel used affect whether customers stay or go?” or “Will better offers significantly reduce the number of high-value customers who leave?”
Produce project plan - Here you'll describe the plan for achieving the rules mining and business goals. The plan should specify the steps to be performed during the rest of the project, including the initial selection of tools and techniques.
Business success criteria - Here you'll lay out the criteria that you'll use to determine whether the project has been successful from the business point of view (measured success factors). These should ideally be specific and measurable, for example reduction of customer churn to a certain level, however sometimes it might be necessary to have more subjective criteria such as “give useful insights into the relationships and apply those insights based on rules and constraints.” If this is the case then it needs to be clear who it is that makes the subjective judgment.

Assess the current situation

This involves more detailed fact-finding about all of the resources, constraints, assumptions and other factors that you'll need to consider when determining your rules analysis goal and project plan.

Inventory of resources - List the resources available to the project including:
- Personnel (business experts, rules experts, technical support, *rule mining experts)
- Rule (fixed extracts, access to live, warehoused, or operational rules)
- Computing resources (hardware platforms)
- Software (rule mining tools, other relevant software)
Requirements, assumptions and constraints - List all requirements of the project including the schedule of completion, the required comprehensibility and quality of results, and any rule/constrain security concerns as well as any legal issues. Make sure that you are allowed to use the rule and how rule is applied. List the assumptions made by the project. These may be assumptions about the rule that can be verified during rules mining but may also include non-verifiable assumptions about the business related to the project. It is particularly important to list the latter if they will affect the validity of the results. List the constraints on the project. These may be constraints on the availability of resources but may also include technological constraints such as the complexity of rule-set(s) that it is practical to use for modelling.
Risks and contingencies - List the risks or events that might delay the project or cause it to fail. List the corresponding contingency plans - what action will you take if these risks or events take place?
Terminology - Compile a glossary of terminology relevant to the project. This will generally have two components:
- A glossary of relevant business terminology, which forms part of the business understanding available to the project. Constructing this glossary is a useful “knowledge elicitation” and education exercise.
- A glossary of rules mining terminology and taxonomies, illustrated with examples relevant to the business problem in question.
Costs and benefits - Construct a cost-benefit analysis for the project which compares the costs of the project with the potential benefits to the business if it is successful. This comparison should be as specific as possible. For example, you should use financial measures in a commercial situation.

*Rule Mining is different from Rules Expert, as this involves discovery of related and impacting rules. Rules Experts may understand their own rules but may not have visibility into rules that impact their area. Much like data mining, this requires metadata pertaining to rules be managed in libraries or Hadoop clusters that use machine learning to drive relevance and other discovery points for such mining (rule-patterns, connections, etc.).

Determine rule mining goals

It is important to understand rule elicitation and rules mining exercises. Rule elicitation is the exercise between rule experts and clients, where the rules mining exercise is mining rules metadata, relevance, and using machine learning to understand the successful use of rules and constraints.

A business goal states objectives in business terminology. A rule mining goal states project objectives in technical terms. For example, the business goal might be “Reduce churn of existing customers.” A rule mining goal might be “Predict how many rules and constraints impact a customer experience, given their care capabilities over the past three years, demographic information (loyalty, age, spend, city, etc.), and the offer in a specific case.”

Business success criteria - describe the intended outputs of the project that enable the achievement of the business objectives.
Rule mining success criteria - define the criteria for a successful outcome to the project in technical terms—for example, a certain level of predictive accuracy or a propensity-to-churn profile with a given degree of “lift.” As with business success criteria, it may be necessary to describe these in subjective terms, in which case the person or persons making the subjective judgment should be identified.

Produce project plan

Describe the intended plan for achieving the rules mining goals and thereby achieving the business goals. Your plan should specify the steps to be performed during the rest of the project, including the initial selection of tools and techniques.

Project plan - List the stages to be executed in the project, together with their duration, resources required, inputs, outputs, and dependencies. Where possible, try and make explicit the large-scale iterations in the rules mining process, for example, repetitions of the modelling and evaluation phases. As part of the project plan, it is also important to analyze dependencies between time schedule and risks. Mark results of these analyses explicitly in the project plan, ideally with actions and recommendations if the risks are manifested. Decide at this point which evaluation strategy will be used in the evaluation phase. Your project plan will be a dynamic document. At the end of each phase you'll review progress and achievements and update the project plan accordingly. Specific review points for these updates should be part of the project plan.
Initial assessment of tools and techniques - At the end of the first phase you should undertake an initial assessment of tools and techniques. Here, for example, you select a rules mining tool that supports various methods for different stages of the process. It is important to assess tools and techniques early in the process since the selection of tools and techniques may influence the entire project. Tools may be similar to profiling data and graphing lineage (e.g. Search for rules based on usage or by keywords, see relevant usage analytics and success criteria to adherence. A technique may be to use Behavior-Driven Development (BDD) with UML Tooling. Behavior-driven development combines the general techniques and principles of Test-Driven Development (TDD) with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development

Stage two – Rules understanding

The second stage of the CRISP-RM process requires you to acquire the rules listed in the project resources (M&P). This initial collection includes rule invocation or loading, if this is necessary for rule understanding. For example, if you use a specific tool for rules understanding, it makes perfect sense to load your rules into this tool (e.g. Excel, iLOG, DROOLS, etc.). If you acquire multiple rule sources, then you need to consider how and when you're going to integrate these.

Initial rule collection report - List the rule sources acquired together with their locations, the methods used to acquire them, and any problems encountered. Record problems you encountered, and any resolutions achieved. This will help both with future replication of this project and with the execution of similar future projects.

Describe rule(s)

Examine the “gross” or “surface” properties of the acquired rules and report on the results.

Rule description report - Describe the rules that have been acquired including its format (including technology and language), its quantity (for example, the number of rules applied to what processing), the identities of the rules (e.g. rule IDs) and any other surface features which have been discovered. Evaluate whether the rules acquired satisfies your requirements (CMO Processes to FMO processes).

Explore rules

During this stage you'll address rules mining questions using querying, rule visualization and reporting techniques. These may include:

Distribution of key attributes (for example, the target attribute of a prediction task)
Relationships between rules and functions
Results of simple aggregations (aggregated functions)
Properties of significant sub-processing
Simple statistical analyses

These analyses may directly address your rules mining goals. They may also contribute to or refine the rules description and quality reports, and feed into the transformation and other rules preparation steps needed for further analysis.

Rules exploration report - Describe results of your rules exploration, including first findings or initial hypothesis and their impact on the remainder of the project. If appropriate you could include graphs and plots here to indicate rules characteristics that suggest further examination of interesting rules and constraints.

Verify data/rules quality

Examine the quality of the data, addressing questions such as:

Is the data/rule(s) complete (does it cover all the cases required)?
Is it correct, or does it contain errors and, if there are errors, how common are they?
Are there missing values in the data? If so, how are they represented, where do they occur, and how common are they?

Rule quality report

List the results of the rules quality verification. If quality problems exist, suggest possible solutions. Solutions to rules and constraints quality problems generally depend heavily on both functions and business knowledge of methods and procedures (M&P).

Stage three – Rules preparation

Select your rules

This is the stage of the project where you decide on the rules that you're going to use for functions and decisions. The criteria you might use to make this decision include the relevance of the rules to your rules mining and machine learning goals, the quality of the rules, and also technical constraints such as limits on functional volume-based rule types. Note that rule selection covers selection of functional concerns as well as selection of data that rules are using.

Rationale for inclusion/exclusion - List the rules to be included/excluded and the reasons for these decisions.

Clean your rules

This task involves raise the rules and constraints quality to the level required by the analysis techniques that you've selected. This may involve selecting clean subsets of the rules, the insertion of use case defaults, or more ambitious techniques such as the estimation of missing rules by modelling.

Rules cleaning report - Describe what decisions and actions you took to address rules quality problems. Consider any transformations of the rules made for cleaning purposes and their possible impact on the analysis results.

Construct required rules

This task includes constructive rules preparation operations such as the production of derived outcomes or entire functional outcomes or transformed values for existing rules and constraints.

Derived outcomes - These are new outcomes that are constructed from one or more existing functions in the same record, for example you might use the rules and constraints to derive functional or machine learning outcomes to drive new outcomes and analytics for a specific concern or area.
Generated records - Here you describe the creation of any completely new outcomes. For example, you might need to create learning models for customers who had no credit adjustments during the past year. There was no reason to have such outcome concerns in the edge case but for modelling purposes it might make sense to explicitly represent the fact that particular customers had no credit adjustments and warrant different treatment.

Integrate rules

These are methods whereby information is combined from multiple rules invocations, to create new outcomes or value.

Merged rules- Merging rules refer to joining together two or more rules sources that have different information about the same objects. For example, a retail chain might have one rule with information about each store’s general characteristics (e.g., demographics, social, loyalty), another set of rules with which drive sales or reduce churn (e.g., profit, percent change in revenue from previous year), and another with information about the demographics and psychographics of the surrounding area. Each of these sets of rules pertains to each store. These sets of rules can be merged together into a new source or machine learning point with one set of rules for each store, combining outcomes from the disposition points.
Aggregations - Aggregations refers to operations in which new values and outcomes are computed by summarizing information from multiple disposition analysis points. For example, testing outcomes based on learning models for customers in one store, and be aggregated in relevance to outcomes with relevant (same) store or channel outcomes. Patterns from outcomes (dispositions) can be mapped for usage in cross-channel operations for consistent outcomes.

Stage four – modelling

Select modeling technique

As the first step in modelling, you'll select the actual modelling technique that you'll be using. Although you may have already selected a tool during the business understanding phase, at this stage you'll be selecting the specific modelling technique e.g. Gherkin for BDD, transformation via XML, import into common Rules Management tools. If multiple techniques are applied, perform this task separately for each technique.

Modelling technique - Document the actual modelling technique that is to be used.
Modelling assumptions - Many modelling techniques (UML, XML, etc.) make specific assumptions about the rules, for example we can simulate in UML Tooling (e.g. Sparx); no simulation is available to decision models in tools such as Visio or Excel; we can also simulate in iLog or Drools tooling with some minor coding. NOTE: "Modeling" doesn't necessarily mean graphical. Models can be derived and simulate in R and run on real data.

Behavior-Driven Development

Behavior-driven development combines the general techniques and principles of TDD with ideas from domain-driven design and object-oriented analysis and design to provide software development and management teams with shared tools and a shared process to collaborate on software development.

Although BDD is principally an idea about how software development should be managed by both business interests and technical insight, the practice of BDD does assume the use of specialized software tools to support the development process. Although these tools are often developed specifically for use in BDD projects, they can be seen as specialized forms of the tooling that supports test-driven development. The tools serve to add automation to the ubiquitous language that is a central theme of BDD.

BDD is largely facilitated through the use of a simple domain-specific language (DSL) using natural language constructs (e.g., English-like sentences) that can express the behavior and the expected outcomes. Test scripts have long been a popular application of DSLs with varying degrees of sophistication. BDD is considered an effective technical practice especially when the "problem space" of the business problem to solve is complex. For more, see appendix at bottom of document.

Generate test design

Before you actually build a model, or simulate outcomes, you need to generate a procedure or mechanism to test the model’s quality and validity. For example, in supervised rules mining tasks such as classification, it is common to use error rates as quality measures for rules mining models. Therefore, you typically separate the dataset into train and test sets, build the model on the train set, and estimate its quality on the separate test set. NOTE: A classification model assigns to each entity the outcome it considers to be the most appropriate.

Test design - Describe the intended plan for training, testing, and evaluating the models. A primary component of the plan is determining how to divide the available rules-set into training, test and validation rules-sets.

Build model

Run the modelling or computational tool on the prepared rule-set(s) to create one or more outcomes.

Parameter settings - With any modelling or IDE tool there are often a large number of parameters that can be adjusted. List the parameters and their chosen values, along with the rationale for the choice of parameter settings.
Models - These are the actual models produced by the modelling tool, not a report on the models.
Model descriptions - Describe the resulting models, report on the interpretation of the models and document any difficulties encountered with their meanings.

Assess model

Interpret the models according to your domain knowledge, your rule mining or elicitation success criteria and your desired test design. Judge the success of the application of modelling and discovery techniques technically, then contact business analysts and domain experts later in order to discuss the rules mining or elicitation results in the business context. This task only considers models, whereas the evaluation phase also takes into account all other results that were produced in the course of the project.

At this stage you should rank the models and assess them according to the evaluation criteria. You should take the business objectives and business success criteria into account as far as you can here. In most rules-driven projects a single technique is applied more than once and rules mining results are generated with several different techniques.

Model assessment - Summarize the results of this task, list the qualities of your generated models (e.g.in terms of accuracy) and rank their quality in relation to each other.
Revised parameter settings - According to the model assessment, revise parameter settings and tune them for the next modelling run (e.g. SOA Limits on Agents, or changes in customer types). Iterate model building and assessment until you strongly believe that you have found the best model(s). Document all such revisions and assessments

Stage five – evaluation

Evaluate your results

Previous evaluation steps dealt with factors such as the accuracy and generality of the model. During this step you'll assesses the degree to which the model meets your business objectives and seek to determine if there is some business reason why this model is deficient. Another option is to test the model(s) on test applications in the real application, if time and budget constraints permit. The evaluation phase also involves assessing any other rules mining results you've generated. Rules mining results involve models that are necessarily related to the original business objectives and all other findings that are not necessarily related to the original business objectives but might also unveil additional challenges, information, or hints for future directions.

Assessment of rules mining results - Summarize assessment results in terms of business success criteria, including a final statement regarding whether the project already meets the initial business objectives.
Approved models - After assessing models with respect to business success criteria, the generated models that meet the selected criteria become the approved models.

Review process

At this point, the resulting models appear to be satisfactory and to satisfy business needs. It is now appropriate for you to do a more thorough review of the rules mining engagement in order to determine if there is any important factor or task that has somehow been overlooked. This review also covers quality assurance issues—for example: did we correctly build the model or collect necessary rules, constraints, methods, and procedures? Did we use only the attributes and outcomes that we are allowed to use and that are available for future analyses?

Review of process - Summarize the process review and highlight activities that have been missed and those that should be repeated.

Determine next steps

Depending on the results of the assessment and the process review, you now decide how to proceed. Do you finish this project and move on to deployment, initiate further iterations, or set up or associate new rules mining to projects? You should also take stock of your remaining resources and budget as this may influence your decisions.

List of possible actions - List the potential further actions, along with the reasons for and against each option.
Decision - Describe the decision as to how to proceed, along with the rationale.

Stage six – deployment

Plan deployment

In the deployment stage you'll take your evaluation results and determine a strategy for their deployment. If a general procedure has been identified to create the relevant model(s), this procedure is documented here for later deployment. It makes sense to consider the ways and means of deployment during the business understanding phase as well, because deployment is absolutely crucial to the success of the project. This is where predictive analytics really helps to improve the operational side of your business.

Deployment plan - Summarize your deployment strategy including the necessary steps and how to perform them.

Plan monitoring and maintenance

Monitoring and maintenance are important issues if the rules mining result becomes part of the day-to-day business and its environment. The careful preparation of a maintenance strategy helps to avoid unnecessarily long periods of incorrect usage of rules mining results. In order to monitor the deployment of the rules mining and execution result(s), the project needs a detailed monitoring process plan. This plan takes into account the specific type of deployment.

Monitoring and maintenance plan - Summarize the monitoring and maintenance strategy, including the necessary steps and how to perform them.

Produce final report

At the end of the project you will write up a final report. Depending on the deployment plan, this report may be only a summary of the project and its experiences (if they have not already been documented as an ongoing activity) or it may be a final and comprehensive presentation of the data mining result(s).

Final report - This is the final written report of the rules mining, elicitation, and testing engagement. It includes all of the previous deliverables, summarizing and organizing the results.
Final presentation - There will also often be a meeting at the conclusion of the project at which the results are presented to the customer.

Review project

Assess what went right and what went wrong, what was done well and what needs to be improved.

Experience documentation - Summarize important experience gained during the project. For example, any pitfalls you encountered, misleading approaches, or hints for selecting the best suited rules elicitation and mining techniques in similar situations could be part of this documentation. In ideal projects, experience documentation also covers any reports that have been written by individual project members during previous phases of the project.

Four Problems in Using CRISP-RM and How To Fix Them

CRISP-RM – the Cross Industry Standard Process for Rules Mining – can be by far the most valuable methodology for rules mining and elicitation. Rules Administrators, Machine Learning, and Analytics Managers should use CRISP-RM because they recognize the need for a repeatable approach. However, there are some persistent problems with how CRISP-RM, just as there are in CRISP-DM, is generally applied. The top four problems are a lack of clarity, mindless rework, blind hand-offs to IT and a failure to iterate. Decision modeling and decision management can address these problems, maximizing the value of CRISP-RM and CRSIP-DM and ensuring analytic success.

The phases of the complete CRISP-RM approach are shown below with emphasis on the four problems. These phases set up the business problem (Business Understanding), review the available data (Rules Understanding), develop rules models and techniques (Rules Preparation and Modeling), evaluate results against business need (Evaluation) and deploy the model (Deployment). The whole cycle is designed to be iterative, repeating as necessary to keep models current and effective.

CRISP-RM should be a great framework and its use on projects should help focus them on delivering real business value. CRISP-RM is a proposition for us to rectify challenges in Rules Management, Elicitation, Aggregation, and Testing. We need to be concerned about what has been around a long time so many projects that are using CRISP-DM, and applies to CRISP-RM, are taking shortcuts. Some of these shortcuts make sense but too often they result in projects using a corrupted version of the approach like the one shown below.

Four Problem

The four problems with this corrupted approach are:

A lack of clarity
Rather than drill down into the details and really get clarity on both the business problem and exactly how an analytic might help, the project team make do with the business goals and some metrics to measure success. Now they “understand” the business objective, they want to minimize “overhead” and leap into the “interesting” bit of the project, analyzing the rules, M&P, and constraints. Too often this results in interesting outcomes that don’t meet a real business need.
Mindless rework
Some application teams simply assess their project results in business terms and technical terms separately – if the outcome or model is to be predictive (using machine learning) then it must be good. Most realize that this is not necessarily true and try and check their analytic results against the business objective. This is difficult without real clarity on the business problem. If the outcome or analytics they have developed does not seem to meet the business objectives, the team has few options. Most try to find new data or new modeling techniques rather than working with their business partners to re-evaluate the business problem.
Blind hand-off to IT
Some application teams don’t think about deployment and operationalization of their models at all. Most do better than that, though, recognizing that the outcomes and models they build will have to be applied to live data in operational sense or embedded in operational systems. Even these teams have typically not engaged with necessary IT prior to this point don’t have clarity on how the outcome or analytic needs to be deployed and don’t really regard deployment as “analytic” work. The end result is a outcome or model that is thrown over the wall to IT. Whether the model is easy to implement or hard (or impossible) and whether it’s really usable once deployed is someone else’s problem. This increases the time and cost of deploying a rule or model and contributes to the huge percentage of outcomes or models that never have the proper business impact.
Failure to iterate
Rules, Constraints, and Machine Learning professionals know that rules models age and that models need to be kept up to date if they are to continue to be valuable. They know that business circumstances can change and undermine the value of a rule-set or rule model. They know that the rule and constraint (decision) patterns that drove the model may change in the future. But they think of that as a problem for another day – they don’t have enough clarity on the business problem to determine how to track the model’s outcomes (business performance) nor do they invest in thinking about to make revision of the outcomes and model less work than the initial creation. After all, it’s much more interesting to tackle another new problem. This leaves aging approaches and models unmonitored and unmaintained, undermining the long-term value of decision making and automation.

Each of these problems adds to the likelihood that the team will build an impressive analytic solution that will not add business value. Organizations that want to really exploit automation and decision engines – especially using more advanced analytics driven by rules mining, data mining, predictive analytics and machine learning – cannot afford these problems to persist.

Fixing the problems revolves around a clear and unambiguous focus on decision-making – what decision must be improved, what does it mean to improve it, does the function, outcome, or decision-making engine actually improve outcomes, which systems and processes embed or support this decision-making and what environment changes might cause a re-evaluation of the decision making?

Conclusion

The old practices for Requirements Engineering have not worked. We still fail to properly elicit simple nouns and verbs in the gaps between current mode of operations (CMO) and desired future mode of operations (FMO). As we move into the world of machine learning, GIGO (Garbage In and Garbage Out) has never been more riskier. Conference calls filled with words and not models can no long continue, or we will fail to deliver automation and desired machine learning-driven automation. We can no longer tolerate ambiguity in requirement, more importantly the rules, constraints, policies, and procedures for doing business, operations, or functions. We not only need to understand every machine that renders decisions, but every human and their roles, position, authorization, skills, and even demeanors.

Picking up on the things that work in the industry, such as CRISP-DM, as well as frameworks such as that from TM Forums and TOGAF are now more necessary than ever. As we have watched hardware and software trade places in evolution over the last 30 years, humans have never been so far behind.

Appendix

Behavior-driven development (a technique)

Behavior-driven development is an extension of test-driven development: development that makes use of a simple, domain-specific scripting language. These DSLs convert structured natural language statements into executable tests. The result is a closer relationship to acceptance criteria for a given function and the tests used to validate that functionality. As such it is a natural extension of TDD testing in general.

BDD focuses on:

· Where to start in the process

· What to test and what not to test

· How much to test in one go

· What to call the tests

· How to understand why a test fails

Before I get into tooling such as Behat, I want to open a discussion around Sparx, UML, and BDD.

So if we understand today that Gherkin (like YAML, Python, and few others), it is a line-oriented language that uses indentation to define structure. We also know that in every feature file, there is the notion of “scenario”, where every consists of keywords (given, when, then, but, & and).

Example

Feature: Alerts
In order to get *alerts
Customers should be able to
find alerts at all times

Scenario: get an alert
Given there alerts provided
And I have authenticated in the system
When I land on a given page
Then I should be see a real alert
*Note: An “alert” is NOT a “offer”!!!

Behat is a test framework for behavior-driven development written in the PHP programming language. Behat was created by Konstantin Kudryashov and its development is hosted on GitHub.

In software engineering, behavior-driven development (BDD) is a software development process that emerged from test-driven development (TDD). This comparison of programming languages compares the features of language syntax (format) for over 50 computer programming languages.

This comparison of programming languages compares the features of language syntax (format) for over 50 computer programming languages.

Behat is a tool to test the behavior of your application, described in special language called Gherkin. Gherkin is a Business Readable, Domain Specific Language created especially for behavior descriptions. It gives you the ability to remove logic details from behavior tests.

Gherkin is a syntax for modelling examples of the realization of functional requirements. These examples are called scenarios and they includes specific information needed for a real execution of functional requirements. Gherkin was created as one part of the Cucumber tool. The main goal of Gherkin is to define examples of the behavior of the system using a nontechnical notation for stakeholders and business people.

Gherkin serves two purposes: serving as your project’s documentation and automated tests. Behat also has a bonus feature: it talks back to you using real, human language telling you what code you should write.

If you’re still new to Behat, jump into the Quick Intro to Behat first, then return here to learn more about Gherkin. Like YAML or Python, Gherkin is a line-oriented language that uses indentation to define structure. Line endings terminate statements (called steps) and either spaces or tabs may be used for indentation. (We suggest you use spaces for portability).

The parser divides the input into features, scenarios and steps. Let’s walk through the above example:

1. Feature: Some terse yet descriptive text of what is desired starts the feature and gives it a title. Learn more about features in the “Features” section.

2. Behat does not parse the next 3 lines of text. (In order to... As an... I want to...). These lines simply provide context to the people reading your feature, and describe the business value derived from the inclusion of the feature in your software.

3. Scenario: Some determinable business situation starts the scenario, and contains a description of the scenario. Learn more about scenarios in the “Scenarios” section.

4. The next 7 lines are the scenario steps, each of which is matched to a regular expression defined elsewhere. Learn more about steps in the “Steps” section.

5. Scenario: A different situation starts the next scenario, and so on.

When you’re executing the feature, the trailing portion of each step (after keywords like Given, And, When, etc) is matched to a regular expression, which executes a PHP callback function. You can read more about steps matching and execution in Defining Reusable Actions - Step Definitions.

Features

Every *.feature file conventionally consists of a single feature. Lines starting with the keyword Feature: (or its localized equivalent) followed by three indented lines starts a feature. A feature usually contains a list of scenarios. You can write whatever you want up until the first scenario, which starts with Scenario: (or localized equivalent) on a new line. You can use tags to group features and scenarios together, independent of your file and directory structure.

Every scenario consists of a list of steps, which must start with one of the keywords Given, When, Then, But or And (or localized one).

In addition to basic scenarios, feature may contain scenario outlines and backgrounds.

Scenarios

Scenario is one of the core Gherkin structures. Every scenario starts with the Scenario: keyword (or localized one), followed by an optional scenario title. Each feature can have one or more scenarios, and every scenario consists of one or more steps.

Backgrounds

Backgrounds allows you to add some context to all scenarios in a single feature. A Background is like an untitled scenario, containing a number of steps. The difference is when it is run: the background is run before each of your scenarios, but after your BeforeScenario hooks (Hooking into the Test Process - Hooks)

Writing Features - Gherkin Language

Gherkin Syntax

Like YAML or Python, Gherkin is a line-oriented language that uses indentation to define structure. Line endings terminate statements (called steps) and either spaces or tabs may be used for indentation. (We suggest you use spaces for portability.) Finally, most lines in Gherkin start with a special keyword:

Feature: Some terse yet descriptive text of what is desired

In order to realize a named business value

As an explicit system actor

I want to gain some beneficial outcome which furthers the goal

Scenario: Some determinable business situation

Given some precondition

And some other precondition

When some action by the actor

And some other action

And yet another action

Then some testable outcome is achieved

And something else we can check happens too

Scenario: A different situation

...

The parser divides the input into features, scenarios and steps. Let’s walk through the above example:

1. Feature: Some terse yet descriptive text of what is desired starts the feature and gives it a title. Learn more about features in the “Features” section.

3. Scenario: Some determinable business situation starts the scenario, and contains a description of the scenario. Learn more about scenarios in the “Scenarios” section.

4. The next 7 lines are the scenario steps, each of which is matched to a regular expression defined elsewhere. Learn more about steps in the “Steps” section.

5. Scenario: A different situation starts the next scenario, and so on.

Features

Every scenario consists of a list of steps, which must start with one of the keywords Given, When, Then, But or And (or localized one). Behat treats them all the same, but you shouldn’t. Here is an example:

Feature: Serve coffee

In order to earn money

Customers should be able to

buy coffee at all times

Scenario: Buy last coffee

Given there are 1 coffees left in the machine

And I have deposited 1 dollar

When I press the coffee button

Then I should be served a coffee

In addition to basic scenarios, feature may contain scenario outlines and backgrounds.

Scenarios

The following scenarios each have 3 steps:

Scenario: Wilson posts to his own blog

Given I am logged in as Wilson

When I try to post to "Expensive Therapy"

Then I should see "Your article was published."

Scenario: Wilson fails to post to somebody else's blog

Given I am logged in as Wilson

When I try to post to "Greg's anti-tax rants"

Then I should see "Hey! That's not your blog!"

Scenario: Greg posts to a client's blog

Given I am logged in as Greg

When I try to post to "Expensive Therapy"

Then I should see "Your article was published."

Scenario Outlines

Copying and pasting scenarios to use different values can quickly become tedious and repetitive:

Scenario: Eat 5 out of 12

Given there are 12 cucumbers

When I eat 5 cucumbers

Then I should have 7 cucumbers

Scenario: Eat 5 out of 20

Given there are 20 cucumbers

When I eat 5 cucumbers

Then I should have 15 cucumbers

Scenario Outlines allow us to more concisely express these examples through the use of a template with placeholders:

Scenario Outline: Eating

Given there are <start> cucumbers

When I eat <eat> cucumbers

Then I should have <left> cucumbers

Examples:

| start | eat | left |

| 12 | 5 | 7 |

| 20 | 5 | 15 |

The Scenario outline steps provide a template which is never directly run. A Scenario Outline is run once for each row in the Examples section beneath it (not counting the first row of column headers).

The Scenario Outline uses placeholders, which are contained within < > in the Scenario Outline’s steps. For example:

Given <I'm a placeholder and I'm ok>

Think of a placeholder like a variable. It is replaced with a real value from the Examples: table row, where the text between the placeholder angle brackets matches that of the table column header. The value substituted for the placeholder changes with each subsequent run of the Scenario Outline, until the end of the Examples table is reached.

You can also use placeholders in Multiline Arguments.

Your step definitions will never have to match the placeholder text itself, but rather the values replacing the placeholder.

So when running the first row of our example:

Scenario Outline: controlling order

Given there are <start> cucumbers

When I eat <eat> cucumbers

Then I should have <left> cucumbers

Examples:

| start | eat | left |

| 12 | 5 | 7 |

The scenario that is actually run is:

Scenario Outline: controlling order

# <start> replaced with 12:

Given there are 12 cucumbers

# <eat> replaced with 5:

When I eat 5 cucumbers

# <left> replaced with 7:

Then I should have 7 cucumbers

Backgrounds

Feature: Multiple site support

Background:

Given a global administrator named "Greg"

And a blog named "Greg's anti-tax rants"

And a customer named "Wilson"

And a blog named "Expensive Therapy" owned by "Wilson"

Scenario: Wilson posts to his own blog

Given I am logged in as Wilson

When I try to post to "Expensive Therapy"

Then I should see "Your article was published."

Scenario: Greg posts to a client's blog

Given I am logged in as Greg

When I try to post to "Expensive Therapy"

Then I should see "Your article was published."

Thens

The purpose of Then steps is to observe outcomes. The observations should be related to the business value/benefit in your feature description. The observations should inspect the output of the system (a report, user interface, message, command output) and not something deeply buried inside it (that has no business value and is instead part of the implementation).

· Verify that something related to the Given+When is (or is not) in the output

· Check that some external system has received the expected message (was an email with specific content successfully sent?)

When I call "echo hello"

Then the output should be "hello"

While it might be tempting to implement Then steps to just look in the database – resist the temptation. You should only verify output that is observable by the user (or external system). Database data itself is only visible internally to your application, but is then finally exposed by the output of your system in a web browser, on the command-line or an email message.

And, But

If you have several Given, When or Then steps you can write:

Scenario: Multiple Givens

Given one thing

Given an other thing

Given yet an other thing

When I open my eyes

Then I see something

Then I don't see something else

Or you can use And or But steps, allowing your Scenario to read more fluently:

Scenario: Multiple Givens

Given one thing

And an other thing

And yet an other thing

When I open my eyes

Then I see something

But I don't see something else

If you prefer, you can indent scenario steps in a more programmatic way, much in the same way your actual code is indented to provide visual context:

Scenario: Multiple Givens

Given one thing

And an other thing

And yet an other thing

When I open my eyes

Then I see something

But I don't see something else

Behat interprets steps beginning with And or But exactly the same as all other steps. It doesn’t differ between them - you should!

Multiline Arguments

The regular expression matching in steps lets you capture small strings from your steps and receive them in your step definitions. However, there are times when you want to pass a richer data structure from a step to a step definition.

This is what multiline step arguments are for. They are written on lines immediately following a step, and are passed to the step definition method as the last argument.

Multiline step arguments come in two flavours: tables or pystrings.