Using Cucumber for functional testing of eXo REST APIs

Why do we need a functional testing tool?

The eXo Platform has a REST API. This API allows developers to skip the UI and access the services of the platform through the API. As our company focuses on the product’s quality, we needed to perform functional testing of the APIs quickly and easily. Actually, eXo tests the UI using Selenium, with an amazing non-regression test harness, but this tool is quite slow and was not made for API testing. So, the result of our research for a testing method for the APIs was Cucumber! It is not as fast as unit testing but it is nothing compared to the UI testing. Because they make HTTP requests, our Cucumber tests are considered as integration tests. Moreover, Cucumber is used for behavior-driven development and thus is perfect for our functional tests. It is amazingly efficient for functional testing of the REST API.

What we are testing

When testing the functionally of the API, we check that:

• The API and the UI offer the same services and controls. When a resource is added, modified or deleted through the API, the UI is updated.
• The functional specifications are respected.
• The API has no security flaws.

How we do the testing

Each Cucumber project has the same structure. All the tests are written into a file called a Feature and each test is a scenario. The eXo-Cucumber project just began a few weeks ago and will grow bigger and bigger. The first step was the Calendar REST API. All the tests for the calendar resource in the Calendar REST API have been completed: many bugs were found and these will be corrected. API testing consists of posting, getting, editing and deleting resources in the platform. Here is a basic example where the scenario is to create a calendar, check it has been created and then delete it.

As you can see, Cucumber features are very easy to read. Anyone can understand the test’s goal.

A Cucumber test always has the same structure, which is given by the keywords “Given”, “When” and “Then”.

All the sentences that appear in the Cucumber output are coded in a Java class. Let’s see:

There is one function for each sentence in the Feature file. Of course, these sentences can be used in other scenarios and features.

To send a request in a JSON file to the API, we use retrofit in this class:

Now we can execute our test and see the result:

In this example, all the tests were executed without any errors. Moreover, the test execution was very fast compared with Selenium tests: they took less than 1.3 seconds.

What are the advantages of Cucumber?

As we have already seen, Cucumber use scenarios divided into steps. The clarity of the steps allows everyone, whether in IT or not, to understand what is happening in the test. So, with a Cucumber script, the scenarios are the documentation. To go further, we can even imagine that it will be possible, soon, to write specifications as Cucumber scenarios and test them immediately. This methodology is called behavior-driven development.

Moreover, the script executes really fast, allowing us to test the product quickly and efficiently. Of course, it can be used for different versions of the platform, and is easily adaptable when the testing rules or the API are modified. Cucumber also supports many languages, including Ruby, PHP and Java.

Last but not at least, Cucumber scripts are a real safety net for functional non-regression and allow for refactoring and fast evolution.

What next for the eXo-Cucumber project?

Well actually, this project is just the first steps and has been developed as a side project by Romain Osso and Charles Mugnier, under the supervision of Benoit De Chateauvieux. The objective was to see if Cucumber was suitable and useful for eXo. The result seems to be positive, so the next step will be integration with the software factory.

Our project can be seen in this repository: https://github.com/exoplatform/poc-api-test

And you? Let us know how you perform functional testing of your APIs!

Special thanks to Romain Osso, who wrote the technical aspects of this post. He was the main programmer in this project.

Join the eXo tribe by registering for the community and get updates, tutorials, support, and access to the Platform and add-on downloads!