[ACCEPTED]-Automated testing for REST Api-automation

At my work we have recently put together 16 a couple of test suites written in Java 15 to test some RESTful APIs we built. Our 14 Services could invoke other RESTful APIs 13 they depend on. We split it into two suites.

• Suite 1 - Testing each service in isolation
  • Mock any peer services the API depends on using restito. Other alternatives include rest-driver, wiremock and betamax.
  • Tests the service we are testing and the mocks all run in a single JVM
  • Launches the service in Jetty

I 12 would definitely recommend doing this. It 11 has worked really well for us. The main 10 advantages are:

• Peer services are mocked, so you needn't perform any complicated data setup. Before each test you simply use restito to define how you want peer services to behave, just like you would with classes in unit tests with Mockito.
• You can ask the mocked peer services if they were called. You can't do these asserts as easily with real peer services.
• The suite is super fast as mocked services serve pre-canned in-memory responses. So we can get good coverage without the suite taking an age to run.
• The suite is reliable and repeatable as its isolated in it's own JVM, so no need to worry about other suites/people mucking about with an shared environment at the same time the suite is running and causing tests to fail.

• Suite 2 - Full End to End
  • Suite runs against a full environment deployed across several machines
  • API deployed on Tomcat in environment
  • Peer services are real 'as live' full deployments

This suite requires us to 9 do data set up in peer services which means 8 tests generally take more time to write. As 7 much as possible we use REST clients to 6 do data set up in peer services.

Tests in 5 this suite usually take longer to write, so 4 we put most of our coverage in Suite 1. That 3 being said there is still clear value in 2 this suite as our mocks in Suite 1 may not 1 be behaving quite like the real services.

Frisby is a REST API testing framework built 2 on node.js and Jasmine that makes testing 1 API endpoints easy, fast, and fun. http://frisbyjs.com

Example:

var frisby = require('../lib/frisby');

var URL = 'http://localhost:3000/';
var URL_AUTH = 'http://username:password@localhost:3000/';

frisby.globalSetup({ // globalSetup is for ALL requests
  request: {
    headers: { 'X-Auth-Token': 'fa8426a0-8eaf-4d22-8e13-7c1b16a9370c' }
  }
});

frisby.create('GET user johndoe')
  .get(URL + '/users/3.json')
  .expectStatus(200)
  .expectJSONTypes({
    id: Number,
    username: String,
    is_admin: Boolean
  })
  .expectJSON({
    id: 3,
    username: 'johndoe',
    is_admin: false
  })
  // 'afterJSON' automatically parses response body as JSON and passes it as an argument
  .afterJSON(function(user) {
    // You can use any normal jasmine-style assertions here
    expect(1+1).toEqual(2);

    // Use data from previous result in next test
    frisby.create('Update user')
      .put(URL_AUTH + '/users/' + user.id + '.json', {tags: ['jasmine', 'bdd']})
      .expectStatus(200)
    .toss();
  })
.toss();

I collaborated with one of my coworkers 7 to start the PyRestTest framework for this 6 reason: https://github.com/svanoort/pyresttest

Although you can work with the tests 5 in Python, the normal test format is in 4 YAML.

Sample test suite for a basic REST 3 app -- verifies that APIs respond correctly, checking 2 HTTP status codes, though you can make it 1 examine response bodies as well:

---
- config:
    - testset: "Tests using test app"

- test: # create entity
    - name: "Basic get"
    - url: "/api/person/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
    - method: 'DELETE'
- test: # create entity by PUT
    - name: "Create/update person"
    - url: "/api/person/1/"
    - method: "PUT"
    - body: '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
    - headers: {'Content-Type': 'application/json'}
- test: # create entity by POST
    - name: "Create person"
    - url: "/api/person/"
    - method: "POST"
    - body: '{"first_name": "Willim","last_name": "Adama","login": "theadmiral"}'
    - headers: {Content-Type: application/json}

I used SOAP UI for functional and automated testing. SOAP 4 UI allows you to run the tests on the click 3 of a button. There is also a spring controllers testing page created 2 by Ted Young. I used this article to create 1 Rest unit tests in our application.

One of the problems of doing automated testing 33 for APIs is that many of the tools require 32 you to have the API server up and running 31 before you run your test suite. It can be 30 a real advantage to have a unit testing 29 framework that is capable of running and 28 querying the APIs in a fully automated test 27 environment.

An option that's good for APIs 26 implemented with Node.JS / Express is to 25 use mocha for automated testing. In addition 24 to unit tests, its easy to write functional 23 tests against the APIs, separated into different 22 test suites. You can start up the API server 21 automatically in the local test environment 20 and set up a local test database. Using 19 make, npm, and a build server, you can create 18 a "make test" target and an incremental 17 build that will run the entire test suite 16 every time a piece of code is submitted 15 to your repository. For the truly fastidious 14 developer, it will even generate a nice 13 HTML code-coverage report showing you which 12 parts of your code base are covered by tests 11 or not. If this sounds interesting, here's 10 a blog post that provides all the technical details.

If 9 you're not using node, then whatever the 8 defacto unit testing framework for the language 7 is (jUnit, cucumber/capybara, etc) - look 6 at its support for spinning up servers in 5 the local test environment and running the 4 HTTP queries. If it's a large project, the 3 effort to get automated API testing and 2 continual integration working will pay off 1 pretty quickly.

Hope that helps.

Runscope is a cloud based service that can monitor 7 Web APIs using a set of tests. Tests can 6 be , scheduled and/or run via parameterized 5 web hooks. Tests can also be executed from 4 data centers around the world to ensure 3 response times are acceptable to global 2 client base.

The free tier of Runscope supports 1 up to 10K requests per month.

Disclaimer: I am a developer advocate for Runscope.

I implemented many automation cases based 7 on REST Assured , a jave DSL for testing 6 restful service. https://code.google.com/p/rest-assured/

The syntax is easy, it 5 supports json and xml. https://code.google.com/p/rest-assured/wiki/Usage

Before that, I tried 4 SOAPUI and had some issues with the free 3 version. Plus the cases are in xml files 2 which hard to extend and reuse, simply I 1 don't like

You can also use Rest Assured library. For 1 a demo with sample script, refer to http://artoftesting.com/automationTesting/restAPIAutomationGetRequest.html

I have used TestNG and Apache HTTP classes 7 to build my own REST API test framework, I 6 developed this concept after working in 5 Selenium for two years.

Everything is same, except 4 you should use Apache HTTP classes instead 3 of Selenium classes.

give a try, its really 2 cute and good, you've all the power to customize 1 your test framework to your fullest possibilities.

API test automation, up to once per minute, is 17 a service available through theRightAPI. You create 16 your test scenarios, and execute them. Once 15 those tests do what you expect them to, you 14 can then schedule them. Tests can be 'chained' together 13 for scenarios that require authentication. For 12 example, you can have a test that make an 11 OAuth request to Twitter, and creates a 10 shared token that can then be used by any 9 other test. Tests can also have validation 8 criteria attached to ensure http status 7 codes, or even detailed inspection of the 6 responses using javascript or schema validation. Once 5 tests are scheduled, you can then have alerts 4 notify you as soon as a particular test 3 fails validation, or is behaving out of 2 established ranges for response time or 1 response size.