Alexander Beletsky's Development Blog: 2010-12

Testing REST services with javascript

In previous article we've reviewed general concept of REST. Now we will implement some basic REST service. And our approach will be - test methods before, implement them later. I'm talking about kind integration tests, the tests that would act exactly as your client, making real calls to storage and return real results. I will use jQuery and qUnit as my weapon of choice. Like in case of FuncUnit it is easy and fun to create those tests.

Why should I start from tests? Pretty simple, by implementing tests before you are looking on your service as client<, not as a developer. When I was working to version 1 of my REST API I didn't do any tests, basically because I didn't know how to do them. When I was ready and started implementation of the client code and documentation, I found out major API issues that I had no time to solve. Those issues were related to: design, security, formats and convenience of usage. TDD principles works the same here: clear and simple design through series of tests.

Simple framework

I rely of jQuery and qUnit. jQuery $.ajax method is used to send and receive data. All tests are done in qUnit fashion. What is good to have more: small wrapper function for doing API calls, that would do initial verification of results and work synchronously. Why is it synchronous? Because tests are not application and you do not need all benefits of async calls. Asynchronous behavior requires additional effort for synchronization of results. Even if qUnit supports asynchronous testing, it should be avoided as possible since it makes test code harder to write and read. So, I came up with such implementation:

function api_test(url, type, data, callback) {
  $.ajax(
    {
      url: url,
      type: type,
      processData: false,
      contentType: 'application/json; charset=utf-8',
      data: JSON.stringify(data),
      dataType: 'json',
      async: false,
      complete: function (result) {
        if (result.status == 0) {
          ok(false, '0 status - browser could be on offline mode');
        } else if (result.status == 404) {
          ok(false, '404 error');
        } else {
          callback($.parseJSON(result.responseText));
        }
      }
    });
}


* This source code was highlighted with Source Code Highlighter.

Internal If/else statement could be extended with some specific result codes you might expect. If API call has been finished successfully, result JSON object will be parsed and submitted to a test callback.

Also, I found useful to create a small helper function that would construct API call signature, based on URL, method and parameters:

  // helper
  function createCallUrl(url, apiToken, method, params) {
    var callUrl = url + apiToken + "/" + method;
    for (var p in params) {
      callUrl += "/" + params[p];
    }

    return callUrl;
  }


* This source code was highlighted with Source Code Highlighter.

Setup the environment

All tests requires a setup. In integration testing we basically rely on existing environment (the same that will be used by real application).

Due to security reasons all API calls receive an api token as first argument for any call. Api token is received after successful authentication, so a StartUp for each test we need to login, receive api token and only then proceed with method tests. For qUnit is is natural to place this code to module setup.

  module("v11 api tests", {
    // setup method will authenticate to v.1.1. API by calling 'authenticate'
    // it will store apiToken, so rest of tests could reuse that

    setup: function () {
      var me = this;

      this.url = 'api/v1.1/';
      this.apiToken = null;

      // authenticate
      var method = 'authenticate';
      var data = { email: 'tracky@tracky.net', password: '111111' };
      var type = 'POST';

      api_test(this.url + method, type, data, function (result) {
        ok(result.success, method + " method call failed");

        me.apiToken = result.data.apiToken;
        ok(me.apiToken.length == 32, "invalid api token");
      });
    }
  }
  );


* This source code was highlighted with Source Code Highlighter.

Module holds API URL and token, so they are reusable through the rest of tests. If setup failed to authenticate, all tests would be failed because they could not use any call without token.

Testing methods

I have a number of REST style methods in my API:

http://trackyt.net/api/v1.1/token/tasks/all
http://trackyt.net/api/v1.1/token/tasks/add
http://trackyt.net/api/v1.1/token/tasks/delete/112
http://trackyt.net/api/v1.1/token/tasks/start/112
http://trackyt.net/api/v1.1/token/tasks/stop/112

and so on..

I'll give some examples of tests, so you will be able to follow main idea:

Get all task call test:

  test("get all tasks method", function () {
    var method = 'tasks/all';
    var data = null;
    var type = 'GET';
    var params = [];

    var call = createCallUrl(this.url, this.apiToken, method, params);

    api_test(call, type, data, function (result) {
      ok(result.success, method + " method call failed");

      var tasks = result.data.tasks;
      ok(tasks.length >= 1, "tasks has not been returned");
    });
  });


* This source code was highlighted with Source Code Highlighter.

Get all task call receives deterministic response test:

  test("get all tasks returns all required fields", function () {
    var method = 'tasks/all';
    var data = null;
    var type = 'GET';
    var params = [];

    var call = createCallUrl(this.url, this.apiToken, method, params);

    api_test(call, type, data, function (result) {
      ok(result.success, method + " method call failed");

      var tasks = result.data.tasks;
      ok(tasks.length >= 1, "tasks has not been returned");

      var task = result.data.tasks[0];
      ok(task.id !== undefined, "Id field is absent");
      ok(task.description !== undefined, "Description field is absent");
      ok(task.status !== undefined, "Status field is absent");
      ok(task.createdDate !== undefined, "CreatedDate field is absent");
      ok(task.startedDate !== undefined, "StartedDate field is absent");
      ok(task.stoppedDate !== undefined, "StoppedDate field is absent");
    });
  });


* This source code was highlighted with Source Code Highlighter.

Add new task method test:

  test("task add method", function () {
    var method = 'tasks/add';
    var data = { description: 'new task 1' };
    var type = 'POST';
    var params = [];

    var call = createCallUrl(this.url, this.apiToken, method, params);

    api_test(call, type, data, function (result) {
      ok(result.success, method + " method call failed");
      ok(result.data != null, "data is null");
      ok(result.data.task.id > 0, "id for first item is wrong");
    });
  });


* This source code was highlighted with Source Code Highlighter.

Delete task method test:

  test("delete task method", function () {
    var me = this;

    var method = 'tasks/all';
    var data = null;
    var type = 'GET';
    var params = [];

    var call = createCallUrl(this.url, this.apiToken, method, params);

    api_test(call, type, data, function (result) {
      ok(result.success, method + " method call failed");

      var taskId = result.data.tasks[0].id;
      ok(taskId >= 1, "could not get task for deletion");

      var method = 'tasks/delete/' + taskId;
      var data = null;
      var type = 'DELETE';
      var params = [];

      var call = createCallUrl(me.url, me.apiToken, method, params);

      api_test(call, type, data, function (result) {
        ok(result.success, method + " method call failed");
        ok(result.data.id != null, "data is null");
      });
    });
  });


* This source code was highlighted with Source Code Highlighter.

Rest of tests are available on github, check it out to get additional ideas.

Running tests

As any kind of qUnit tests they could be easily run in browser.

For continues integration system, they have to be run from command-line. It is easily possible using FuncUnit + Selemium Server and described here.

Debugging the tests

Sure, you need to be able to run tests under debugger to see what might went wrong. For debuging test code, there is nothing better than FireBug. Just place the breakpoint on a line you need, press F5 to restart tests.

If you need to debug actual API implementation code (which in my case is C#/ASP.net MVC application), I start the web site under debugger (F5 in VS2010), place breakpoint in corresponding method and press F5 in browser to to restart tests.

Conclusions

I liked the idea of those integration tests by means of javascript. I was happy to get final results: the interface is more strict and more corresponds to REST principles. It is much more faster to write tests with javascript instead of C# or Java. Just compare this and this and feel the difference. Write less, get more.

As javascript could be treated as "pseudo language", since it is easy to read it - API test suite could be used as a developers documentation. If you need to do a call, check the corresponding test, everything there.

Let's take a REST

Nowadays, REST is becoming so popular, that any web developer must take it into consideration while architecting new application. REST is acronym for Representational State Transfer and being formulized first by Roy Fielding in his PhD dissertation.

What is REST?

It is rather style or pattern of development resource-oriented web applications. Beauty of REST is that its really easy to understand and basically you are using REST everyday but may not noticing that. REST works on top of HTTP protocol, but is is not protocol itself. It seems to me that it actually appears with HTTP/1.1 but only with Roy Fielding work it became well understood, defined and attractive.

REST popularized by such applications as twitter, flickr, bloglines, technorati etc. And of cause, by Ruby On Rails framework.

Unique ID

Unique ID is key concept in REST. Everything in Web is resource, every resource must be addressed, each address is unique.

Here are examples of ID. Of cause in a world of HTTP ID's are URI's (Unified Recourse Identifiers):

http://mysite.com/blog/page/1
http://mysite.com/blop/post/my-first-post

What is representation and state transfer?

Let's see the first URI, it point for first page on some blog. As a client I ask for resource, representation of resource is returned back to client. By receiving the representation client transfers (changes) to particular state. As I ask for next resource, next representation of resource is back. The new representation changes the client application into yet another state. Between previous and next state, client stays is rest mode. Thus, the client application transfers state with each resource representation. That forms concept of - Representational State Transfer.

It doesn't depend what exact representation is, it could be: HTML, XML, JSON, RSS etc.

Recourses and actions

You can do a different actions to resources. REST achitecture maps CRUD (Create, Read, Update, Delete) to the set of operations supported by the web service using HTTP methods (e.g., POST, GET, PUT or DELETE).

URI Action Description
http://mysite.com/blog/page/1 GET Gets the representation of resouce. Get request does not change the state of server. Typically you do not submit any data in GET request. You can read the URL like, "get page 1 from blog located at http://mysite.com"
http://mysite.com/blog/entries/entry POST Posts the changed state of resouce. With post you can change the state of server. Post contains data in POST body. It is up to web server how to treat and use this data. You can read URI like, "post an entry to entries collection in blog located at http://mysite.com"
http://mysite.com/blog/entries/entry/changename/211 PUT Updates some existing resouces. PUT is similar to POST, since it changes web server state and contains data in body. You can do different change actions that would update resource. Typically recourse identified by ID, like 211. You can read URI like, "update an entry with Id 211, by changing its name, in entries collection in blog located at http://mysite.com"
http://mysite.com/blog/entries/entry/211 DELETE Deletes some existing resouces. DELETE changes web server state but typically contains no data in body. You can read URI like, "delete an entry with Id 211, in entries collection in blog located at http://mysite.com"

Logical and Physical URL's

If you do a lot of classic ASP.net programming you probably get used that URL reflects physical structure of application. For instance, http://mysite.com/index.aspx corresponds to c:\inetpub\wwwroot\mysite\index.aspx. In REST style URL stand not for physical, but logical URL. It means, http://mysite.com/blog/post/1 doesn't have to have c:\inetpub\wwwroot\mysite\blog\post\1 file with static content.

Clean and logical URL's one of the attractive points of REST. It moves away from ugly URL's like http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/ref=sr_1_1?s=books&ie=UTF8&qid=1293077887&sr=1-1.

From implementation point of view: to allow REST style URL for ASP.net applications you should either create your own HttpHandler or use already created start-kits and frameworks, like WCF REST starter kit, OpenRasta.

REST vs. SOAP

Sure, REST is not first who approaches issue of using recourses in Web, it rather trendier new kind in a block. We know bunch of web technologies, SOAP, WSDL, ATOM, WS-*, WCF, ODATA and many many more.. So, what are the difference?

Major difference is that all above are protocols, but REST is style. This has pros and cons. Protocol's are more strict and heavyweight, with a number of rules, formats etc. SOAP is using XML as data exchange format, REST could work any format depending on client needs. SOAP is using its own security model, REST relies on HTTP and web server security. SOAP requires tools, REST learning curve is small and less less reliance on tools. SOAP designed to handle distributed computing environments, REST assumes a point-to-point communication model.

But my opinion is simplicity always win against complexity. Key popularity of REST is because is simple, easy understand by developers and as a result - implemented in applications. My believe that SOAP and other heavyweight protocols will slightly die more and applications will be using REST.

ASP.net MVC and REST

Developers of ASP.net MVC framework designed it to be REST compatible. In the level of framework, there is an URL routing system (System.Web.Routing) that allows you easily follow REST design principles. It gives you total control over your URL schema and its mapping to your controllers and actions, with no need to conform to any predefined pattern.

So, basically ASP.net MVC web applications development is: create a controller class (LoginController for instance), implement number of actions (Index, CheckCredentials) and map those actions to particular URL. For instance http://mysite.com/login mapped to LoginController.Index method, that handles GET request from server and return View, containing Login form. http://mysite.com/login/check mapped to LoginController.CheckCredentials method, that handles POST and checks users credentials.

It is much more easier to create web applications API's with MVC framework. The ActionResult is polymorphic, so it could return HTML, JSON, XML results (and you are free to implement own ActionResult, for any format you might need).

Functional testing by javascript with FuncUnit

Functional tests are something that is better to use as early as possible, to get any valuable results. Obvious choice for ASP.net applications are Seleminum for .NET, WatiN or SpecFlow. But all of them are supposed to use C# as programming language and run their tests by special runners. Thought the experience I saw how cool is to write tests with full dynamic languages, like javascript and be able to run tests directly from browser, so I can debug some broken functionality. As I saw FuncUnit project and it really attracted my attention! Here I describe some initial experince of testing ASP.net MVC application with FuncUnit.

What it is all about?

FuncUnit is very compact and elegant functional testing framework. It combines the power of jQuery, qUnit, Selemium, Synthetic. Later it was a part of javascriptMVC framework, but now it is available as separate framework. Major features are:

  • It is only javascript
  • Could run tests in browser mode and command line by means of Selemium
  • Simple API
  • Easy to debug
  • Automate your existing qUnit tests
  • Run file mode and server page mode

Yes, it simple.. just from the reading of documentation and downloading the package you are ready to start you testing. I've managed to cover mostly all functionality of my small application in 3 hours or so.

Integration

Download FuncUnit package here. It contains all required javascript files, as well as selemium jar's, so as you have Java runtime installed on your machine you immediately able to run tests from command line. FuncUnit does not say what particular framework should be behind of your application, it would work with any.

It requires that tests page are on the same level as tested page. At the beginning it was confusing me, because in ASP.net MVC we don't have pages at all, only controllers and corresponding actions; my existing qUnit tests were placed to Scripts/Tests/index.html runner and I expected to have same layout. But it have to be changed a little for FuncUnit. So, filemode testing is not really applicable for ASP.net MVC, you should go with server pages mode.

I placed both FuncUnit and qUnit test pages at the root of application. Test code itself is referenced from Scripts/Tests/acceptance and Scripts/Tests/acceptance.

Content of FuncUnit placed to Scripts/Test/framework, so I got something like:

Test page

It is very similar (even identical) to qUnit page. Just reference framework and tests scripts.

<html>
  <head>
    <title>Trackyt.net Functional Test</title>

    <link rel="stylesheet" type="text/css" href="http://v3.javascriptmvc.com/funcunit/dist/qunit.css" />
    <script type='text/javascript' src='Scripts/Tests/framework/funcunit/funcunit.js'></script>
    
    <!-- Tests -->
    <script type='text/javascript' src='Scripts/Tests/acceptance/tests.home.js'></script>
    <script type='text/javascript' src='Scripts/Tests/acceptance/tests.signup.js'></script>
    <script type='text/javascript' src='Scripts/Tests/acceptance/tests.bugs.js'></script>
  </head>
  <body>

    <h1 id="qunit-header">FuncUnit Test Suite</h1>
    <h2 id="qunit-banner"></h2>
    <div id="qunit-testrunner-toolbar"></div>
    <h2 id="qunit-userAgent"></h2>
    <ol id="qunit-tests"></ol>
  </body>
</html>


* This source code was highlighted with Source Code Highlighter.

Test code

As I said its fun and easy to write tests with FuncUnit. You have everything that should be in functional testing framework: open pages, read values and put values to controls, wait for events.. clicking and dragging mouse. Tests code/layout is the same as you get used with qUnit. Each test basically does:

  • Open test page, by S.open("page_url_from_root") (ex. S.open("Admin/Login"), will run http://localhost/app/admin/login)
  • Wait for some html elements appear in browser, with S.wait();
  • Type some values to control S('#myControlId').type('bla-bla') and read values S('#anotherControlId').val();
  • Assert for results with ok, same, equal methods;

I'll do several example from my code.

Simple one, I check that user is not able to login with empty password:

// I as user could not login with empty password
test("password is empty", function () {
  // arrange
  S('#Email').type('a@a.com');

  // act
  S('#submit-button').click();

  // assert
  S('#PasswordValidationMessage').visible(function () {
    var message = S('#PasswordValidationMessage li:nth-child(1)').html();

    ok(message == "Password is empty", "I as user could not login with empty password");
  });
});


* This source code was highlighted with Source Code Highlighter.

Or that I'm not able to register with already registered email:

// I as user could not register with same email twice
test("register twice", function () {
  // arrange
  var email = "test" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
  S('#Email').type(email);
  S('#Password').type(email);
  S('#ConfirmPassword').type(email);
  S('#submit-button').click(function () {

    // wait till registration is done
    S.wait();

    // act
    // go back to register page and try to register with same credentials
    S.open("Registration", function () {
      S('#Email').type(email);
      S('#Password').type(email);
      S('#ConfirmPassword').type(email);
      S('#submit-button').click(function () {

        // assert
        S('#PasswordValidationMessage').visible(function () {
          var message = S('.validation-summary-errors li:nth-child(1)').html();
          same(message, "Sorry, user with such email already exist. Please register with different email.", "I as user could not register with same email twice");
        });

      });
    });
  });
});

* This source code was highlighted with Source Code Highlighter.

To more complex, bug reproducing issues:

// https://github.com/alexanderbeletsky/Trackyourtasks.net/issues/#issue/20
test("tasks are disappeared", function () {
  // go to sign up page
  S.open("Registration");

  // create new account
  var email = "test_bugs" + new Date().getSeconds() + new Date().getMilliseconds() + "@trackyt.net";
  S('#Email').type(email);
  S('#Password').type(email);
  S('#ConfirmPassword').type(email);

  S('#submit-button').click(function () {
    S('#tasks').exists(function () {
      // create several tasks
      S('#task-description').click().type("fix issue 20");
      S('#add-task').click();
      S.wait(500);

      S('#task-description').click().type("fix issue 20 2");
      S('#add-task').click();
      S.wait(500);

      // and log off of dashboard
      S('#sign-out').click();
    });
  });

  // now sign in with same account
  S.open("Login");
  S('#Email').type(email);
  S('#Password').type(email);
  S('#submit-button').click(function () {

    // wait till tasks are ready
    S('#tasks').exists(function () {
      S.wait(function () {
        var tasks = S('.task').size();

        // tasks created before must exist
        ok(tasks == 2, "tasks created before must exist");
      });
    });
  });
});


* This source code was highlighted with Source Code Highlighter.

I found that those tests are also a kind of documentation! I could document user stories with such tests and maybe even use tools to generate documentation just from tests code.

Running tests from browser

While development it is more preferable to run tests directly from browser. To do that I just select FunctionalTests.html file in Solution Explorer and press Ctrl+Shift+W that mean "View in browser". Please note, that FuncUnit tests are run in pop-up window, so you have to enable pop ups in your browser. Browser will open http://localhost/virtualdir/FunctionalTests.html and tests will run. For more convenience I also put a bookmark for test page in browser, so I could always re-run all tests by one click.

Running tests from command-line

Major feature of FuncUnit is of cause it's ability to run from command-line, so you can easily integrate to your continuous integration system. There a script envjs.bat that would easily to start the tests under Selenium and provide with results as command line output (you can also run qUnit tests from command line with envjs.bat).

From the root of web application, I start:

.\Scripts\Tests\framework\funcunit\envjs.bat http://localhost/tracky/FunctionalTests.html

BTW, there is a small issue in envjs.bat that stops me to successfully run it initially, it does not take into accounts that it could be run as envjs.bat, but simpy as envjs. Please see my version of this file here.

Deployment

Sure you don't want to make those tests public on your site. There are 2 options: change Web.config to disable users to access FunctionalTets.html, UnitTests.html or just delete those 2 before deployment to production. Since I do simple xcopy deployment, option 2 works for me.

Demonstration

To make picture full, I prepared small screencast that would demonstrate how I used the framework. I like it, it fits my requirements (at least for now). I hope you also like it and find this information useful! And also, you should check the repository for test code examples.

Run and debug your tests with shortcuts

While I'm working in Visual Studio, I try to use my mouse as less as possible. You should be really comfortable to do 99% of activities just by keyboard. It saves a time.

I re-run tests hundred times per day, so clicking by a mouse is a bad option. Doesn't matter you use Testdriven.net or Resharper, you have ability to use shortcuts for quick run.

Testdriven.net

Tools -> Options -> Keyboard -> Testdriven.NET.RunTests and assign it to "Ctrl + R, Ctrl + T" (run tests)

Tools -> Options -> Keyboard -> Testdriven.NET.Debugger and assign it to "Ctrl + R, Ctrl + D" (run debugger)

Resharper

Tools -> Options -> Keyboard -> Resharper.Resharper_UnitTest_ContextRun and assign it to "Ctrl + R, Ctrl + T" (run tests)

Tools -> Options -> Keyboard -> Resharper.Resharper_UnitTest_ContextDebug and assign it to "Ctrl + R, Ctrl + D" (run debugger)