In this post I’ll explain what Lightweight Test Automation Framework does and how you can use it with ASP.NET MVC.

What is Lightweight Test Automation Framework?

Lightweight Test Automation Framework is an *integration testing *tool, which works by scripting the web browser to perform a sequence of actions against your application’s UI – entering text and clicking links and buttons exactly as a real user would – and then checking that the expected results can be seen in the browser window.

Just in case you don’t know the difference between unit testing and integration testing:

• Unit tests work against the API of a particular component in your code. Ideally, they test only that single isolated unit (hence the name), which allows them to pinpoint any problems exactly where they occur. Unit testing tools for .NET include NUnit, mbUnit, and xUnit.
• *Integrations tests *combine multiple components. Typically, they work at the UI level to test that your JavaScript, your HTML, your controller code, your model code, and your database all work together. Integration testing tools for .NET include Selenium, Watin, and Lightweight Test Automation Framework.

How does it work?

Note: If you’re not interested in the mechanisms that run behind the scenes, just skip ahead to the tutorial “How to use Lightweight Test Automation Framework with ASP.NET MVC” below.

With Lightweight Test Automation Framework, you write tests in C# (or another .NET language) – each test looks very similar to a traditional unit test. You write a series of commands (e.g., navigate to this URL, click that button, get the text from that element, assert that the text is XYZ).

Instead of running these through a traditional test runner, such as NUnit GUI or Visual Studio’s built-in unit test runner, Lightweight Test Automation Framework’s test runner actually is *the web browser – *any web browser. Each command (e.g., navigate to this URL, click that button) is delivered from the server to the client using Ajax.

All you have to do is add to your web application a folder called Tests containing the supplied Default.aspx and DriverPage.aspx, and then browse to /Tests. Here’s how it works:

As the tests run, TestcaseExecutor.js highlights passing tests in green and failing tests in red. You can click on any red test to see the server-generated stack trace corresponding the failed assertion or unhandled exception.

##

How does this compare to Selenium RC?

Selenium is a well-established open source web integration testing tool. Just like Lightweight Test Automation Framework, it automates your application’s UI so you can verify the results of performing a known sequence of actions. There are two main ways of using it:

• Selenium IDE is a Firefox plugin that records a sequence of actions and generates a test script based on what you did.
• Selenium Remote Control (RC) is a system for writing test scripts in C#, VB, JavaScript, Python, etc, which are then used to remotely control the browser UI and make assertions about what your application should display. This is what I was using to test xVal’s client-side behaviour.

Clearly, Lightweight Test Automation Framework is very similar to Selenium RC. There is a significant architectural difference, though. Selenium RC integrates with any traditional unit test runner – it does not use the browser as a test runner. Selenium RC gives you a browser automation API which works by making remote procedure calls (RPCs) to a Java-based “test server” running on your development workstation. The Java-based test server then in turn hosts a browser instance (Firefox, IE, or Safari) and uses the browser’s API to run your commands in the browser. There’s no Ajax involved. Here’s how it works:

So, how do we weigh up the pros and cons of each approach?

• Selenium RC has the advantage that your application **isn’t **hosted in a frameset (so the test environment can’t interfere with the code being tested)
• Selenium RC has the advantage that it works with any traditional unit testing framework (e.g., NUnit or xUnit) and any traditional unit test runner (e.g., NUnit GUI or ReSharper’s test runner). Your test code calls the Selenium RC API and makes plain old assertions about the results
• Lightweight Test Automation Framework has the disadvantage that it relies on an asynchronous client-side polling loop (unlike Selenium RC’s RPCs, which run synchronously). Therefore, Lightweight Test Automation Framework is relatively slow – each test takes 1-3 seconds to run, versus Selenium RC which takes about 1 second per test. If you have a lot of tests then you’ll really care about this. Tip: Run your tests in Google Chrome, where tests run about 50% faster than in IE, and at least twice as fast as in Firefox.
• Lightweight Test Automation Framework has the advantage that it doesn’t need the Java-based “test server”, and it works in any browser – not just ones it knows how to host.

Why I’m moving from Selenium RC to Lightweight Test Automation Framework

My scenario was automating tests for xVal’s client-side code. For every possible type of validation rule, we have a test to check certain inputs that should trigger a validation error message and other inputs that should be accepted. This implicitly tests all parts of xVal:

• The system of defining and detecting each type of rule
• The system of describing each type of rule to the client
• The system of detecting and configuring the active client-side validation engine (e.g., jQuery Validator)
• The system of customising validation error messages
• The client-side validation engine itself

If any of these stop working, an integration test should detect it.

Selenium RC was doing a perfectly fine job of this, but the critical fault is that you have to run a “test server” on your workstation, and that test server is written in Java. Now I’m perfectly happy with Java, but this setup added a lot of unwanted friction in an open source .NET project. Most .NET developers don’t even have the Java runtime on their PCs, and even those that do wouldn’t realise that they have to install the Selenium RC server and run it before the unit tests could pass.

Lightweight Test Automation Framework eliminates the Java test server and makes it far easier for people to contrib
ute to xVal. That’s why I’m switching to it. The main disadvantage is that client-side tests now take longer to run, but as I pointed out before, Lightweight Test Automation Framework has the extra benefit of supporting Google Chrome, which itself is so fast that the speed loss is hardly noticeable.

How to use Lightweight Test Automation Framework with ASP.NET MVC

The ASP.NET QA team say that the Lightweight Test Automation Framework is designed to work with “ASP.NET”. They don’t say what form of ASP.NET they’re talking about, but from experience, I can tell you it’s designed with ASP.NET WebForms and ASP.NET AJAX in mind.

Nonetheless, it also works fine with ASP.NET MVC if you make a certain change to your routing config and avoid one or two features that rely on ASP.NET AJAX.

Let’s imagine you have a small ASP.NET MVC application that responds to Ajax requests. It renders an Ajax-powered form as follows:

<h2>The Magnificent Age Calculator</h2>
The following form is submitted via Ajax:
 
<% using(Ajax.BeginForm(new AjaxOptions { UpdateTargetId = "results" })) { %>
    <p>Current year: <%= Html.TextBox("currentYear", DateTime.Now.Year) %></p>
    <p>Your year of birth: <%= Html.TextBox("yearOfBirth") %></p>
 
    <p><input type="submit" id="submitButton" value="Calculate My Age" /></p>
<% } %>
 
<div id="results"></div>

Since we’re using Ajax.BeginForm(), the form will be submitted asynchronously, and the results will be injected into the DIV with ID “results”. The following action methods handles the Ajax requests:

[AcceptVerbs(HttpVerbs.Post)]
public string Index(int? currentYear, int? yearOfBirth)
{
    if(!currentYear.HasValue)
        return "Please enter the current year";
    if (!yearOfBirth.HasValue)
        return "Please enter your year of birth";
 
    var computedAge = currentYear.Value - yearOfBirth.Value;
    return "Your age is approximately: " + computedAge;
}

This behaves as shown:

This Ajax-powered UI behaviour cannot be tested using traditional unit testing tools, which focus on testing server-side code only. However, you can use Lightweight Test Automation Framework to automate the browser and test the application’s UI behaviour.

To get started, download Lightweight Test Automation Framework and then add a reference from your project to Microsoft.Web.Testing.Light.dll. Then create a test method as follows:

[WebTestClass]
public class AgeCalculatorTests
{
    private const int ajaxRequestTimeoutSeconds = 1;
 
    [WebTestMethod]
    public void Can_Calculate_Age()
    {
        // Arrange
        HtmlPage page = new HtmlPage("/");
 
        // Act
        page.Elements.Find("currentYear").SetText("2010");
        page.Elements.Find("yearOfBirth").SetText("1932");
        page.Elements.Find("submitButton").Click();
 
        // Assert
        var resultsElement = page.Elements.Find("results");
        resultsElement.WaitForInnerText("Your age is approximately: 78", ajaxRequestTimeoutSeconds);
    }
}

As you can see, it uses the test framework’s API to navigate to a certain URL, enter some text, click the submit button, and check the result.

To run the test, copy the /Test/ folder and its contents from the Lightweight Test Automation Framework project into your ASP.NET MVC project. To work around an incompatibility with ASP.NET MVC routing, add the following line to your routing config:

routes.IgnoreRoute("Test/{resource}.axd/{*pathInfo}");

Now you can open /Test/ in a browser and you’ll get the test runner. You can run the test you just created:

You can then proceed in a TDD fashion, adding a test for each new UI behaviour (before you’ve coded the behaviour!), seeing the test fail, adding the behaviour, and then seeing the test pass.

For example, perhaps there should be a special error message if you claim to be born in the future:

[WebTestMethod]
public void Gives_Error_If_Birth_Year_In_Future()
{
    // Arrange
    HtmlPage page = new HtmlPage("/");
 
    // Act
    page.Elements.Find("currentYear").SetText("2009");
    page.Elements.Find("yearOfBirth").SetText("2010");
    page.Elements.Find("submitButton").Click();
 
    // Assert
    var resultsElement = page.Elements.Find("results");
    resultsElement.WaitForInnerText("Year of birth must be in the past", ajaxRequestTimeoutSeconds);
}

I haven’t yet implemented this behaviour, so the test runner will show a failure:

If you want to play with this yourself, then download this demo project.

Bugs and Gotchas

Besides the minor issue with ASP.NET MVC routing, I also hit a few more confusing problems as I was porting xVal’s test code to Lightweight Test Automation Framework:

• Test classes (i.e., ones marked with [WebTestClass]) must not be in a namespace, otherwise the test runner will break. I’m pretty sure this must be a bug, so I’ve reported it.
• You can’t use the “WaitFor” API. This API would be very neat if it worked: it’s supposed to help you pause the test execution to leave time for asynchronous requests to complete. However, it assumes you’re doing all your Ajax requests through ASP.NET AJAX with a script manager, so it just throws an exception if you’re using jQuery or even ASP.NET MVC’s Ajax.* helpers. That’s why in my tests above I had to use resultsElement.WaitForInnerText() to wait for the Ajax requests to complete.

Other than these issues – which have obvious workarounds – it seems to work very nicely. I’m looking forward to see where this project goes in future.