Site Meter
 
 

Behavior Driven Development (BDD) with SpecFlow and ASP.NET MVC

Test Driven Development (TDD) has been around for about a decade, and has been mainstream for at least five years now. During this time, TDD practitioners have been gradually changing and refining the methodology, the mindset, and the terminology in an effort to increase its usefulness and avoid some of the problems that newcomers often experience.

The mindset and terminology has shifted so much that some folks have started using a new name for the updated methodology: BDD. Here’s a brief overview*:

  • Test Driven Development seeks to improve software quality by getting you to follow an iterative workflow often called red-green-refactor. In theory, it keeps quality high because if any change breaks earlier functionality, you should be notified by way of a failing test. Also, it should help you to make more pragmatic decisions about what code to write, because you only write the simplest code needed to make all your tests pass.
    Despite its unfortunate name, it became clear over time that TDD’s greatest potential benefit is not as a technique for testing code or detecting bugs, but in fact as a code design aid, because it provides a precise mechanism for specifying how your code should behave before you get mentally caught up in the actual implementation. However, in reality, I’ve seen many developers still struggle to break free from a test-oriented mindset and continue to amass reams of unit tests that simply exercise every possible code path without usefully describing what this proves or why it represents correct behaviour. This doesn’t aid design, very rarely detects bugs, and yet consumes a huge amount of maintenance time.
  • Behaviour Driven Development retains the basic red-green workflow, but dramatically puts the emphasis on specifying behaviours that are understandable to people (often, business domain experts). It addresses questions such as “How much should I specify?” and “How should I organise and name my specifications so they are most useful?”
    To do this, BDD says you should elevate your mind to a level of behavioural abstraction above the code implementation. So don’t say “Constructor_Test” or even  “Constructor_NullParam _ThrowsArgumentNullException” but instead say “A shipment cannot be created without a recipient address”. The change of emphasis and terminology leads people to write more useful specifications.
    I won’t focus on the term BDD itself, because the term is not hugely useful: nobody has ever given it a precise definition as far as I’m aware, and there isn’t a lot of consensus about what it should mean (some say it should only be about the UI; others say not). Instead, in this blog post, I’m going to focus on the major tools that have come out of the BDD stable and the advantages they can give you.
Unit level or UI level?

You can write BDD-style specifications about individual code units, in which case you’ll  probably use a context/specification style framework like RSpec (for Ruby) or in .NET something like MSpec or SpecUnit (personally, I don’t like either). Unit-level specifications work brilliantly for most code that ultimately exposes an API for other code. But personally, I spend most of my time writing UI code, usually with ASP.NET MVC, and UI code is fundamentally different. It isn’t about producing an API – it’s about producing user experience (UX) behaviours. These behaviours typically aren’t atomic (contained within a single click or HTTP request); their essence exists only across a sequence of interactions. As such, unit-level specifications can fall short and fail to capture the UI behaviours you have in mind. I wrote before about why, in real projects, I’ve found unit tests to be of limited value for ASP.NET MVC controllers.

Alternatively, you can write BDD-style specifications about UI interactions. Assuming you’re building a web application, you’ll probably use a browser automation library like WatiR/WatiN or Selenium, and script it either using one of the frameworks I just mentioned, or a given/when/then tool such as Cucumber (for Ruby) or SpecFlow (for .NET). In this blog post, I’ll describe these tools and provide an example based on ASP.NET MVC.

* I don’t claim to be an authority on BDD, so if you think I’ve got this all wrong, I invite you to come along and tell me what I’ve misunderstood. Try to be specific, please. My existing understanding is based on having done a lot of TDD, seeing its limitations especially in large teams, and recognising that some of these BDD ideas look like they could help with that.

So, what’s Cucumber?

image Cucumber is a Ruby tool that offers a natural-language way to specify behaviours in terms of “Given, When, Then” (GWT) sentence fragments. This language is called Gherkin. You create a plain-text file (called a “.feature” file) containing a set of scenario definitions written in Gherkin, for example

Scenario: Show logged in user name
  Given I am logged in as a user called "Steve"
   When I visit the homepage
   Then the page header displays the caption "Hello, Steve!"

… and also provide Ruby scripts that define what the runner should do when it hits matching steps, for example

Given /I am logged in as a user called "(.*)"/ do |name|
    create_user(name)
    sign_in_as(name)
end
 
...
 
Then /the page header displays the caption "(.*)"/ do |caption|
    page_header.should_contain(caption)
end

The syntax is extremely flexible: Cucumber only cares about the Gherkin keywords “Given”, “When”, “Then”, “And” (plus a few others); everything else is simply matched against your regular expressions. Any tokens captured by capture groups in your regexes are passed to your step definition blocks as parameters. For web applications, it’s normal for your step definitions to work by using a browser automation library such as WatiN/WatiR or Selenium.

When you invoke Cucumber, it finds, parses and runs your .feature files, reporting the results (success in green, pending in yellow, failure in red):

image

Because the GWT model inherently describes sequences of interactions, and because natural-language .feature files are inherently decoupled from the implementation code (they don’t mention the name of any classes or methods), this whole approach is a perfect fit for UI automation rather than unit-level testing. 

Because you can describe scenarios in the language of your domain and user interface, your resulting feature files should make perfect sense to business experts. So, unlike traditional unit test driven development, this methodology can finally deliver on the promise of executable specifications that are actually useful as documentation.

Doing it in .NET with SpecFlow

image

As I mentioned, Cucumber itself is a Ruby tool. At the end of this blog post I’ll discuss some options for running the original Cucumber tool in a .NET environment, but for now let’s forget about that – you can sidestep all that complexity with SpecFlow.

SpecFlow is an open-source .NET tool that lets you write specifications using 100%-Cucumber-compatible Gherkin syntax, and has a number of advantages over Cucumber itself:

  • It integrates with Visual Studio, which means you get File->New templates for creating new feature files
  • More crucially, it gives complete VS debugger support, so you can set breakpoints on Given/When/Then lines in your .feature files and step through their execution
  • You can implement your step definitions in any .NET language
  • When you compile a project containing SpecFlow feature files, the output is an NUnit test assembly, so you can use your favourite NUnit-compatible test runner or existing CI infrastructure to run the specifications with no additional configuration.
  • It doesn’t have such a ridiculous name :) It’s sad, but at one of my recent clients, several managers refused to take Cucumber seriously and wouldn’t pay attention to Cucumber specifications purely because of the name. That’s the real world for you!

Example: Combining ASP.NET MVC, SpecFlow, and WatiN

To give you something concrete to get started with, I’ve made a very simple ASP.NET MVC 2 example application, built following the BDD workflow and with a full set of specs. To try it out,

You should easily be able to understand in high-level terms what the application does by reading the Gherkin specifications. Just imagine these specifications are the written result of a design meeting where you’ve got your client or boss to tell you how the user interactions should work.

Feature: Browsing
   In order to see who's been on the site
   As a user
   I want to be able to view the list of posts

Scenario: Navigation to homepage
   When I navigate to /Guestbook
   Then I should be on the guestbook page

Scenario: Viewing existing entries
   Given I am on the guestbook page
   Then I should see a list of guestbook entries
    And guestbook entries have an author
    And guestbook entries have a posted date
    And guestbook entries have a comment
Feature: Posting
   In order to express my views
   As a user
   I want to be able to post entries into the guestbook

Scenario: Navigation to posting page
   Given I am on the guestbook page
   Then I should see a button labelled "Post a New Entry"
   When I click the button labelled "Post a New Entry"
   Then I am on the posting page

Scenario: Viewing the posting page
   Given I am on the posting page
   Then I should see a field labelled "Your name"
    And I should see a field labelled "Your comment"

Scenario: Posting a valid entry
   Given I am on the posting page
     And I have filled out the form as follows
       | Label          | Value             |
       | Your name      | Jakob             |
       | Your comment   | Das ist gut!      |
   When I click the button labelled "Post"
   Then I should be on the guestbook page
    And I see the flash message "Thanks for posting!"
    And the guestbook entries includes the following
       | Name      | Comment      | Posted date          |
       | Jakob     | Das ist gut! | (within last minute) |

Whenever you hit save while editing one of these files, SpecFlow generates a C# NUnit test fixture containing a [Test] method for each scenario. You don’t have to know or care about that, though. All you have to do is create step definitions in C# to match the lines in the Gherkin files.

At first, you won’t have any matching step definitions, so the NUnit test runner will show the tests as “inconclusive”:

image

Notice that in the “Text Output” tab it provides C# stub code to create a matching step definition. Here’s some examples of the finished step definitions, written using WatiN to automate a browser:

[When(@"I navigate to (.*)")]
public void WhenINavigateTo(string relativeUrl)
{
    var rootUrl = new Uri(ConfigurationManager.AppSettings["RootUrl"]);
    var absoluteUrl = new Uri(rootUrl, relativeUrl);
    WebBrowser.Current.GoTo(absoluteUrl);
}
 
[Then(@"I see the flash message ""(.*)""")]
public void ThenISeeTheFlashMessage(string message)
{
    var flashElement = WebBrowser.Current.Element("flashMessage");
    Assert.That(flashElement.Text, Is.EqualTo(message));
}
 
[Given(@"I have filled out the form as follows")]
public void GivenIHaveFilledOutTheFormAsFollows(TechTalk.SpecFlow.Table table)
{
    foreach (var row in table.Rows) {
        var labelText = row["Label"] + ":";
        var value = row["Value"];
        WebBrowser.Current.TextFields.First(Find.ByLabelText(labelText)).TypeText(value);
    }
}

There’s lots more I could say about WatiN syntax and Gherkin best practices, but this is a pretty long post already so I’ll leave that to others.

In case anyone’s wondering, I’m not trying to claim with this post that Cucumber-style UI automation tests are the one true way to do BDD, or that unit testing is dead. UI automation tends to be much more difficult to do than unit-level tests, because there are many more moving parts and you have to concern yourself with test data, setup and teardown, etc. Seriously, it is hard work, and requires a lot more persistence and dedication than mere unit testing, though it can also give many extra benefits especially with regard to detecting regressions.

What about Running Cucumber Itself on .NET?

If, for some reason, you don’t want to use SpecFlow and would rather run the original Ruby-based Cucumber tool, there are two main options for .NET programmers:

  • Use IronRuby. The current version of IronRuby is entirely capable of running Cucumber. It isn’t as easy to set up as SpecFlow, but only took me 30 minutes or so. The drawbacks are that (1) you have to write step definitions in Ruby, unless someone can come up with a clever way to call arbitrary .NET methods directly, (2) it runs Cucumber pretty slowly, at least in my testing, and (3) it doesn’t trivially integrate with existing CI infrastructure like an NUnit-based solution does.
  • Use Cuke4Nuke. This is an open-source project that aims to bridge the gap between the original Cucumber tool running on any Ruby interpreter (not necessarily IronRuby) and step definitions written in .NET. It works by hosting your .NET step definitions in a small .NET-based “server” application, which Cucumber communicates with over a TCP/IP connection. I must admit I had a bad experience trying to make this work – it took hours. The latest Ruby interpreter didn’t include all the necessary libraries for Cuke4Nuke, and then when I’d hacked all the dependencies together, it wasn’t clear to me where to put the magic “cucumber.wire” file or what parameters you can use with the .NET server app. And then it was also pretty slow compared to keeping everything in .NET with SpecFlow. And it had a very limited API for running setup/teardown code around scenarios.

If you want to write step definitions in a .NET language, I don’t know why you wouldn’t just use SpecFlow. Since it’s 100% Gherkin-compatible, your feature files could in future be moved to a different implementation if desired. If you’ve tried others, I’d be interested to hear about your experiences. And no, I haven’t tried Spectre yet.

Injecting Mocks into a Running ASP.NET Application

In the next post, I’ll show you one possible way to simplify or speed up UI-level specifications a little. It’s a cross-process code injection library that, among other things, you can use to inject a mock database into a running ASP.NET application without it knowing about it. Or, you can override configuration, or maybe simulate the passing of time.

Sometimes, it isn’t desirable to meddle with your app’s internals, but sometimes it is. Ruby coders can already do it using Webrat or cross-stub, and I’ve found reasons to want that power over .NET apps too.

kick it on DotNetKicks.com

52 Responses to Behavior Driven Development (BDD) with SpecFlow and ASP.NET MVC

  1. I’ve been doing BDD style tests from the controller down for a while now, but I always knew I was missing the part that was important to the end user. I’ve recently started looking at WaTin as a integration test tool but don’t want to go through the hassle of having a “Test” Database.

    I’m curious to see how you’ll mock the repository part of the system in a running Asp.Net application.

  2. Dominick

    This would make an outstanding tekpub video.

  3. Rob

    Hi Steve

    If you have time, I’d be really interested to know what you think of doing BDD with StoryQ (storyq.codeplex.net). The biggest difference between Specflow and StoryQ is that StoryQ is an internal DSL, so you don’t type code that’s not C# into visual studio. The advantage there is that the steps become more DRY (and refactorable) and that it’s a lot easier for developers to track what’s going on in their unit tests. If installing a VS plugin is an issue then StoryQ is also great.

    Sorry that this sounds like an ad for my OSS project, but I’m just trying to get you interested in trying it out! Your feedback would be invaluable.

    WRT the rest of your article, you seem to be spot on about what BDD is. Your distinction between C/S and GWT being for unit level / UI level is something I agree with but haven’t seen documented in many places. One thing though: it can be hard to find a link to the nascent Gherkin specification so I’ll post it here: http://wiki.github.com/aslakhellesoy/gherkin/

  4. paul

    Fantastic article, as always Steve. Are you planning on putting BDD concepts / UI testing concepts into your 2.0 version of the MVC book?

  5. Hi,

    Great article and thanks for using WatiN in your example.

    When using SpecFlow or Cuke4Nuke, I can recommend installing Cuke4VS (http://github.com/henritersteeg/cuke4vs) and you’ll get story syntax highlighting and a nice integration between your code and the stories/scenarios/steps.

    Hth,
    Jeroen (lead dev WatiN)

  6. av

    how about a Tekpub episode on this? I’ll pay for that!

    Thanks!

  7. Hello Steven,

    great article. I share many of you experiences and opionions of other frameworks, I mean how on earth can you call = () => readable for example…

    I been playing around a bit with SpecFlow and I like it a lot so far. One of the great features for me as being a developer in Sweden, is that it supports Swedish (and several other languages) for writing your .features in.

    That makes the whole “communication” bit with your customer much more interesting and real I think.

    Very interesting to see your use in conjunction with UI testing.

    Thank you
    /Marcus

  8. Steve,

    Great article thanks. Can I suggest that you have a look at StoryQ – http://storyq.codeplex.com. It is lightweight running within xUnit test runner of choice. I have combined it even with your IntegrationTest framework to great effect. For teams in dotnet new to BDD has it a low barrier to entry (having gone through the same exercise you document here).

    Keep up the great work.

    Cheers Todd.

  9. Hi Steve

    Great article, I’m relatively new to TDD and BDD (about a years experience) and have just started a new job where we are looking to improve our unit tests and move to CI with a look to introducing BDD, this article has been a great opener for me and our other devs and it would be interesting to see more.

    Rob mentioned the OSS StoryQ which was the tool I had found and was looking to implement seems a great tool. I particularly liked the reporting side of it and integration into Team City. It would be really good to see your thoughts on that compared to SpecFlow??

    Keep up the great work. Looking forward to Pro MVC 2 will this have any new approaches to TDD/BDD in it?

  10. Pingback: The Morning Brew - Chris Alcock » The Morning Brew #552

  11. [comment moved to http://bit.ly/dDsyOb (post about unit testing)]

  12. Steve

    Rob, Toddb, Andrew – Thanks for the pointers towards StoryQ. I had seen that before but didn’t at the time understand why an internal DSL (in C#) would be advantageous over an external one (e.g., Gherkin). Since you’ve all suggested it, I’ll have another look – maybe I missed an important benefit.

    Paul – I plan to get a little coverage of UI-level BDD into the MVC 2 book I’m currently working on. However, it’s really a totally separate (and huge) topic from ASP.NET MVC, so I can’t digress into it too deeply.

    Jeroen – Thanks very much for the Cuke4VS recommendation. I hadn’t heard of that before, though I was keenly aware of the need for IntelliSense and navigation assistance when you’ve got a lot of Feature files. That tool looks like it could be the missing piece I’ve been hoping to find :)

    Todd – since your comment was about unit testing practices, I hope you don’t mind me moving it to a different post that is about unit testing.

  13. Rob

    Steve and Andrew – internal DSL versus external DSL is probably a project-specific preference, but to me, the pros and cons as follows:

    External DSL:
    +++++ Can be written DIRECTLY by non-coders
    ++ The source is very easy to read
    – If the ubiquitous language evolves, it’s very hard to refactor old test suites. This can be partially solved by aliasing step methods (I think NBehave was the first to do this)
    - Slightly more ceremony around installing, using, writing and running the tests.

    Internal DSL:
    – Translating the english to code introduces friction (StoryQ has a converter GUI)
    - Source is a little harder to read, especially for non-coders. Workaround is to read the test runner output instead?
    +++ Refactorable: Since the code is DRYer (and, well, it’s code), it’s easier to refactor
    + Code completion: Developers can go a little faster with IDE support
    + Simplicity / Portability: You don’t have to install anything on your CI server or development machines, and it’s pretty obvious to new developers where the moving parts are / go.
    + Integration with existing unit test runners. I like the way specflow generates NUnit tests, but when we started StoryQ, NBehave and Cucumber (via ironruby) were the only options. Both of these needed to be run via an external console app, and I couldn’t live without my resharper test runner!

  14. ArnisL

    BDD + UI combo works only if application has good enough architecture. Most important thing – proper view model and ability to figure out parts of it from html. Otherwise all these tests are just smoke tests – they are too fragile.

  15. Hi Steve,

    Thanks for posting this. Very helpful. I have been wondering about Cucumber for some time, and I was not aware of SpecFlow. Also good summary of TDD vs BDD.

    Thanks!

  16. Steve,

    The assertion that xSpec frameworks are more appropriate to unit testing and xBehave (or Cucumber) frameworks are more appropriate to functional testing is a perpetuation of cargo cult hysteria. It’s a claim that has been repeated again and again over the past couple of years and has never substantiated. It’s a line that was made up as a way to differentiate rSpec from rBehave back in the day and has never held up to scrutiny. The only thing keeping this meme going is the people new to the tools and practices who are apt to repeat what they are told until they discover otherwise through practice and experience.

    I can, and do, as do many others, write functional specs using xSpec frameworks. Not only do I do this, I’ve found that it yields an all-around better result – for test producers as well as for consumers.

    Please stop perpetuating this fiction.

  17. Marcus,

    Since the assignment of a parameterless lambda can only be done using the ()=> symbols in C#, how do you otherwise get around it when using this language feature in other scenarios?

    Since your argument is about readability, can you describe how the use of this text makes specifications less readable in spec frameworks that make use of lambda assignment?

    What is your understanding of the cognitive mechanisms that enable readability?

    The lambda assignment syntax is a terminator. How does a terminator reduce readability if it appears after the part of the text where readability is the concern? Even if it does have an effect on readability, how does it effect solubility? Are you putting readability before solubility?

    Are you thrown off by other terminators in C#? Many VB developers don’t like C# because it uses the ; line terminator universally. How come you can be thrown off by one terminator and not the other? How come VB developers learn to stop paying attention to the ; terminator when they transition to C#? Are you stuck in the same cognitive trap as VB developers who can’t get over C#’s syntax?

  18. Drew

    You’ve just been Bellwared; stick that on your CV :)

    Ignore him he has his own agenda; context specification rules!

    No one would argue that functional testing cant be done using xSpec frameworks; in fact you don’t really need a framework at all. The point is it speeds up the process. Using a tool like specflow allows you to create fixtures in a natural language; these fixtures/conditions of acceptance are mapped directly to test skeletons which specflow generates for you.

    Our QA team and product teams are working quite happily with specflow.

    Would be interesting to hear how Bellware produces all-around better results using xSpec frameworks/context specification; what are the benefits over specflow/GWT?

    Cheers

  19. Scott,

    take it easy – it’s just code. Don’t strain yourself…

    I was just saying, without know that much about the “cognitive mechanisms that enable readability”, that I think that natural language is more readable than, for example =()=>.

    And I have never met an user/product owner/customer that think that code is more readable than a natural language such as Swedish.

    Come to think of it, when it come to the parameterless lambda, many programmers I know (myself included) have to think for a while when we see =()=>.

    I think =()=> is very expressive and compact – but not very readable.

    For the record I hate VB.NET, with a passion (http://www.marcusoft.net/2010/02/vbnet-considered-harmful.html). But I respect the people who want to use it enough to let them use it as they please.

    So, I like the SpecFlow (Cucumber-style) way of doing BDD over the MSpec-style. At least for now. I really hope I don’t stay like that forever. Just imagine… to already have decided everything… Brrr…

    I also like brass band music over heavy metal, apples over pears and C# over VB.NET.
    What about you?

    And I will most definitely put that I’ve been Bellwared. It’s an honor. :)

    Take care
    /Marcus

  20. Pingback: Deleporter: Cross-Process Code Injection for ASP.NET « Steve Sanderson’s blog

  21. Jesus Garza

    Steve, I am ashamed to say this is my first time on your blog. You have great content here.
    Are you using SpecFlow with your MvcIntegrationTestFramework (good work, btw)? I am going to try it, but it would be nice to have a heads up about any issues.
    Thanks.

  22. Newt

    Bellawarrogant

  23. Steve,

    there is a typo in Global.asax (http://github.com/SteveSanderson/GuestbookDemo/blob/master/Guestbook/Global.asax.cs)

    The default controller name for the default route is still “Home”. It should be “Guestbook”, I presume? That fails the test if you download the code.

    Otherwise I am still happy that you put this out there. I miss some good real-life examples on how to write great scenarios for a real app in all that’s written for BDD. This is a start.

    Thank you!

  24. Steve,

    The issue that tools like Cucumber resolve by having a text based file to define the tests is allowing seperation of fixture definition from implementation. This becomes important where whoever plays the Customer role wants to define scenarios to use as acceptance criteria prior to development. Typically the goal would be to use these scenarios in planning and as a guide to development from the outside-in.

    In this model however you are not running the UI through these tests but the business rules. UI tests have value but can be expensive to maintain, whereas business rule derived tests are cheaper to maintain. It also seperates ‘what’ from ‘how’ clearly. The storytest fixture captures ‘what’ and ‘how’ both code and UI is an implementation detail.

  25. Steve

    Ian,

    Like most methodologies, it depends on the nature of your project. If your product owners want to give specifications in terms of UI interactions (perhaps because your software focuses on complex user interactions – implemented as a mixture of server-side code, JavaScript, CSS, browser behaviours, etc – which is the case for the project I’m mainly thinking of) then naturally you’ll write specifications at that level. For us, the UI wasn’t just an implementation detail! But if your software focuses on complex business rules, then I can see why you’d prefer to write specifications directly about those rules instead.

    We found Cucumber’s extra layer of abstraction (natural language on top of code) to be advantageous for talking about UI interactions because it does transcend all those technologies. But we also found the extra abstraction to be positively disadvantageous when specifying lower-level logic, because it’s an extra thing to maintain as compared to plain NUnit/MSpec style specifications about classes and methods, which consumes a lot of extra time in the long run and wasn’t interesting to our product owners anyway.

    Thanks for your comment – it’s helped me to clarify some thoughts further.

  26. Hi Steve,

    Thanks for writing this blog entry. It has helped me in a number of ways – particularly in how to implement the ‘Step’ in a specFlow example.

    Thanks

  27. I must say that your current web log is extremely topical. I’ve been investing quite a lot of time for the last last couple months on the lookout at what is on the market influenced by the very fact that I’m arranging to launch a weblog. The information you have put on here is essentially to the point. It just would seem to be so difficult related to all the modern advances that are around, but I just like the way your appears to be. Gotta take pleasure in where technologies has come through the previous 11 years.

  28. Pingback: Getting Started with SpecFlow in ASP.NET | Bryant Rethinks Software

  29. Pingback: Technology Puneri Style » Behaviour Driven Development (BDD) & tools for .Net

  30. Nick

    Does anyone have good examples of how to organise features and step files with specflow? For example, with a typical crud scenario, how do lay it out in spec flow files? do you have one feature and one step file that enables you to reuse controllers and other utilities, or do you put them in separate step files so you run through the steps of a scenario quite easily?

  31. Pingback: Behavior Driven Development (BDD) with Cucumber and ASP.NET MVC

  32. Kevin

    Hi everyone,

    three things I wanted to point out about SpecFlow (which let me say, I have been trying yesterday for the first time seriously, and there a couple of things that are really neat… some others I am still not convinced about):

    * The extreme verbosity in how SpecFlow reports the outcome of the tests, either in the NUnit console or in the Output window console when using TestDriven.NET. Sometimes it becomes a bit difficult to realize what the problem is when a test fails.

    * The way SpecFlow suggests to group the specs for a class.. all in one single “Step” file. I believe that the Step file can grow very quickly, and it may become a bit difficult to detect which test failed or where.
    I think I prefer the way MSpec suggests to organize the files: one file per scenario, so you would have when_user_edits_profile.cs, when_user_logs_in.cs …etc

    * The text that needs to be written in the specs it is also quite verbose. Usually, in a “Then” I had to write no less than 3 lines just to do one Assert. I don’t quite like the variables being sent from here to there within the collection in the Scenario.Context.
    Maybe there are some helpers out there somewhere that could alleviate that a bit.. although I don’t think it is possible, because of the way SpecFlow is built, you always need to store/recover the variables from the ScenarioContext.

    Look at a very simple When and Then I wrote, how much text they contain… am I doing it the right way?
    (disregard the ‘technical’ language used in my specs rather than being more user-driven specs… I was just trying at the smallest unit level, just to give it a try)

    [When(@"I create an apartment")]
    public void WhenICreateAnApartment()
    {
    Address address = ScenarioContext.Current.Get();
    int floor = (int)ScenarioContext.Current["floor"];
    string apt = (string)ScenarioContext.Current["apt"];

    Apartment apartment = new Apartment(address, floor, apt);
    ScenarioContext.Current.Set(apartment);
    }

    [Then(@"it should set the address")]
    public void ThenItShouldSetTheAddress()
    {
    Address address = ScenarioContext.Current.Get();
    Apartment apartment = ScenarioContext.Current.Get();

    address.ShouldEqual(apartment.Address);
    }

    * Lastly (I promise), is there some helpers to handle exceptions more elegantly? I know that since SpecFlow is for more user-oriented specifications one shouldn’t talk about exceptions, right?
    But what if we want to test lower level functionality (ie: unit tests)? Or do you suggest not to use SpecFlow for that purpose?
    In unit level tests, you may have to write some expectations for Exceptions for sure.
    These are 2 very simple helpers I wrote, but maybe there is a better way:

    public static class Expect
    {
    public static void Exception(Action throwingAction)
    {
    Exception(“Exception”, throwingAction);
    }

    public static void Exception(string id, Action throwingAction)
    {
    try
    {
    throwingAction();
    }
    catch (Exception ex)
    {
    ScenarioContext.Current.Set(ex, id);
    }
    }
    }

    public static class Catch
    {
    public static Exception Exception()
    {
    Exception ex;

    ScenarioContext.Current.TryGetValue(“Exception”, out ex);

    if (ex != null)
    {
    return ScenarioContext.Current.Get(“Exception”) as Exception;
    }
    else
    {
    return null;
    }
    }
    }

  33. I’ve written about how to use SpecFlow in Visual Studio 2010 Express, so you don’t need a premium license: http://watirmelon.com/2011/02/18/c-sharp-atdd-on-a-shoestring/

  34. Pingback: Development trends « webINTELLECT

  35. Rob Whitener

    Hello, and thank you for the great post. I have recently come over to the .NET world from being mainly a Java developer but a recent convert to Ruby. I have worked on a few Ruby on Rails projects with Cucumber so I was delighted to find SpecFlow, which I feel is a much better solution to a Gherkin style BDD testing in .NET than a wire protocol or IronRuby. I notice that this article uses WatiN as the verification tool for HTML/views. What I am wondering is if there are any tools out there similar to webrat or capybara (from Rails testing) that are headless browsers with javascript support?

    Thanks,

    Rob

  36. Alena

    Thanks for your post, i also using SpecFlow – it’s very easy tools for our project, we write our tests on russian language. Any people can understand which part of system is testing and done. I work with SpecFlow for 3 month and count of tests become 3190 from 1000 (these 1000 “scenario” we wrote from May 2010). If you want to cover the system tests at 100%, you must use SpecFlow

  37. Pingback: My talk on WatiN , SpecFlow, BDD, Code and additional material | Taswar Bhatti Blog

  38. Pingback: Week 2 – The first feature | My Care Tracker

  39. Pingback: BDD with ASP.NET MVC using SpecFlow and MSpec | Coding Answers

  40. Phillis Lilla

    One thing I’d really like to discuss is that weightloss system fast may be possible by the suitable diet and exercise. People’s size not simply affects appearance, but also the complete quality of life. Self-esteem, depressive disorders, health risks, plus physical ability are affected in fat gain. It is possible to just make everything right and still gain. If this happens, a problem may be the perpetrator. While too much food and never enough work out are usually accountable, common health conditions and trusted prescriptions might greatly add to size. Thanks for your post in this article gywl512.

  41. ElMohanned Ahmed

    Hi steve,
    I like your post and I think you have actually pointed out the right tools for .Net: Watin and SpecFlow with MVC.
    However, I do not think you can limit business acceptance testing to testing over the UI. There are many arguments against UI testing. It is lengthy to write, very brittle since UI changes alot and very slow to run the whole suite which delayed feedback. In my experience, on a relatively mid size project, we spent lot’s of time rewriting UI tests and the suite took around 2 hours to run (opening browsers, sending requests, etc).
    It is better to write the majority of your business acceptance tests on the controller level with some sanity tests covering the UI.
    May I refer you to Uncle bob’s post: http://tech.groups.yahoo.com/group/fitnesse/message/428
    I also like his tweet:
    Testing through the GUI (e.g. Watir, Selenium, Cucumber/Webrat) has it’s place; but you shouldn’t test business rules that way
    Also Gojko Adzic beautifully explains it:
    The Sine of Death by UI Test Automation
    http://gojko.net/2010/07/29/the-sine-of-death-by-ui-test-automation/
    I actually say my personal experience followed this curve he plotted except it was watin bnot selenium:)
    Naresh Jain talks about the inverted pyramid:
    http://blogs.agilefaqs.com/2011/02/01/inverting-the-testing-pyramid/
    Thank you again for a great post…

  42. Pingback: Everything You Need to Get Started with SpecFlow and WatiN : Steve Smith's Blog

  43. Kevin,

    I’ve been using SpecFlow for a little while (I’ve written a number of features/steps/etc) and I think you can make your life easier by doing a few things:

    1) Don’t think about organizing steps as 1:1 with feature files. SpecFlow looks at any class with the attribute [Binding] and matches against any methods it finds… meaning you can spread out common steps into dedicated step files to keep things clean (or even create Given/When/Then sub-folders, etc. etc.)

    2) I’ve never had to use ScenarioContext like you are doing. It’s been a year since your post, maybe that was the old way, but my equivalent steps for your code would be like this:

    private Address _address;
    private int _floor;
    private string _apt;
    private Apartment _apartment;

    [When(@"I create an apartment")]
    public void WhenICreateAnApartment()
    {
    // The other private vars should have been set in a Given statement
    _apartment = new Apartment(_address, _floor, _apt);
    }

    [Then(@"it should set the address")]
    public void ThenItShouldSetTheAddress()
    {
    _address.ShouldEqual(_apartment.Address);
    }

    Since your step class is instantiated by SpecFlow, you can simply pass in common data using private variables or if you need more advanced features, through a constructor injection (see SpecFlow’s documentation).

    Notice how much more streamlined and easy to understand it is now? You could add some Asserts in there to verify stuff but I usually don’t as I should be asserting most things in the Then statements.

  44. Rishi

    HI,
    This discussion is really very helpful.

    My team is new to BDD and we are just moved to SpecFlow and CodedUI to test our desktop application developed in C#.

    Can anyone please provide some concrete examples, where in which real time classes/functions are being tested? I am very new to this and not able to find any specific material which coulde help me to learn and practise these tools.

    Please help me.

    Thanks in advance.

  45. Obaid

    Hi Steve,

    I see you have given below example where you have defined two behaviors within a scenario. Shouldn’t the scenarios be separated out based on two different behaviors.

    I am relatively new to BDD and trying to understand the principles of Given, When and Then usage.

    Scenario: Navigation to posting page
    Given I am on the guestbook page
    Then I should see a button labelled “Post a New Entry”
    When I click the button labelled “Post a New Entry”
    Then I am on the posting page

    Thanks

  46. WhoPutTheDingDongInTheBellware

    Why can’t I go to a BDD article without this guy with the Napolean Complex popping in and condescending to everyone? At first it was slightly amusing. However, I meet people like this IRL and they are typically the biggest drains on projects, yet somehow they co-opt management who fear they really DO have all the answers. It makes for a dreadful experience. I appreciate the true enlightenment so many post in response to this guy. Too bad he is stuck in such a dark place, believing he is ‘the one’. Keep up the good fight everyone.

  47. Jay Greasley

    Hi Steve,

    Very balanced and interesting post.
    I wonder if you know, given your experience of Rails and Asp.net MVC, whether there is a .Net equivalent for Rails + Factory Girl + rspec?

    TIA

    Jay

  48. Gleb

    Hello Steve,
    thanks for many interesting posts including this one.
    I have a question – there seems to be at least 2 approaches for writing specs in SpecFlow for ASP.NET MVC apps: the one you showed in this post and this one http://bit.ly/sE3nK5, instantiating controllers inside steps and using them to navigate. Which approach do you think is better for which situations?

  49. I’m getting started in SpecFlow and I am pretty confident it will give us what we need, especially given that our ace product owner speaks English, not C# and he will be able to understand the tests.

    Here are the two aspects that are harder to decide on.

    1) WatiR/WatiN or Selenium? I have some experience with Selenium and it isn’t 100% brilliant, but is it any better with WatiR/WatiN?

    2) How do you determine how much UI test to perform over API level / unit tests? I know I have 100% coverage on the business rules, but it is hard to proove the UI is behaving without testing the same things again, so do I just accept that even though I’m not testing the business rule, a change in the business rule may affect the UI tests?

    I’d love suggestions on these two points from anyone!

  50. Kristian

    I don’t mean to be gravedigging – but I do seem to be in a similar situation as Steve Fenton and Rob Whitener.
    What would be “the right way” to go in regards to browser automation for this (in .NET)
    Currently considering WatiN and Selenium. Any thoughts or links to (dis/)advantages would be highly appreciated.
    Also, great article! Gave me both insight and overview, as I am new to the BDD world and just getting the grips now.

  51. My understanding about BDD is that it is aimed at solving the language barrier between the developers and the business people. If we do a reality check, the BDD features and scenarios are written by developers. Is it good? Not really.

    The next struggle is using imperative vs declarative style in scenarios. Follow declarative is what bdd gurus ask us to do. The declarative style will remove all the details from the scenarios. This will not help the UI/UX developers who look for minute details of how a UI should look like.

    Steve’s approach of doing BDD with specflow and asp.net mvc is more developer directed bdd.

    Another thing I want to point out is that GIVEN is to describe the given context or pre-condition. It is not intended to define an action. The step ‘Given I am on the guestbook page’ is an action from a user. Rather you could say ‘When I am on the guesbook page having guestbook entries’.