I’m going to take a short diversion from my series on BDD with ASP.NET MVC to talk about the difference between Features and Specs in Behaviour-Driven Development.
I am currently reading the beta version of The RSpec Book. This is an excellent book that covers doing BDD in Ruby using RSpec and Cucumber. From reading this book, I have come to realise there is a distinct difference between higher-level user acceptance criteria (features) and and lower-level context-specifications (specs).
Features
Features are the things that the customer is likely to be most concerned with. They are high-level behaviours that are derived from the acceptance criteria of a user story. They are purely concerned with how the user interacts with the application and don’t provide any details of implementation.
Features can be used to facilitate communication with the customer on what business objectives the application should achieve.
A feature consists of a user story and one or more example scenarios, which serve as our acceptance criteria. An example of a feature would be:
Story: Account Holder withdraws cash
As an Account Holder
I want to withdraw cash from an ATM
So that I can get money when the bank is closed
Scenario 1: Account has sufficient funds
Given the account balance is $100
And the card is valid
And the machine contains enough money
When the Account Holder requests $20
Then the ATM should dispense $20
And the account balance should be $80
And the card should be returned
Features can be turned into automated acceptance criteria. In Ruby, we can use Cucumber to parse a plain-text file and run each scenario against the application. These are usually slow-running, end-to-end integration tests. In .NET we would use a BDD framework such as NBehave, or a standard unit-testing framework to implement the features as automated acceptance criteria.
Recently, I have been using Cucumber for automating the acceptance criteria for a .NET application, using Waitr to drive the browser. With IronRuby, we will be able to access our .NET objects directly from Cucumber. As of this post date, IronRuby does not work with Cucumber, but a fix should be out very soon.
Specs
When we have defined a new feature, we must create the objects required to implement it. To do this we need to “drop-down” to the object-level and use a context-specification style to define how these objects should behave.
Specs are implemented as fast-running, unit-level tests with stubbed dependencies. In Ruby, RSpec is used to define the specs. In .NET we would use a standard unit-testing tool, such as NUnit.
A spec consists of an action on an object, within a certain context, and the observations made from that action. An example of a spec would be:
When ATM requests withdrawal and account has sufficient funds
Should check account balance
Should send “Please wait” message to screen
Should begin transaction
Should withdraw funds from account
Should dispense cash
Should end transaction
Should return card
Should send “please take your card” message to screen
These steps describe the interaction between our object and its dependencies. They do not necessarily reflect the user’s observations (e.g the user doesn’t observe a transaction taking place).
We are not as concerned about testing the state of an object as we are about defining how it interacts with other objects. For instance, using the Model-View-Controller pattern, if we’re specifying a controller, we want to make sure an action sends a certain “message” to the view, rather than testing how it affects the state of the view. By specifying interaction over state, we are better able to define the roles and responsibilities of our objects.
I will try to incorporate some of this into the PlaylistShare sample application to demonstrate the distinction between Features and Specs.
Tim,
“By specifying interaction over state, we are better able to define the roles and responsibilities of our objects” – awesome.
WRT IronRuby – once it gets going and you test interaction with objects with cucumber, do you see any reason to maintain watir tests? Are all the browser based test cases that cover /state/ covered by the testing of object /interation/, do you think? E.g. does one sit on top of the other?
Very good points, Hemal.
Having IronRuby will mean that we can bypass the browser and interact directly with the surface-level objects (e.g. controllers). However, this won’t reach client-side code so we will still need browser automation to some degree.
Even though the fine-grained object-interaction tests should cover all our behaviour, I think it is still worthwhile having end-to-end automated acceptance tests that sits on top. When coding at the unit-level, it is comforting to know there is something looking at the bigger picture. However, as your application grows it might not be worthwhile maintaining older automated acceptance tests. So maybe a cull is needed every now and then to ensure the regression suite doesn’t grow too large and too slow.
Good post…
I think that making the distinction between user-level features and dev-level specs is a really good way of looking at it (and a sign I really should start reading the copy of the Rspec book I bought).
I think when I started looking at all this stuff, I lumped the features and specs in all together into one set of tests – which was confusing for the main reason that I would have vague ‘user language’ quickly dissolving to technical language (‘object was persisted’ or something) often within the same sentence.
If you can just keep the user stuff slightly separated but still related to the lower level, you keep the value without confusing the hell out of all parties concerned.
That sums it up perfectly, thanks Jen