Sunday, November 15, 2009

GivWenZen and FitLibrary Part 4

I did not see this topic going any further; however, I had a question about turning the test green and I think I should explain how to do this and why I do it the way I do.

Please see the other posts if this post has no context for you read part1, part2, and part3.

So let's start with the simple part, the how.  To turn a test step green or red your step method must return a boolean.


@DomainStep(“my business domain step”)
public boolean myStep() {
  return isTrue();

If the method returns true the step is turned green and if it returns false the step in turned red. (see the image from the previous post) Only the first column in the row is turned green/red.  In our given-when-then test this means the given, when or then column is turned red (and steps are turned green/red too).  Remember that we still have a table it is just not visible to the user.  From a technical standpoint a step written as ‘given the application is in some state’ is a two column table with ‘given’ in the first column and ‘the application is in some state’.  FitLibrary does the magic of turning our plain text into a table by looking for methods in our system under test and/or our fixture.

Now the more difficult part of the post.  Why do I only make the ‘then’ steps in the test turn green?  The simple answer is the ‘then’ step is the only ‘test’ of the actions the test has performed. So first let’s talk about the steps in a BDD test.   My reasons are influenced by and I will borrow heavily from Dan North and Cucumber’s Aslak Helles√ły in this section.  Please read there descriptions as they are much more knowledgeable that I am on this subject.

I am going to describe each step from two points of view.  First I will describe the step as part of a specification or acceptance criteria.  I think this is the most important because in BDD we are first describing the business value we are delivering to the customer.  Then I will describe the step from a test point of view (i.e. what should the methods do when executed as part of a test).

Given specification steps are meant to describe the starting state of the application in order for the actions in the current scenario (the 'when' steps) to be performed.  Given test steps are meant to put the application in to the state described so that the 'when' step can execute successfully (once the application is written correctly :) ).  I do not turn these steps green.  It is my assumption that this part of the application is working and covered in another spec/test.  If something goes wrong here then an exception should be thrown.  Of course, since all your test were passing when you started you should create a new spec that describes how and test that the application can be put into the desired state.

When specification steps are meant to describe an action that is valid or invalid based on the current application state as described in the given steps.  When test steps perform the described action.  If something goes drastically wrong here I like to throw an exception that describes where the problem is in the system.  Why would I turn this step green/red?  What does it mean?  Really it can only mean that the action was performed and no exception was thrown.  Maybe that is enough but usually it is not which leads us to the ‘then’ steps.  It should be impossible for this step to be green if any of the ‘then’ steps are red so I think it is best to not color this step and throw an exception on serious failures.

Then specification steps tell us what is expected after the action of the when steps occur.  Do I expect the application to be in a new state?  Do I expect the action to fail?  Do I expect a message to be sent?  Then test steps verify that thee expected outcomes are true.  These steps must be turned green or red because they are the tests.

The problem with turning the 'given' and 'when' steps green is it gives some false impression that the expected outcomes of the test are true.  It is very likely when you are adding new functionality or changing existing functionality in an existing system that the 'given' and 'when' steps will be green the day the specification is written.  And in the case of FitNesse it will say you have N number of passing assertions and in reality you have not asserted anything.

In conclusion I must ask you to please read Dan North’s articles on BDD and check out the Cucumber pages as they have some good stuff (even if you cannot use Cucumber which is why I wrote GivWenZen).  Please let me know what you think about this method of doing BDD with FitNesse/FitLibrary.

1 comment:

  1. Wow, I didn't realize you wrote GivWenZen, too! Smashing. I'm just about to delve into all of this since I'm now on a Java team (had seen Uncle Bob's references to GivWenZen before, but was on a .NET team).