Just about every mature testing framework out there supports the concept of custom matchers, which provide us with the ability to define our very own assertions seamlessly into the tests. Even though this ability is quite old, we don’t see it used too often and I think that’s a shame. I’ve seen this practice heavily used in the mind expanding GOOS book and just now am starting to realize its awesomeness.

Your Testing Toolbox

Matchers 101

Creating your own matcher usually means creating a Matcher class that performs the assertions, supplies human readable error messages and a nice constructor.

Here’s an example from the RSpec documentation:

RSpec::Matchers.define :be_a_multiple_of do |expected|
  match do |actual|
    actual % expected == 0
  end
end

Matchers increase readability and intent

As you should know, one of the most important rules for design is Reveals Intent. Take a quick look here, which way do you think reveals more intent?

# This
response['X-Runtime'].should =~ /[\d\.]+/

# .. or this?
response['X-Runtime'].should be_a_number

Also, which error message do you prefer? “expected false to be true” or something along the lines of “expected comment to be anonymous”?

Matchers create robust tests

The most important advantage of all is how using matchers easily allows you to steer away from fragile tests which are the bane of a lot of testing efforts.
The mark of good tests is that a change in your code doesn’t require you to perform changes in multiple tests that don’t really care for the change.
Take this code for example:

expected_comment = Comment.new(anonymous: true, user: "the dude", reply_to: nil)
commentor.should_receive(:add).with(expected_comment)

This might seem like a standard test, but that’s not really the case. A test should assert for a single piece of knowledge, and this test actually checks several. If the purpose of this test is to check the behavior of anonymous comments, why should it change if we no longer allow replies? Or if we no longer require users for posting comments?

The magic of matchers is exactly here. You create a new matcher to check specifically the aspect your test cares about and *boom*, you’re decoupled!

commentor.should_receive(:add).with(anonymous_comment)

This simple change makes your tests DRY and cool.

Happy testing!

Your should subscribe to my feed or follow me on twitter!

I’ve seen a lot of posts about testing in startups, some saying startups shouldn’t bother to test because they’ll have to change the whole damn thing 5 minutes after they’re done, others claim testing is the only reason they were able to keep working. Here are some of my thoughts looking back.

Our Background

When we started, only two of us had a test-infected background out of the five technical guys, me being big on TDD. Two other developers never wrote tests before. We agreed that tests were important, but that’s about it. I set up a continuous integration server and with that we were off. With time, the habit of writing tests spread out among the team. Some are TDD passionate, some write tests after the fact, but we generally all believe that tests should be written extensively.

None of us, ever

Not everything is worth testing

We’ve seen several quite rapid changes to our UI. Having less tests in this area makes sense. We rely on QA for making sure all buttons are displayed etc. To make this clear: we have no selenium-like tests for UI components but have tests for most logic being done by the UI. I think this is generally a good practice, since having to maintain selenium tests would be hard when you throw things around a lot and change flows. Some basic automated sanity tests pretty much does it.

Everyone learned to love tests

I love seeing other guys in the team delete a line of code to see which test breaks and understand why it’s there. Even more I love the frowning face when no tests break. This addiction to tests shows how much value the team’s getting out of having solid tests, hands down. No need to stress this further I believe.

Tests save our asses repeatedly

Having an extensive suite of tests allows us to make rapid changes to our code base, as is needed in most startups, and rely on the solid tests to tell us whether we’ve screwed something up. All the code that has anything whatsoever to do with sensitive and important information is heavily tested which is a huge bonus and a necessity in our line of business (personal finance protection).

TDD is just magical with complex algorithms

We have quite a few complex algorithms that require multiple entities and ideas to perform. I find that the parts we’re most satisfied with maintainability-wise are the heavily TDD-ed algorithms we’ve got. Being written with rigorous TDD gives us so many advantages:

• This critical code usually has a lot less defects.
• The code is a lot more readable, well decomposed and allows for easy changing once we find out a need for tweaking the algorithms.
• Working in TDD magically forces us to form our problem domain better, making us have a language of our own in talking about the problem. This happens less naturally in other forms of working on algorithms.

Summing our testing experiences

All in all, I think the whole team would agree that dedicating time to writing thorough tests is proving itself valuable and because of that people are writing more and more tests without any of us ever stopping and saying “we should write tests” (well, I swear I didn’t do it too much). It happens naturally when people get the value out of it. It’s fun seeing how today BillGuard has become a company that organically values testing so much I don’t even feel a great need to stress it to new people because they’ll quickly see there’s no real other way. We’re far from being the poster children of Clean Code, but I’ve got my fingers crossed.

If you’re interested in accomplishing the same at your work, you might find this recent post of mine of some help.

You should subscribe to my feed or follow me on twitter!

It might be small, like unused imports, instance variables and methods. It can be whole classes that make up entire features no longer used.

The biggest PITA is when you’re not really aware of the fact the code’s dead and unused, sitting there and occupying precious bits, and stumble across it as part of a task, trying to understand how it influences what you want to do next.

Whenever I recognized something that looks like dead code but I’m not entirely sure, I find it pretty easy to delete it quickly, once I take a look in the version control logs and see when it became no longer in use and why.

Zombie code is code that was never alive, and so couldn’t really become dead. It’s the undead code – code that died right when it was committed. Code that never ran or never worked. The are two reasons I hate zombie code more than “plain” dead code.

The first reason is that it simply wastes more of my time. Looking back in version control won’t help me see the commit in which the code was “decommissioned”, it would just appear to always sit there. That means I have to take extra care to verify that it, in fact, never worked.

The second reason is that it’s plainly someone saying he doesn’t give a damn. I mean, let’s put aside TDD. Heck, let’s put aside unit testing at all. It means the code never even ran the damn thing and saw in his own eyes it did what he claims it did.

Be kind to your teammates. If you’re not a good enough coder to test it, at least see that it runs once in your own eyes.

You should subscribe to my feed and follow me on twitter!

Today I set out with my pair to add some functionality to a certain class. That class had about 30-40 lines of code and about 10 test cases, which seemed quite decent. We added our changes TDD style and just couldn’t get the thing working. After digging into it for a few more minutes we suddenly realized the class shouldn’t be working at all and checking in the DB showed that indeed the last time that specific feature had any effect was 3 months ago!

Fortunately for us, all the problems that caused this bug are solved problems, we just need to get better at implementing the solutions:

Isolated tests go much better hand in hand with a few integration tests (some might say the right term is acceptance tests) that execute the whole system and make sure the features are working. Had we had those, we would have caught the bug much sooner.

The bug was introduced in a huge commit that changes 35 files and 1500 lines of code. We usually try and go over every commit made, even if it was paired, because we believe in collective code ownership, but it’s impossible to go over such a huge diff and find these intricacies. Working in small baby steps makes it far less likely to break something and more likely that someone else will spot your mistakes. Huge refactorings give me the creeps.

After the change was committed, it was not followed-through: this specific feature is a feature you usually notice over a few days and we missed out on making sure it kept working. We moved on to other tasks and forgot all about it, thinking it was working all this time. Had we taken the time to make sure we were seeing, it would have been squashed by the next deployment.

Any of these would have helped us spot sooner that the isolated tests were actually testing the code against a scenario that never happens. These tiny changes of our workflow would have made several of our users happier over this timeframe.

Hopefully all is well now and the feature is back at 100%, but only time will tell whether we were able to learn from this mishap.

You should subscribe to my feed and follow me on twitter!

After attending a workshop of his about a year ago and listening to him talk a whole day I was astonished. I tried to pick his brains to understand what makes him tick. Of course there are many factors here – reading over 10,000 books and being smarter than most would help anyone. But something a bit less common takes a major part in this in my opinion, what Kent told me he has: a “habit of desperately wanting things to make sense” and his ability to take things apart until they do.

I recently picked up another of Kent’s books, Implementation Patterns. I love this book because it shows exactly that: his process of thinking and breaking things apart in order to understand them. The book provides a rare glimpse to his method of decomposition. Since I’ve been coding for years, a lot of the patterns made sense to me or seemed trivial. But the “magic” is the fact he was able to put into words things that for me were just hunches. Actually explaining what makes you sense a method is too long or what is a proper name for a variable is something I’ve never seen done with such care to specifics.

Because it’s such a quick read, I think anyone will benefit by reading Implementation Patterns. More than helping you understand our craft better, it will provide a new outlook on decomposing and judging your designs and pretty much everything else.

You should subscribe to my feed and follow me on twitter!

A simple example might be handing out jobs to a few workers. If your algorithm for doing that is random, you can usually assert that no one of 3 workers gets all 10 jobs, for example. But, being random, that assert should eventually fail. We’ll assume that with the frequency the team runs the tests, a failure is expected every few days.

Surely no one wants to see the tests fail a couple times a week (especially if you’re keeping score for who broke the build). On the other hand, you’d like to keep the tests. What is a pragmatic coder to do?

If you’re not that meticulous to your suite rarely failing, you might just leave it as it is, which, I think, sucks.

The mega-tester’s approach, which I’ve tried in the past, is usually to stub out the random number generator with values that make sure the failures won’t happen. This is usually cost-effective only for the simplest of cases, and the more complex ones results in brittle tests that are coupled to the implementation and that might need to be changed frequently.

What I rather is to postpone the problem! Say we change our test’s parameters to 10 workers and 3000 jobs. The chances of one worker getting all jobs becomes quite minor. This tweak of parameters in the test is usually simple to do and can guarantee quite a safety net.

And still, sometimes bad stuff happen. 64bit hash collisions are somewhere, out there in the world. If you’re one of those guys that are bugged by that chance, I give you a simple JUnit rule that will retry a specific test in case it fails, making it twice as unlikely to fail. Those 64bit collisions are now more like 128bit! woohoo!

The rule allows you to simply annotate a test to make it retry in case it fails:

public class RetrierTest {
  private static int count = 0;

  @Rule public RetryRule rule = new RetryRule();

  @Test
  @Retry
  public void failsFirst() throws Exception {
    count++;
    assertEquals(2, count);
  }
}

And the implementation is as simple as:

@Retention(RetentionPolicy.RUNTIME)
public @interface Retry {}

public class RetryRule implements MethodRule {
  @Override public Statement apply(final Statement base, final FrameworkMethod method, Object target) {
    return new Statement() {
      @Override public void evaluate() throws Throwable {
        try {
          base.evaluate();
        } catch (Throwable t) {
          Retry retry = method.getAnnotation(Retry.class);
          if (retry != null) {
            base.evaluate();
          } else {
            throw t;
          }
        }
      }
    };
  }
}

With the tests so unlikely to fail, I’d start a lottery at work for whoever breaks them.

Happy testing!

You should subscribe to my feed and follow me on twitter!

The Dilemma

Nowadays, it’s hard to get more than a 100 lines of code before adding an external resource to our code. It might be a web service to manage something, or some convoluted API to receive data from or just about anything. Usually, writing tests for code that directly talks with these resources using the resources themselves is very problematic, for numerous reasons:

• It significantly slows the tests, because it requires network access and processing on the service’s side.
• It might cost you money, send emails, tweet stuff and do things you’d rather not do 300 times a day as you run your tests.
• Making your code handle error conditions with the service is hard or impossible, as you can’t control when those occur.

Basically, all of these factors usually amount up to you having crappy tests that you rarely run. That sucks.

Decouple & Isolate

The best solution I’m aware of is simply isolating the thing. We usually strive to wrap whatever service we’re using with a single-point interface. The decoupling is great since I’ve yet to encounter a service with an API that matched my thinking of the domain problem. Wrapping it up allows us to keep using our own language and logic throughout the system.

A benefit of that is we now have a simple interface or facade we need to stub/mock out during tests. That’s usually relatively easy, and allows us to run our tests blazingly fast and test all those hard to reach to corner cases.

But what if the service changes?

That’s the finishing touch. You should still maintain a suite of tests that run against the real service. Those should be the plain tests that make sure you’re using the API right and that would break if anything you’re relying on changes. These tests won’t be part of your regular suite that gets run constantly. Instead have your CI server run them daily/weekly and let you know when something changes.

This puts us basically in a win-win situation, with us being able to run our tests quickly and yet have the assurance that we won’t miss API changes and the likes.

Happy testing!

You should subscribe to my feed and follow me on twitter!

Starting with a test means that we have to describe what we want to achieve before we consider how.

Here’s a snippet from the official GWT MVP tutorial:

In this example, you see that our Presenter, when bound, registers a click handler for a button, in order to perform some action when it is called. This might seem nice and all, but there’s a smell. This is a violation of the Law of Demeter (the missing SOLID rule, one might say). This simply makes it harder to test, since we now have to add another layer of indirection between the SUT and its collaborators. Instead of making the view a tiny bit smarter, we use it as a dumb collection of widgets the presenter manages. This is clearly not in “Tell, don’t ask” form.

The thing that really bothers me is how coupled the presenter gets with its view. Take the above example, and say that you decided that it would be better to have two “save” buttons on the UI. Does the presenter really care? Should it even change? And what if you actually want the save button to change to a remove button when the user picked something? Should the presenter now deal with getSaveOrRemoveButton() ? Of course not.

GOOS it up

After beating around this bush for quite some time, I decided to try and find a better way. I’m currently reading the brilliant Growing Object Oriented Software book, and decided to try its approach to push a better implementation. After a bit of refactoring I got this:

This might seem like a tiny change. And it is. But it makes all the difference in the world in how more responsive your design gets, especially in our world where the view is most likely to change a dozen times before settling on something. Once there are enough of these, I push the presenter as a dependency into the view, and let it call the presenter directly. The funny thing is this style is actually implicitly mentioned in the second part of the GWT MVP tutorial. Just some GOOSing helped us get to a better, more malleable design!

Don’t be afraid to do something differently than the documentation, especially if you gave it a fair shot and it didn’t work out.

You should subscribe to my feed and follow me on twitter!