Testing Times

I came across a test breakage recently. I had committed some code and a test had failed. The test was unrelated to my changes, but I took a look anyway because maybe I was wrong. As it turned out the test was unrelated to my changes other than the fact that my changes had triggered a build. So the test was intermittently failing.

In my experience there are three common causes for test intermittency.

• Interaction between test-cases – static methods, shared memory, stuff left on the file-system and so on. Basically all down to state shared between tests.
• Concurrency – It is extremely hard to write good tests that deal, within the scope of the test, with concurrency. It’s too easy for small changes in the timing of the execution of different threads to give different results.
• Time – time based tests are a pain, they are often flaky and slow because of the need to wait for results.

The last one of these was, I think, the cause of the problem with this test. Fortunately, of the three, time based testing is the simplest to fix.

There is one simple rule for dealing with time in your application: Never, Never, Never retrieve time directly from the system, always get it via a level of indirection, and then in your tests, fake the source of time so that you have complete control.

e.g.

    interface Clock {
        long getNanos();
    }

    class MyObjectThatNeedsTime {
        private final Clock clock;

        public MyClassThatNeedsTime(Clock clock)
        {
            this.clock = clock;
        }

        public void doSomethingNeedingTime()
        {
            ...
            // Instead of this...
            // time = System.getNanos();
            // Do this...
            long time = clock.getNanos();
            ...
        }
    }

Now your test has complete control of time, simply supply a fake implementation of Clock and change what it returns in the scope of your testing to do whatever you like.

This works for all kinds of testing, not just simple unit testing. I generally use this approach for all uses of time. At LMAX we had a suite of whole-system integration tests called “time-travel tests” where we manipulated time, skipping over minutes, hours even months to test daylight saving time changes.

If for some reason you don’t want to inject the Clock into your code, you can use a singleton pattern and replace the singleton instance within the scope of tests (always remembering to replace the TestClock with a RealClock at the end of the test). Not as nice as dependency injection perhaps but it can be easier to fit in when you are adding tests to some legacy code.

e.g.

    class SystemClock implements Clock {

        public long getNanos()
        {
            System.getNanos();
        }
    }

    class ClockFactory {
        private static Clock clock = new SystemClock();

        public Clock getClock()
        {
            return clock;
        }

        public void setClock(Clock newClock)
        {
            this.clock = newClock;
        }
    }

    ...

    public MyClassThatNeedsTime()
    {
        this.clock = ClockFactory.getClock();
    }

This approach not only gives you better control in your tests, but it also speeds them up – no more sleeping threads, which can add up in large test suites. As well as all that it enables classes of testing that were simply impossible before (e.g. long duration waits).

Try it, you’ll like it!