2013-10-31

The kinds of tests we write

When starting work on version v3 of our software PriceOn, we decided to take unit testing seriously. This was basically our CTO's idea, and I am very happy that we managed to adopt it as a default coding practice in virtually no time. The reason it went so smoothly for us is just that all three of us backend developers were highly motivated to master it, and we had some previous experiences trying it and failing when not taking things seriously.

What we deliver is a service API reachable via HTTP. This API is consumed by our frontend clients: website and mobile apps for iOS and Android. We code in Microsoft's C# - some would say non typical choice for a startup, but we decided to stick with the thing we know best, as there are far more important challenges than picking 'cool' language. So let me just explain some points related to how/why we perform testing.

Simple integration tests that ping service via HTTP

These primarily test that the  wiring is correct and service does not crash on typical request. Just a simple service ping, which checks that server returns 200 OK, no business logic testing here.


[Theory, NinjectData]
public void FollowCardApidPing(ICanSave<Card> repo, FollowCardReq req, Card card)
{
    req.UserId = MockedUserId;
    repo.Save(card.CardId, card);
    AssertPostTakesAndReturns<FollowCardReq, FollowCardRes>(UrlRegistry.FollowCard.Mk(new { userId = MockedUserId, cardId = card.CardId }), req);
} 
Some details:

  •  we use custom NinjectData attribute which resolves test parameters by first looking at Ninject kernel specifically configured for tests, and if that fails, creating something with AutoFixture.
  • AssertPostTakesAndReturns is a method of base class integration tests derive from. This class Launches in-process instance of ServiceStack which hosts our services, so that we can interact with them via http, and that is what this Assert method does.
  • Currently, this is integration test just in a sense that it launches http server and tests everything via http interface. All problematic dependencies within service are replaced with in-memory implementations. We may consider changing them to real ones sometime in the future when reliable performance of 3rd party software starts to weigh in.


High-level unit tests that fake only external dependencies

These are the majority of tests we write. We test service logic by trying to mock out as little dependencies as possible. Similar to Vertical Slice Testing by @serialseb. The unit tests get System Under Test (SUT) in fully wired up state, with only external dependencies such as Database, Timer, Filesystem faked. As in production, unit tests resolve dependencies from IOC container which is just slightly tweaked from production configuration to inject lightweight implementations for external services.
The most prevalent external dependency is a database. How do we deal with it? We have IRepository interface, and then we have InMemoryRepository : IRepository. For each test, we seed this InMemoryRepository with relevant data, and inject into SUT.


[Theory, NinjectData(With.NullUser)]
public void QueryingShops_SingleSellerExists_ReturnsThatSeller(ProductAdminService sut, ShopsReq req)
{
    const string seller = "Mercadona";
    var repo = SetupData(new[] {Product.CreateGoodProduct().WithSeller(seller)}, p => p.Id);
    sut.ProductRepo = repo;   
   
    var res = sut.Get(req);
   
    res.Items.Select(u=> u.Seller).Should().BeEquivalentTo(new[]{seller});
}


  • As with integration tests, we resolve our dependencies from Ninject and AutoFixture, with just external dependencies faked. In this case, sut is taken from Ninject, and req is some randomly generated request made by AutoFixture. We may have to tune req according to test case, but in this case it is empty object, so nothing to be done there.
  • Our InMemoryRepository with data for the test is injected into the SUT. This is more stable than faking response from repository directly.
  • Repository is injected into sut via property setter. As we are resolving sut from IOC container, we already have default repository implementation, but we have to swap it with the one containing our predefined data.

"Real" unit tests in specific places with nontrivial business logic

We write these just in the places we feel logic is not trivial and may tend to change. This usually happens when we have to evolve API method to support more interesting scenarios, and it practically boils down to extracting domain specific code into isolated easily testable units. For easiest testability, it is very nice to isolate complex logic from all dependencies.

[Fact]
public void RegularPrice_IsPriceThatRepeatsMostDays()
{
    var date = new DateTime(2000, 1, 1);

    var sut = new Offer();
    sut.Prices.Add(date, new PriceMark(){Price = 1, ValidFrom = date, ValidTo = date.AddDays(10)});
    sut.Prices.Add(date.AddDays(11), new PriceMark() { Price = 2, ValidFrom = date.AddDays(11), ValidTo = date.AddDays(13) });
    sut.Prices.Add(date.AddDays(14), new PriceMark() { Price = 2, ValidFrom = date.AddDays(14), ValidTo = date.AddDays(16) });

sut.RegularPrice.ShouldBeEquivalentTo(1);
}


  • This is fairly obvious code, our SUT does not require any dependencies, just plain business logic. Breeze to test.