The kinds of tests we write

When starting work on version v3 of our software PriceOn, we decided to take unit testing seriously. This was basically our CTO's idea, and I am very happy that we managed to adopt it as a default coding practice in virtually no time. The reason it went so smoothly for us is just that all three of us backend developers were highly motivated to master it, and we had some previous experiences trying it and failing when not taking things seriously.

What we deliver is a service API reachable via HTTP. This API is consumed by our frontend clients: website and mobile apps for iOS and Android. We code in Microsoft's C# - some would say non typical choice for a startup, but we decided to stick with the thing we know best, as there are far more important challenges than picking 'cool' language. So let me just explain some points related to how/why we perform testing.

Simple integration tests that ping service via HTTP

These primarily test that the  wiring is correct and service does not crash on typical request. Just a simple service ping, which checks that server returns 200 OK, no business logic testing here.

[Theory, NinjectData]
public void FollowCardApidPing(ICanSave<Card> repo, FollowCardReq req, Card card)
{
    req.UserId = MockedUserId;
    repo.Save(card.CardId, card);
    AssertPostTakesAndReturns<FollowCardReq, FollowCardRes>(UrlRegistry.FollowCard.Mk(new { userId = MockedUserId, cardId = card.CardId }), req);
} 

Some details:

•  we use custom NinjectData attribute which resolves test parameters by first looking at Ninject kernel specifically configured for tests, and if that fails, creating something with AutoFixture.
• AssertPostTakesAndReturns is a method of base class integration tests derive from. This class Launches in-process instance of ServiceStack which hosts our services, so that we can interact with them via http, and that is what this Assert method does.
• Currently, this is integration test just in a sense that it launches http server and tests everything via http interface. All problematic dependencies within service are replaced with in-memory implementations. We may consider changing them to real ones sometime in the future when reliable performance of 3rd party software starts to weigh in.

High-level unit tests that fake only external dependencies

These are the majority of tests we write. We test service logic by trying to mock out as little dependencies as possible. Similar to Vertical Slice Testing by @serialseb. The unit tests get System Under Test (SUT) in fully wired up state, with only external dependencies such as Database, Timer, Filesystem faked. As in production, unit tests resolve dependencies from IOC container which is just slightly tweaked from production configuration to inject lightweight implementations for external services.
The most prevalent external dependency is a database. How do we deal with it? We have IRepository interface, and then we have InMemoryRepository : IRepository. For each test, we seed this InMemoryRepository with relevant data, and inject into SUT.

[Theory, NinjectData(With.NullUser)]
public void QueryingShops_SingleSellerExists_ReturnsThatSeller(ProductAdminService sut, ShopsReq req)
{
    const string seller = "Mercadona";
    var repo = SetupData(new[] {Product.CreateGoodProduct().WithSeller(seller)}, p => p.Id);
    sut.ProductRepo = repo;   
   
    var res = sut.Get(req);
   
    res.Items.Select(u=> u.Seller).Should().BeEquivalentTo(new[]{seller});
}

• As with integration tests, we resolve our dependencies from Ninject and AutoFixture, with just external dependencies faked. In this case, sut is taken from Ninject, and req is some randomly generated request made by AutoFixture. We may have to tune req according to test case, but in this case it is empty object, so nothing to be done there.
• Our InMemoryRepository with data for the test is injected into the SUT. This is more stable than faking response from repository directly.
• Repository is injected into sut via property setter. As we are resolving sut from IOC container, we already have default repository implementation, but we have to swap it with the one containing our predefined data.

"Real" unit tests in specific places with nontrivial business logic

We write these just in the places we feel logic is not trivial and may tend to change. This usually happens when we have to evolve API method to support more interesting scenarios, and it practically boils down to extracting domain specific code into isolated easily testable units. For easiest testability, it is very nice to isolate complex logic from all dependencies.

[Fact]
public void RegularPrice_IsPriceThatRepeatsMostDays()
{
    var date = new DateTime(2000, 1, 1);

    var sut = new Offer();
    sut.Prices.Add(date, new PriceMark(){Price = 1, ValidFrom = date, ValidTo = date.AddDays(10)});
    sut.Prices.Add(date.AddDays(11), new PriceMark() { Price = 2, ValidFrom = date.AddDays(11), ValidTo = date.AddDays(13) });
    sut.Prices.Add(date.AddDays(14), new PriceMark() { Price = 2, ValidFrom = date.AddDays(14), ValidTo = date.AddDays(16) });

sut.RegularPrice.ShouldBeEquivalentTo(1);
}

• This is fairly obvious code, our SUT does not require any dependencies, just plain business logic. Breeze to test.

User input validation

A very well understood concept in computer programming is that of 'validation'. In its basic form it's just a predicate which determines whether data conforms to some predetermined rules. Combined with a principle that problems should be identified as early as possible, this leads to the most common application: having a method call, we first validate input arguments before proceeding with any business logic (checking preconditions). If validation doesn't pass, we return some hints as to what went wrong. Sometimes the information we provide event suffices to painlessly fix the issue.

Now, validation is also very widespread in HCI, where computer validates user input. As with all thing human however, this gets a little bit more tricky. You see, as a user, I'm not interested in bureaucracy in the slightest. It annoys me when I'm told "your application form is invalid". I expect software to guide me as I try to express my intentions. And while validation errors are a form of guidance, they tend to occur too late and tend to suggest user failure. You'd usually want someone who is willing to provide as much help as possible instead of accepting/rejecting your application and referencing some article no. in "rules".

Anyway, looks like this exploration-based model will dominate the transaction-based model in HCI. And the  simple "form validation" is falling apart here. At least, it should be way more granular, and ideally there should be no "validation errors". You just want your computer usage to be a pleasant experience, without any error popups. It seems often to be much better to just ignore incorrect input, rather than scream loudly "I don't understand what you mean, fool". Quite the opposite to what we do with validation in our code for API.

CQRS and ReadModel

While CQRS seems to be very useful tool for dealing with complex domains, I kinda got burned by it more times than I would like to admit. I'm talking about traditional interpretation, CQRS+ES+DDD ala Greg Young. The biggest frustration I'm facing is how it claims that readmodel can be somewhat an afterthought, yet how it still remains the crucial part, and unless you take a good care of it, you gonna have a bad time.

CQRS lets you defer decisions, not avoid them. You may think that choosing the reporting database is exactly the kind of decision CQRS lets you avoid. Well, not entirely. It allows you to offload the decision to someone else who is responsible for reporting. But still it's gonna be you for small one-man projects. And you'd better damn know well your database, as there are tons of nasty surprises awaiting with these beasts.

It is completely unsuitable for CRUD scenarios. If there is no complex processing of data coming in, using CQRS is certainly overkill. If your commands map 1:1 to your events map 1:1 to your read model, it is just common sense then this is a nonsense. Whoever says there is no overhead in doing this and you'd better just do it everywhere has no clue or is lying.

Naive implementations of read model only take you so far. There are some people around boasting of how CQRS+ES allowed them to get rid of the database. So this is really neat point with ES, that your primary data model (Events) is very simple. You can put serialized events anywhere you want, no need to tie  yourself to any particular DB. On the reporting side however, this is entirely different story. Unless you are writing most primitive and thus probably useless app, just storing your (demormalized) data in files is not gonna work. Where is fulltext search? Where are those adhoc queries business always bugs you about? Or even, where is the list of users sorted by date of registration? Pretty tricky to implement when all you have is partitioning by id, eh?

So, while I believe this is the way we will be building software in the future, the fact remains, CQRS just lets you ask the right questions at the right time, but never be fooled that this makes the answers somehow less important.