The kinds of tests we write

When starting work on version v3 of our software PriceOn, we decided to take unit testing seriously. This was basically our CTO's idea, and I am very happy that we managed to adopt it as a default coding practice in virtually no time. The reason it went so smoothly for us is just that all three of us backend developers were highly motivated to master it, and we had some previous experiences trying it and failing when not taking things seriously.

What we deliver is a service API reachable via HTTP. This API is consumed by our frontend clients: website and mobile apps for iOS and Android. We code in Microsoft's C# - some would say non typical choice for a startup, but we decided to stick with the thing we know best, as there are far more important challenges than picking 'cool' language. So let me just explain some points related to how/why we perform testing.

Simple integration tests that ping service via HTTP

These primarily test that the  wiring is correct and service does not crash on typical request. Just a simple service ping, which checks that server returns 200 OK, no business logic testing here.

[Theory, NinjectData]
public void FollowCardApidPing(ICanSave<Card> repo, FollowCardReq req, Card card)
{
    req.UserId = MockedUserId;
    repo.Save(card.CardId, card);
    AssertPostTakesAndReturns<FollowCardReq, FollowCardRes>(UrlRegistry.FollowCard.Mk(new { userId = MockedUserId, cardId = card.CardId }), req);
} 

Some details:

•  we use custom NinjectData attribute which resolves test parameters by first looking at Ninject kernel specifically configured for tests, and if that fails, creating something with AutoFixture.
• AssertPostTakesAndReturns is a method of base class integration tests derive from. This class Launches in-process instance of ServiceStack which hosts our services, so that we can interact with them via http, and that is what this Assert method does.
• Currently, this is integration test just in a sense that it launches http server and tests everything via http interface. All problematic dependencies within service are replaced with in-memory implementations. We may consider changing them to real ones sometime in the future when reliable performance of 3rd party software starts to weigh in.

High-level unit tests that fake only external dependencies

These are the majority of tests we write. We test service logic by trying to mock out as little dependencies as possible. Similar to Vertical Slice Testing by @serialseb. The unit tests get System Under Test (SUT) in fully wired up state, with only external dependencies such as Database, Timer, Filesystem faked. As in production, unit tests resolve dependencies from IOC container which is just slightly tweaked from production configuration to inject lightweight implementations for external services.
The most prevalent external dependency is a database. How do we deal with it? We have IRepository interface, and then we have InMemoryRepository : IRepository. For each test, we seed this InMemoryRepository with relevant data, and inject into SUT.

[Theory, NinjectData(With.NullUser)]
public void QueryingShops_SingleSellerExists_ReturnsThatSeller(ProductAdminService sut, ShopsReq req)
{
    const string seller = "Mercadona";
    var repo = SetupData(new[] {Product.CreateGoodProduct().WithSeller(seller)}, p => p.Id);
    sut.ProductRepo = repo;   
   
    var res = sut.Get(req);
   
    res.Items.Select(u=> u.Seller).Should().BeEquivalentTo(new[]{seller});
}

• As with integration tests, we resolve our dependencies from Ninject and AutoFixture, with just external dependencies faked. In this case, sut is taken from Ninject, and req is some randomly generated request made by AutoFixture. We may have to tune req according to test case, but in this case it is empty object, so nothing to be done there.
• Our InMemoryRepository with data for the test is injected into the SUT. This is more stable than faking response from repository directly.
• Repository is injected into sut via property setter. As we are resolving sut from IOC container, we already have default repository implementation, but we have to swap it with the one containing our predefined data.

"Real" unit tests in specific places with nontrivial business logic

We write these just in the places we feel logic is not trivial and may tend to change. This usually happens when we have to evolve API method to support more interesting scenarios, and it practically boils down to extracting domain specific code into isolated easily testable units. For easiest testability, it is very nice to isolate complex logic from all dependencies.

[Fact]
public void RegularPrice_IsPriceThatRepeatsMostDays()
{
    var date = new DateTime(2000, 1, 1);

    var sut = new Offer();
    sut.Prices.Add(date, new PriceMark(){Price = 1, ValidFrom = date, ValidTo = date.AddDays(10)});
    sut.Prices.Add(date.AddDays(11), new PriceMark() { Price = 2, ValidFrom = date.AddDays(11), ValidTo = date.AddDays(13) });
    sut.Prices.Add(date.AddDays(14), new PriceMark() { Price = 2, ValidFrom = date.AddDays(14), ValidTo = date.AddDays(16) });

sut.RegularPrice.ShouldBeEquivalentTo(1);
}

• This is fairly obvious code, our SUT does not require any dependencies, just plain business logic. Breeze to test.

User input validation

A very well understood concept in computer programming is that of 'validation'. In its basic form it's just a predicate which determines whether data conforms to some predetermined rules. Combined with a principle that problems should be identified as early as possible, this leads to the most common application: having a method call, we first validate input arguments before proceeding with any business logic (checking preconditions). If validation doesn't pass, we return some hints as to what went wrong. Sometimes the information we provide event suffices to painlessly fix the issue.

Now, validation is also very widespread in HCI, where computer validates user input. As with all thing human however, this gets a little bit more tricky. You see, as a user, I'm not interested in bureaucracy in the slightest. It annoys me when I'm told "your application form is invalid". I expect software to guide me as I try to express my intentions. And while validation errors are a form of guidance, they tend to occur too late and tend to suggest user failure. You'd usually want someone who is willing to provide as much help as possible instead of accepting/rejecting your application and referencing some article no. in "rules".

Anyway, looks like this exploration-based model will dominate the transaction-based model in HCI. And the  simple "form validation" is falling apart here. At least, it should be way more granular, and ideally there should be no "validation errors". You just want your computer usage to be a pleasant experience, without any error popups. It seems often to be much better to just ignore incorrect input, rather than scream loudly "I don't understand what you mean, fool". Quite the opposite to what we do with validation in our code for API.

CQRS and ReadModel

While CQRS seems to be very useful tool for dealing with complex domains, I kinda got burned by it more times than I would like to admit. I'm talking about traditional interpretation, CQRS+ES+DDD ala Greg Young. The biggest frustration I'm facing is how it claims that readmodel can be somewhat an afterthought, yet how it still remains the crucial part, and unless you take a good care of it, you gonna have a bad time.

CQRS lets you defer decisions, not avoid them. You may think that choosing the reporting database is exactly the kind of decision CQRS lets you avoid. Well, not entirely. It allows you to offload the decision to someone else who is responsible for reporting. But still it's gonna be you for small one-man projects. And you'd better damn know well your database, as there are tons of nasty surprises awaiting with these beasts.

It is completely unsuitable for CRUD scenarios. If there is no complex processing of data coming in, using CQRS is certainly overkill. If your commands map 1:1 to your events map 1:1 to your read model, it is just common sense then this is a nonsense. Whoever says there is no overhead in doing this and you'd better just do it everywhere has no clue or is lying.

Naive implementations of read model only take you so far. There are some people around boasting of how CQRS+ES allowed them to get rid of the database. So this is really neat point with ES, that your primary data model (Events) is very simple. You can put serialized events anywhere you want, no need to tie  yourself to any particular DB. On the reporting side however, this is entirely different story. Unless you are writing most primitive and thus probably useless app, just storing your (demormalized) data in files is not gonna work. Where is fulltext search? Where are those adhoc queries business always bugs you about? Or even, where is the list of users sorted by date of registration? Pretty tricky to implement when all you have is partitioning by id, eh?

So, while I believe this is the way we will be building software in the future, the fact remains, CQRS just lets you ask the right questions at the right time, but never be fooled that this makes the answers somehow less important.

Defensive programming, YU so popular?

This term can have a lot of meanings. Everyone likes to attribute all the good thing to  a term they like while saying the bad things are misinterpretations or don't belong to that term altogether (Agile, anyone?). Usually when I hear people mentioning defensive coding, this is along the following lines, and this is what I take it for:

Defensive coding is doing everything possible to not make a fool of yourself.

Looks like people believe this is good thing, but to me it looks potentially very devious. This is like curing symptoms, not a problem. Covering your ass may be good for you in the short term, but definitely not for your team.

While defensive coding may contain often useful practices such as validating data coming from the external system, other more questionable practices tend to be associated to it ("check everything for null", "log all random things you can think of" ). To me, these are just Good practices, and Bad practices (in some context ofc.) - and defensive programming seems to contain practices from both of those sets.

Now there is another term, Offensive programming, which basically says, when you fail, scream as loud as you can, so that everyone else knows. This looks much more appealing to me. It looks like programming model Erlang advocates, where if worker fails, it does not attempt any tricks to recover, it just exits. It is responsibility of that worker's supervisor to decide what should be done in face of this failure.

There is but one place where defensive coding makes sense. And that place is most important of all in software. It is Production. Here, crashing the system is the worst thing that could possibly happen. It is often much worse than corrupting data, hiding the fact that something was not accomplished, etc. This is lying to your user. But if you lie rarely enough so that user doesn't notice it, this is better than telling this software is full of bugs by crashing and thus interrupting workflow.

So, defensive coding is bad for you (in the long term). It is bad for your team. It seems to be good for your business. Yet, I feel it is a sorry state of software development, where writing crap, and then defending against the crap you and your colleagues wrote is regarded as proper way of coding. Offensive coding raises the bar higher, and I want to believe it will be a way of doing things sometime in the future.

Sand castle in software

Consider what it means to write software following Open-Closed Principle (OCP).  You never modify existing code. If you need new functionality, you write a new code. Or maybe delete unused software units, to keep things clean.

In traditional statically-typed OO language, you would create class once, and never modify it again. What is the most important thing of that class? A name. You'd have to think really hard how to name this class, or else it will be the source of daily WTFs. I guess, that would be extremely hard.

The obvious benefit is that you never break anything that depends on your code. This enables you to write new code without fear that you may break something - which is very serious problem in software. This is my sand castle, a code that is as flexible as language allows it.

Now look at typical software. Each line of code is modified 10s if not 100s of times. Often changes have cascading effects. Code is full of unexpected behavior where names do not reflect the actual behavior, or even intent.

Why is it so hard to write code following OCP? I think, it is because we do not know what we are doing. Software we are supposed to create is beyond our comprehension. And so, we try creating random things, until something works as we thought it should. We may call this TDD, where we code to examples, and not full specification. On the other hand, there can be no specification other than code itself. The set of what -> how mappings is so big and with so many interesting options, there is no way we can choose The One, The Right option.

Decision making

I often hear this, that decision making capabilities have high influence on one's success. While this is true in  sense... I like to distinguish between 2 types of thinking behind decision-maker's mask.

Bull

The Bull has a strong personality. He sees the red flag, and he will hit it, no matter at what cost. He makes the decision, and then copes with the consequences. He does not think of the consequences upfront. He is strong enough to stand for his decisions, and commit to them. Once he has committed to a decision, he will not recognize he was wrong, no matter what it takes.

If faced with a hard problem, he will make a random decision. He will not admit he has no idea what the consequences will be. If there is a tree in his way, he doesn't give it a second thought before battering into it.

When asking a Bull to solve your problem, you will see his continuous efforts towards the goal. You will see total commitment and restless determination. Ultimately, despite all the complications, he will successfully deliver what you had asked. But you may be wondering, if the problem is indeed as hard as he was trying it to look like.

Butterfly

The Butterfly is a free thinker. He loves exploring the world. Decisions he makes come to him naturally, so as to best fit the problem into their world view. He likes to often change his opinion, and so he also hates to commit to his decisions. If he realizes he was wrong, he has no problem to admit it - he doesn't take this personally, he is just glad he now has better understanding of the world.

If faced with a hard problem, he will bounce off it like he had hit a tree. He will be helpless. The only way for him to proceed is to take time, investigate what the problem looks like, and only then, once he if familiar enough with the problem, he is comfortable with making a decision.

When asking a Butterfly to solve you his problem, you may not see progress for long periods of time. His reluctance to commit to anything may lead up to you considering to cancel the contract. But then suddenly he will do this all in a very short time. You may be wondering what he could be capable of accomplishing if he wasn't such a lazybones.

Coin

Our coin of 2 sides, each bad, is this:

1. Bull will commit to a bad decision and will not abandon it
2. Butterfly will not commit to any decision, and will frequently change his mind

Estimate: Guess and Commitment

Guess

So this is a typical situation. Manager comes to you, and says "Good day today, we've just got this new cool feature request... blabla ... This is extremely important feature, our bottom line greatly depends on how quickly we can deliver it. How much will it take you?".

 Being a nice day indeed, with a fresh head in the morning, excited about the great news, you feel noone can stop you. So you say "two weeks" (Thinking you could pull this off in a couple of all-nighters, but of course I will be clever and take reserve to cope with all those risks which of course will not affect Me, the Superhero. Just for the ease of mind.).

 "Great, so two weeks it is", says the manager rubbing his hands. "Please do your best. This is very important. It would be great if you could do this in one week. Now, the details...". And so, now you take a look at all the details. Needless to say, they are not what you expected at all.

 By this time, you start realizing how stupid you were. You barely know the requirements yet, having spent only a few moments to consider them. And still, you have managed to fully commit to do this, in a week (well, 2 weeks actually, but you know the expectation is 1 week, 2 being the worst case). You know you should have asked for time to consider and clarify requirements. But at the moment, you thought you fully understand what business means with this feature, and there is no need to clarify anything. Turns out you were wrong. They have very different view on what this feature means. And this view is at odds with your current architecture. And noone wants to hear about any compromises you are offering.

 And so, bit by bit, your motivation starts dropping. You start remembering how awfully you are underpaid and how your manager had no moral right to ask you to do this oh-so-important feature virtually for free, in the first place. To the level that you take full two months to complete this goddamn feature. It is buggy as hell. But of course it's not your fault. It's the fault of your manager, who was so stupid not to understand how you had to compromise your system's architecture to fit this abomination in. "How could anyone be so short-sighted", you wonder.

Commitment

 The next time you are much more careful. You are asked to estimate this apparently trivial feature. You ask for time to consider requirements. You clarify everything you can. Yes, this is indeed as easy as it looks, you conclude. Having spent 2 hrs estimating, you finally say "this will take me 4hrs, no more". Manager wonders "What took you so long to estimate this? I thought this was a no-brainer you should be able to do in a few hours".

 And indeed, by the 2nd hour, you are already done. Of course, the manager is not happy again. He accuses you of how was that extremely wasteful of you to spend 2 hrs estimating a feature, when you could have actually gotten it done by that time.

You are extremely frustrated by his assertion. Hadn't you put effort in understanding the problem, you'd probably had slipped on one of those edge cases. How can anyone be so blind?

Estimate

The two examples above are edge cases of what estimate could mean. The first is your rough guess. The second is your full commitment. They are very different things, and yet, when one is asked to give estimate, these two are often used interchangeably. One time, you just pull a number off the top of your head. Next time, you are very confident you have all cases covered; you know every little step that has to be made.

It is no wonder managers tend to look at estimates as commitments, while developers would like to take them as guesses. In ideal case, estimate should be a sweet spot somewhere in between. It is a probability distribution. It is your task as a developer to work with manager and find this sweet spot, so that you didn't spend too much time considering all the unlikely cases, and still have a reasonable prediction business can rely on.

As a developer, I am very guilty of doing this vile thing, being very inconsistent with my estimates. One time they are pure guesses based on my mood, the other time they are inevitable fact, based on deep knowledge of the problem. Managers hate that. They like to look at the numbers you give them in spreadsheet, and be confident those numbers are correct. The worst thing that could happen is when your "1" is sometime "2". Oh no, wait, it's more like it's sometimes "20".

In reality, you shouldn't give all your estimates with the same level of confidence. That would be wasteful. You need to find the sweet spot for each problem. And be very explicit where it lies. If you had seen this problem before and you know how to solve it, you should say so. Everyone will be happy. On the other hand, if you have no idea, you should tell your manager this could take years. He will not be happy and ask you to be more specific. You then take a few hours researching the problem, find a blog post describing solution to similar problem, and now you are confident this will take 1-2 weeks with prob. 90%. Manager is happy with such estimate, he asks you to stop researching and start coding.

Epilogue


Estimates are necessary when you want to have predictable results. In that case, we should ask how much predictable these results should be, and base our estimates accordingly. Estimate is probability distribution. It will probably be normal-distribution shaped for non-trivial problems. That means, there will be non-zero probability the task could take 100 years to complete.

If you think that's a nonsense, think about NP-completeness. Let's say the problem you need to solve is in NP. And you need efficient solution, for a general case. It is not known at the time whether the problem is in P. But who knows, maybe in 100 years someone will discover it is indeed in P. So what you do? You go to business and say "Here is and approximation, I am confident it will fit our case". If they say "No, no, we need it exactly as we specified. Continue searching. And please hurry, this thing is taking you too long" - I'm sorry for you mate. As someone said, "change company, or change company".

As you minimize risk, you minimize failure. But so you minimize success. The catch is: the more experience you have, the more you are minimizing failure rather than success when minimizing risk. Why? Because minimizing risk means making choices. The more experience you have, the better choices you can make. It doesn't mean experience will lead you to better choices always. But it is a vector, your North Star. It's like playing chess if you know nothing about openings vs someone who knows his openings. It is very unlikely you will avoid all opening traps and pull it off just because of your "lucky star", "genius mind", or anything of that sort.

As we see in software world today, we are strongly lacking in experience to minimize failure efficiently, while at the same time not to limit our success chances severely. Many successful software companies just made a random thing that worked. There are as many companies who made a random thing that failed. While we are at this level, there is no need for estimates, they just waste time. Build your MVP in a day, if people like it, start iterating based on feedback. and here you have it, a random product that works.

Multiple identical html page requests issue

If you have noticed your browser is doing multiple identical requests to load a single html page, chances are you have run into the same issue, empty string url in html. In my case, it was empty css reference:

    <link href="" rel="stylesheet" type="text/css"></link>

I consider myself lucky that it took just a couple of hours to figure out what is going on. Seriously, this could have gone a lot worse.

Here is my story. I noticed that one page was taking a little longer than it should to load. Sure, it may be Amazon's EC2 micro instance not having enough steam, considering the page was a bit heavy both computationally and by size. Still, I fired up Fiddler, and of course was a bit surprised to see that the page has been requested twice. I browsed to some other pages on that website, and Chrome was constantly issuing two apparently identical requests for a single page. Switched to IE9 - all OK. Opera - OK too. So this must be issue with Chrome. Maybe it is optimizing browsing experience, and when response doesn't arrive in timely manner, it retries, I thought. And indeed, I managed to find different traces of information that Chrome might be doing something like that; however, there was nothing authoritative. And by the way, why is it doing this to all pages, even those with sub-0.1s response? Now some really strange ideas start creeping into my mind. Maybe Chrome on my PC has decided that this particular website doesn't respond in timely manner and now has created behavior model for it - request every page twice? (yeah, I must be crazy) Check from my laptop - nope, the same issue. Then it comes to my attention that the second request is issued not immediately, perhaps after the first request got response? Checking timings with Fiddler confirms that this must be the case. Now then the last thing to notice -second request is a little bit different from first one, it has this header:

Accept: text/css,*/*;q=0.1

I take a look at my master template, and there it is, empty link css, a product of my hand-rolled css/js minimization solution. Yet another time I regret having opted for homemade stuff. Gotta go look at SquishIt once again.

There is no such thing as programming language performance

It seems obvious to me that noone should be talking about absolute performance of a particular programming language. Claims such as "C++ is faster than Java" don't make any sense at all. Comparing performance of compilers is better: "GCC 4.6.0 is faster than javac 1.6.0_14" makes more sense, although still cannot be called research.

The thing that can be measured is performance of a program for some programming language, compiled with particular compiler, which was run on particular machine.

People comparing "performance of programming languages" do exactly this. They take some problem, write programs to solve it in chosen languages, compile programs with some selected compilers, and then run them on some machine. This measures exactly what it measures. It is not absolute performance of a programming language, which even cannot have any reasonable definition.

I can think of 2 different options of performance measurement of a compiler (in context of solving some particular problem).

1. Program optimized for best performance.
2. Program written in idiomatic style for a language.

The 2nd option is more important. Language should be used in a way it was meant to be used. If program is written as if "fighting the language", then different language should be chosen, the one which allows to model solution in more straightforward way. (If you are measuring performance of C++, you probably should not write custom GC or Prolog interpreter)

So this is what performance of programming language usually means. It is subjective evaluation of performance of popular compiler(s), given various different problems and typical solutions in that language. It's not that it is bad or something, but such statements should always be preceded with "In my opinion...",  and some elaboration on context is very much in order.

Framework vs library

My simplistic interpretation of what is framework and what is library.

Framework

Library

Many try to distinguish them by direction of control,  a rather technical distinction:

1. If it calls you, this is framework.
2. If you have to call it, this is library.

Another slightly different point of view:

1. Framework is a skeleton for an application.
2. Library performs specific tasks for its clients.

So, I'm a fan of second definition. And although first one sounds reasonable too, I have a feeling it may not be entirely correct for all the cases.

Lockerz 24/7 BS

 

Foreword. If you don’t know what lockerz is, my rant won’t help you here. go read somewhere else.

impression of lockerz

So there is this site lockerz, which some people call scam, and I cannot say I disagree. As I see it, their methods for giving away free prizes are completely disrespectful to their users. Partially because in case user wants to take prize for his honestly earned ptz (thru redemption or 24/7), he’s taken into arena to compete with bots.

If they catch bot users post-factum, after the prize has been claimed by the bot, well the question arises where that prize goes. And why they only allow to place bid with just 5ptz increment? Is there any human being out there who enjoys playing according to such ridiculous rules?

Anyway, that’s not my topic. The topic is that after I’ve realized I’m competing against bots in 24/7, being software engineer, I decided to dedicate one day to think of something that could even the chances to compete with bot. That is: create one on my own.

 

Recaptcha

Lockerz uses reCAPCHA capcha service in their 24/7. And so, user has to enter captcha provided by this service to increment bid by this measly amount of 5ptz, and therefore prove that he is no bot. And yet, the fastest one wins, and has his bid placed while the others are left with a message “you’ve been outbid” and their wasted efforts entering captcha.

We assume automatic recaptcha solver is intractable. Whas is left? Human. One may use human workforce, like decaptcher service. Or, can spend little of his effort and enter some captchas by himself. Captchas provided by reCaptcha have expiration time (usually a few minutes), and (of course) lockerz doesn’t track how many captchas to be solved user has requested. So it must be clear now. We can just “presolve” some captchas, say 10, few minutes before auction ends, and that’s it, we’re in a good position to compete against other bots.

 

who wins?

Noone, but atleast I no longer feel being treated like a fool. It’s always unpleasant to realize you were not fully aware of game rules. So know the rules, dear reader, before you start playing the game seriously, and evaluate your strengths over your opponents. I was lucky in this regard.

Silverlight grid brush

Intro

I was developing Silverlight 4 application for drawing floor plan on a plane. One requirement was, there should be a grid mode, and while in this mode, drawing element should snap to the grid. So I need to display this grid on drawing surface. I could think of 2 options how to display a grid:

1. Many rectangle elements
2. Background brush

The problem with creating rectangle for each grid point was that it appeared to be extremely slow, considering the fact that the grid was supposed to be constantly changing. Maybe it had to do something with the implementation, as grid points were stored in ItemsControl, which was databound to constantly changing ItemsSource.

The problem with background brush is that there is no such a thing in silverlight 4 off-the-shelf. There is just extremely limited range of basic brushes (like SolidColorBrush, LinearGradientBrush). Although it is written in MSDN that you can derive from Brush class, I couldn't find example on how to do this. By definition, brush maps from a point coordinate to a color. Apparently, Brush class does much more, and since it doesn’t provide something like

public abstract class Brush
{
 public abstract Color Map(Point point);
}

(apparently, for performance considerations), it seemed like too much of a hassle to mess with the internals of Silverlight to implement my custom brush.

So, after some googling, I came up with 3rd option:

• Pixel shader effect

The idea of pixel shader effect is that each UIElement can have a pixel shader effect applied to it. Pixel shader is somewhat similar to a brush: it maps a pixel from the input image to a color at a corresponding location in the destination image. So it is like a better brush, as it can obtain color at any coordinates from the source image which regular brush cannot.

Silverlight shader effects are written in High-Level Shader Language(HLSL), a C-like language, originally invented by Microsoft for Direct3D. Ideally, pixel shaders should be executed on GPU, utilizing its high parallel throughput ability (each pixel can be processed independently). However, as of now(Silverlight 4), it only utilizes CPU’s SIMD instructions. In other words, today it tends to be quite costly, but this may improve in the future.

So, my idea was, I can create a shader effect which works just like a plain old brush: mapping from source coordinates to destination color(source color can be totally ignored). Shazzam is a good to help with WPF/Silverlight effects development. Having no experience with HLSL, I wrote the simplest thing I could think of, like this:

sampler2D input : register(s0);

// Grid effect shader

/// <summary>Size of the lattice</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>40</defaultValue>
float LatticeSize : register(C0);

/// <summary>Size X of the texture</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>1024</defaultValue>
float SizeX : register(C1);

/// <summary>Size Y of the texture</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>1024</defaultValue>
float SizeY : register(C2);

/// <summary>point size</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>2</defaultValue>
float PointSize : register(C3);

/// <summary>point color</summary>
/// <defaultValue>#FF707070</defaultValue>
float4 PointColor : register(C4);

float4 main(float2 uv : TEXCOORD) : COLOR
{
 float stepX = LatticeSize/SizeX;
 float stepY = LatticeSize/SizeY;
 float pointSizeX = (1 / SizeX)*PointSize;
 float pointSizeY = (1 / SizeY)*PointSize;

 float4 Color;
 Color= tex2D( input, uv.xy);

 float dx = uv.x % (stepX);
 float dy = uv.y % (stepY);

 if(stepX - dx < dx) dx = stepX - dx;
 if(stepY - dy < dy) dy = stepY - dy;

 if ((dx < pointSizeX)&&(dy < pointSizeY) )
 {
   Color = PointColor;
 }

 return Color;
}

and this kind of worked, but there was a problem of edge smoothing. Instead of smoothing edges of a grid point, my shader would create them of a different size, which looks very unpleasant:

Another issue was, this shader effect was also quite slow, considering the fact that it had to be applied for 10000x10000 canvas. Possibly due to all the if’s in the shader code. It is certainly possible to improve this shader so that it would smooth the point edges, and also optimize it big time.

Final solution

The final solution came to my mind after I saw it is possible to create striped brush in Silverlight using LinearGradientBrush, like this

<LinearGradientBrush StartPoint="0,0" EndPoint="0.1,0" SpreadMethod="Repeat">
  <GradientStop Offset="0" Color="Red"/>
  <GradientStop Offset="0.5" Color="Red"/>
  <GradientStop Offset="0.5" Color="White"/>
  <GradientStop Offset="1" Color="White"/>
</LinearGradientBrush>

and the idea that we can overlay one canvas on top of the another. So the solution looks like this:

<Grid>
 <Canvas Width="300" Height="300">
     <Canvas.Background>
         <LinearGradientBrush StartPoint="0,0" EndPoint="0,0.05" SpreadMethod="Repeat" >
             <GradientStop Offset="0" Color="#FF101010"/>
             <GradientStop Offset="0.1" Color="#FF101010"/>
             <GradientStop Offset="0.1" Color="#FFF0F0F0"/>
             <GradientStop Offset="1" Color="#FFF0F0F0"/>
         </LinearGradientBrush>
     </Canvas.Background>
 </Canvas>
 <Canvas Width="300" Height="300">
     <Canvas.Background>
         <LinearGradientBrush StartPoint="0,0" EndPoint="0.05,0" SpreadMethod="Repeat" >
             <GradientStop Offset="0" Color="Transparent"/>
             <GradientStop Offset="0.1" Color="Transparent"/>
             <GradientStop Offset="0.1" Color="#FFF0F0F0"/>
             <GradientStop Offset="1" Color="#FFF0F0F0"/>
         </LinearGradientBrush>
     </Canvas.Background>
 </Canvas>
</Grid>

Which produces the output:

It is still not ideal, as the grid points are somewhat blurred which becomes very apparent as point size approaches 1px on the display. This approach, however, has huge advantage over the others: it is lightning-fast! I think that this, combined with “right” resolutions, will be a reasonable solution.

Scala: reduceLeft vs foldLeft

Let’s say we have two types: , .

Also, let’s say we have operator +, for which += and +=. Then, speaking in types:

val a = List(, , , )

a.reduceLeft( (, ) => + ) : ==

((+)+)+

a.foldLeft()( (,) => + ) : ==

(((+)+)+)+

Edge cases:

val b = List[]() //empty list of

b.reduceLeft(_+_) //throw exception

b.foldLeft()(_+_) //returns

val c = List[]() //list of single

c.reduceLeft(_+_) // returns , the single element in the list

c.foldLeft()(_+_) //returns , result of operation +

Notice the difference: when list is empty, reduceLeft throws an exception, while foldLeft returns starting value. This can often help to decide which one to use.

Examples:

val a = List(1, 2, 3, 4)

//sum of integers in the list
val sum = a.reduceLeft( (u, v) => u+v )
//integers in the list separated by comma, using reduce
val strRepr1 = a.map( u => u.toString() )
  .reduceLeft( (u, v) => u+","+v ) 
  
//sum of squares, we use Long to support larger values
val sqrSum = a.foldLeft(0L)( (u, v)=> u+v*v )
//number if items in the list
var count = a.foldLeft(0) ( (u, v) => u+1 )
//integers in the list separated by comma, using fold
val strRepr2 = a.tail.foldLeft(a.head.toString())( (u, v) =>
  u+","+v )

The Right Methodology

Is Lean better than Agile? Scrum vs XP? Of course it all depends, and all methodologies can only be to some degree beneficial for current situation. And in the future new methodologies will appear, which will better address problems of the day. There can be no single “body of knowledge” which is relevant forever.

“But what about math”, you may ask. It seems to have stable body of knowledge (say, axiomatic set theory). Well yes, but we are solving problems on a different level today, taking as a foundation all that has been proven before. It’s not that the foundation is irrelevant, it’s that our focus is elsewhere, as we believe the foundation is stable :).

Also, there is no “single true” foundation. We may take another set of axioms and start building all over from there. It just depends on a problem at hand which theory is better.

That said, i feel Lean and Agile are both very beneficial methodologies for the environment I’m working in, ant it’s really pity how badly they can be misinterpreted. (“You’re not following the plan! Can’t you be a little more agile?”, “Agile is total and thorough crap. Lean, that’s out thing!”)

Making crap to hit deadline

Of course this works, and of course the other part (and then cleaning up the mess afterwards) doesn’t. If you have to clean things up afterwards, the only way is to sacrifice your own spare time.

I’ve never been allowed to do “code improvements” after feature perceivably works. “It works, go on with the next task! Can’t you see how far behind schedule are we?”. No one builds the perfect thing first time, and sadly I often find that my newborn code is not of high enough quality to be sufficiently maintainable. This is reality. I can write good code, but only when I spend some iterations improving it. I cannot make it good enough first time. Yes, it may be possible to do this, but only when working on familiar problem nth time. For instance good OO skills may do half the job, but what remains, the problem domain, and I don’t see a way to model it perfectly the first time.

I don’t want to work on the same damn problem for the nth time…

C# Switch statement & enums

I wish i could write something like this:

void foo(MyEnum e)

{

  switch(e) complete

  {
    case MyOption1:

      bar(); break;

    case MyOption2:

      baz(); break;

  }

}

and expect compiler error is MyEnum also contains MyOption3. Of course, I can add

default: throw new Exception();

but this is not always appropriate when the code is not covered by automated tests. In my case, that’s how it is exactly. Some code similar to above is executed only on rare occasions, so there is little probability that the problem will be spotted by manual testing. And when it is in production, it would rather fail silently than throw unhandled exception.

Of course is is considered a bad practice to litter code with such switch statements everywhere, and bar() and baz() would better be virtual functions on polymorphic objects.

Semi-Complex code

The codebase I am currently working on has some parts having as I call it, semi-complex logic. It has good names most of times, it adheres to reasonable coding standard, and it actually works. Well, most of time. However, it has copy-paste-alike constructs all around. There are lots of dead / redundant code. And each class has just too many responsibilities. This looks like a kind of code our developers seem to be happy with. Must be because they are most productive producing it. I too seem to be most productive when producing highly-entangled, low level of abstraction code. But in a few past years I settled to opinion that long-term qualities of code matter the most, event for small applications. And hence, nowadays I tend to write code I am not most productive writing, taking time to manage dependencies, eliminate duplication, extract methods. I’m still not that good at it, and of course no one appreciates it. But still I try…

What troubles me about this semi-complex code, is that it carries lots traits of bad code. Most notably, out-of-place dependencies. It suffices to have one wrong dependency, and soon inconveniences start to emerge. Like, having executable module link to seemingly unrelated one. And everyone seems to be OK with it, taking this as necessary evil. Another trouble point is that the code itself is easy to read, but it doesn’t mean you understand how it is working. Because responsibilities are not clearly separated, and some side-effects take place occasionally.

Exception handling in ASP.NET MVC. Clear the output when returning error view

Today spent half a day figuring out why my error handling code sometimes returns “strange” view, as if error page was rendered inside partial view in which exception occured. Which by the way is exactly what was happening.

Exception handling looked like:

public abstract class MyController : Controller

{

//… other stuff

protected override void OnException(ExceptionContext filterContext)
{

    //log exception
    var exception = filterContext.Exception;              
    var model = new ErrorModel(exception);
    filterContext.Result = View("Error", "Site", model);
    filterContext.ExceptionHandled = true;

}

}

Until finally I found the fix:

public abstract class MyController : Controller

{

//… other stuff

protected override void OnException(ExceptionContext filterContext)
{

    //log exception
    var exception = filterContext.Exception;              
    var model = new ErrorModel(exception);
    filterContext.Result = View("Error", "Site", model);

    filterContext.HttpContext.Response.Clear();
    filterContext.HttpContext.Response.StatusCode = 500; 
    filterContext.ExceptionHandled = true;

}

}

Travian

Generally, I don’t like http because of its long response times. AJAX may have improved it a little, but another problem arises: JavaScript, which is also just too slow (not to mention i am really not approving the fact that it’s not compiled – it’s just a waste of memory). However, it has one outstanding feature – incredible availability. And as technology (in terms of http (or maybe whatever replacement protocol Google is building) request response time :) ) progresses, more and more applications will benefit from having web interface.

Travian is quite simple web-based real-time war strategy game. Yet, it earned quite a fame in some previous years. Why do I play it?

1. I’m just a child. I need games :)
2. It’s possible to play it at work without annoying those above me too much.
3. It’s simple. It has simple interface, and simple mechanics. Which allowed it to be quite balanced, in terms of races. Like SC.
4. It’s social network. No matter how much I despised idea of finding friends online, it seems viable one. It’s due to Travian that I have some certain friends now.
5. It models war craft & social engineering quite well, IMO.

However, it’s dying. Because of lack of innovation, and some stupid changes, like nerfing Gauls, leading to race imbalances. Also, like other online games, I’ve seen, it carries two drawbacks: $ advantage and bots. While they are finally trying to solve 2nd problem – 1) It might be too late, 2) They may be implementing the wrong approach. The only solution I see to fighting against bots is making them obsolete. Just let everyone do what they are trying to do, or if you don’t have enough resources to be on track - enable plugin system like in Hattrick. But it seems as always, short-term $ is getting in the way :(.

So, in the end greed seems be be ruining in all. To Hell.

Software, Soft

I’m against attempts to bring discipline to SE akin to other engineering disciplines. Like the SEMAT movement. Software is just so much softer than materials of other engineering disciplines, that it allows for so much more freedom and agility (as indicated by success of agile). Attempts to prevent failures also prevent successes, and leads to stagnation. I don’t mind if someone creates those restrictions for themselves, but just hate when they try imposing such huge restrictions on me. Yes, it’s true in software it’s so easy to shoot yourself in the foot, and such disciplines can help to prevent that from happening. You may say this is an “Adult” approach, as opposed to “childish”. But there is just so much potential in Software, even child cannot embrace it, not speaking about adult. Bad practices will simply go away because they are not working, and there is no need to present me with The One And Only set of right practices, beyond which I should not be looking further. Software principles/guidelines/methodologies/etc should be soft.