2011-09-30

Multiple identical html page requests issue

If you have noticed your browser is doing multiple identical requests to load a single html page, chances are you have run into the same issue, empty string url in html. In my case, it was empty css reference:
    <link href="" rel="stylesheet" type="text/css"></link>
I consider myself lucky that it took just a couple of hours to figure out what is going on. Seriously, this could have gone a lot worse.

Here is my story. I noticed that one page was taking a little longer than it should to load. Sure, it may be Amazon's EC2 micro instance not having enough steam, considering the page was a bit heavy both computationally and by size. Still, I fired up Fiddler, and of course was a bit surprised to see that the page has been requested twice. I browsed to some other pages on that website, and Chrome was constantly issuing two apparently identical requests for a single page. Switched to IE9 - all OK. Opera - OK too. So this must be issue with Chrome. Maybe it is optimizing browsing experience, and when response doesn't arrive in timely manner, it retries, I thought. And indeed, I managed to find different traces of information that Chrome might be doing something like that; however, there was nothing authoritative. And by the way, why is it doing this to all pages, even those with sub-0.1s response? Now some really strange ideas start creeping into my mind. Maybe Chrome on my PC has decided that this particular website doesn't respond in timely manner and now has created behavior model for it - request every page twice? (yeah, I must be crazy) Check from my laptop - nope, the same issue. Then it comes to my attention that the second request is issued not immediately, perhaps after the first request got response? Checking timings with Fiddler confirms that this must be the case. Now then the last thing to notice -second request is a little bit different from first one, it has this header:
Accept: text/css,*/*;q=0.1
I take a look at my master template, and there it is, empty link css, a product of my hand-rolled css/js minimization solution. Yet another time I regret having opted for homemade stuff. Gotta go look at SquishIt once again.

2011-06-15

There is no such thing as programming language performance

It seems obvious to me that noone should be talking about absolute performance of a particular programming language. Claims such as "C++ is faster than Java" don't make any sense at all. Comparing performance of compilers is better: "GCC 4.6.0 is faster than javac 1.6.0_14" makes more sense, although still cannot be called research.

The thing that can be measured is performance of a program for some programming language, compiled with particular compiler, which was run on particular machine.

People comparing "performance of programming languages" do exactly this. They take some problem, write programs to solve it in chosen languages, compile programs with some selected compilers, and then run them on some machine. This measures exactly what it measures. It is not absolute performance of a programming language, which even cannot have any reasonable definition.

I can think of 2 different options of performance measurement of a compiler (in context of solving some particular problem).

  1. Program optimized for best performance.
  2. Program written in idiomatic style for a language.
The 2nd option is more important. Language should be used in a way it was meant to be used. If program is written as if "fighting the language", then different language should be chosen, the one which allows to model solution in more straightforward way. (If you are measuring performance of C++, you probably should not write custom GC or Prolog interpreter)

So this is what performance of programming language usually means. It is subjective evaluation of performance of popular compiler(s), given various different problems and typical solutions in that language. It's not that it is bad or something, but such statements should always be preceded with "In my opinion...",  and some elaboration on context is very much in order.

2011-05-10

Framework vs library

My simplistic interpretation of what is framework and what is library.


Framework
Library

Many try to distinguish them by direction of control,  a rather technical distinction:

  1. If it calls you, this is framework.
  2. If you have to call it, this is library.

Another slightly different point of view:

  1. Framework is a skeleton for an application.
  2. Library performs specific tasks for its clients.
So, I'm a fan of second definition. And although first one sounds reasonable too, I have a feeling it may not be entirely correct for all the cases.

2011-01-26

Lockerz 24/7 BS

 

Foreword. If you don’t know what lockerz is, my rant won’t help you here. go read somewhere else.

impression of lockerz

So there is this site lockerz, which some people call scam, and I cannot say I disagree. As I see it, their methods for giving away free prizes are completely disrespectful to their users. Partially because in case user wants to take prize for his honestly earned ptz (thru redemption or 24/7), he’s taken into arena to compete with bots.

If they catch bot users post-factum, after the prize has been claimed by the bot, well the question arises where that prize goes. And why they only allow to place bid with just 5ptz increment? Is there any human being out there who enjoys playing according to such ridiculous rules?

Anyway, that’s not my topic. The topic is that after I’ve realized I’m competing against bots in 24/7, being software engineer, I decided to dedicate one day to think of something that could even the chances to compete with bot. That is: create one on my own.

 

Recaptcha

Lockerz uses reCAPCHA capcha service in their 24/7. And so, user has to enter captcha provided by this service to increment bid by this measly amount of 5ptz, and therefore prove that he is no bot. And yet, the fastest one wins, and has his bid placed while the others are left with a message “you’ve been outbid” and their wasted efforts entering captcha.

We assume automatic recaptcha solver is intractable. Whas is left? Human. One may use human workforce, like decaptcher service. Or, can spend little of his effort and enter some captchas by himself. Captchas provided by reCaptcha have expiration time (usually a few minutes), and (of course) lockerz doesn’t track how many captchas to be solved user has requested. So it must be clear now. We can just “presolve” some captchas, say 10, few minutes before auction ends, and that’s it, we’re in a good position to compete against other bots.

 

who wins?

Noone, but atleast I no longer feel being treated like a fool. It’s always unpleasant to realize you were not fully aware of game rules. So know the rules, dear reader, before you start playing the game seriously, and evaluate your strengths over your opponents. I was lucky in this regard.

2010-12-10

Silverlight grid brush

Intro

I was developing Silverlight 4 application for drawing floor plan on a plane. One requirement was, there should be a grid mode, and while in this mode, drawing element should snap to the grid. So I need to display this grid on drawing surface. I could think of 2 options how to display a grid:

  1. Many rectangle elements
  2. Background brush

The problem with creating rectangle for each grid point was that it appeared to be extremely slow, considering the fact that the grid was supposed to be constantly changing. Maybe it had to do something with the implementation, as grid points were stored in ItemsControl, which was databound to constantly changing ItemsSource.

The problem with background brush is that there is no such a thing in silverlight 4 off-the-shelf. There is just extremely limited range of basic brushes (like SolidColorBrush, LinearGradientBrush). Although it is written in MSDN that you can derive from Brush class, I couldn't find example on how to do this. By definition, brush maps from a point coordinate to a color. Apparently, Brush class does much more, and since it doesn’t provide something like

public abstract class Brush
{
public abstract Color Map(Point point);
}

(apparently, for performance considerations), it seemed like too much of a hassle to mess with the internals of Silverlight to implement my custom brush.

So, after some googling, I came up with 3rd option:

The idea of pixel shader effect is that each UIElement can have a pixel shader effect applied to it. Pixel shader is somewhat similar to a brush: it maps a pixel from the input image to a color at a corresponding location in the destination image. So it is like a better brush, as it can obtain color at any coordinates from the source image which regular brush cannot.

Silverlight shader effects are written in High-Level Shader Language(HLSL), a C-like language, originally invented by Microsoft for Direct3D. Ideally, pixel shaders should be executed on GPU, utilizing its high parallel throughput ability (each pixel can be processed independently). However, as of now(Silverlight 4), it only utilizes CPU’s SIMD instructions. In other words, today it tends to be quite costly, but this may improve in the future.

So, my idea was, I can create a shader effect which works just like a plain old brush: mapping from source coordinates to destination color(source color can be totally ignored). Shazzam is a good to help with WPF/Silverlight effects development. Having no experience with HLSL, I wrote the simplest thing I could think of, like this:

sampler2D input : register(s0);

// Grid effect shader

/// <summary>Size of the lattice</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>40</defaultValue>
float LatticeSize : register(C0);

/// <summary>Size X of the texture</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>1024</defaultValue>
float SizeX : register(C1);

/// <summary>Size Y of the texture</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>1024</defaultValue>
float SizeY : register(C2);

/// <summary>point size</summary>
/// <minValue>1/minValue>
/// <maxValue>10000</maxValue>
/// <defaultValue>2</defaultValue>
float PointSize : register(C3);

/// <summary>point color</summary>
/// <defaultValue>#FF707070</defaultValue>
float4 PointColor : register(C4);

float4 main(float2 uv : TEXCOORD) : COLOR
{
float stepX = LatticeSize/SizeX;
float stepY = LatticeSize/SizeY;
float pointSizeX = (1 / SizeX)*PointSize;
float pointSizeY = (1 / SizeY)*PointSize;

float4 Color;
Color= tex2D( input, uv.xy);

float dx = uv.x % (stepX);
float dy = uv.y % (stepY);

if(stepX - dx < dx) dx = stepX - dx;
if(stepY - dy < dy) dy = stepY - dy;

if ((dx < pointSizeX)&&(dy < pointSizeY) )
{
Color = PointColor;
}

return Color;
}

and this kind of worked, but there was a problem of edge smoothing. Instead of smoothing edges of a grid point, my shader would create them of a different size, which looks very unpleasant:

grid_shader

Another issue was, this shader effect was also quite slow, considering the fact that it had to be applied for 10000x10000 canvas. Possibly due to all the if’s in the shader code. It is certainly possible to improve this shader so that it would smooth the point edges, and also optimize it big time.

Final solution

The final solution came to my mind after I saw it is possible to create striped brush in Silverlight using LinearGradientBrush, like this

<LinearGradientBrush StartPoint="0,0" EndPoint="0.1,0" SpreadMethod="Repeat">
<GradientStop Offset="0" Color="Red"/>
<GradientStop Offset="0.5" Color="Red"/>
<GradientStop Offset="0.5" Color="White"/>
<GradientStop Offset="1" Color="White"/>
</LinearGradientBrush>

and the idea that we can overlay one canvas on top of the another. So the solution looks like this:

<Grid>
<Canvas Width="300" Height="300">
<Canvas.Background>
<LinearGradientBrush StartPoint="0,0" EndPoint="0,0.05" SpreadMethod="Repeat" >
<GradientStop Offset="0" Color="#FF101010"/>
<GradientStop Offset="0.1" Color="#FF101010"/>
<GradientStop Offset="0.1" Color="#FFF0F0F0"/>
<GradientStop Offset="1" Color="#FFF0F0F0"/>
</LinearGradientBrush>
</Canvas.Background>
</Canvas>
<Canvas Width="300" Height="300">
<Canvas.Background>
<LinearGradientBrush StartPoint="0,0" EndPoint="0.05,0" SpreadMethod="Repeat" >
<GradientStop Offset="0" Color="Transparent"/>
<GradientStop Offset="0.1" Color="Transparent"/>
<GradientStop Offset="0.1" Color="#FFF0F0F0"/>
<GradientStop Offset="1" Color="#FFF0F0F0"/>
</LinearGradientBrush>
</Canvas.Background>
</Canvas>
</Grid>

Which produces the output:

grid_combinedBrush

It is still not ideal, as the grid points are somewhat blurred which becomes very apparent as point size approaches 1px on the display. This approach, however, has huge advantage over the others: it is lightning-fast! I think that this, combined with “right” resolutions, will be a reasonable solution.

2010-11-10

Scala: reduceLeft vs foldLeft

Let’s say we have two types: Soccer ball, Auto.

Also, let’s say we have operator +, for which Soccer ball+Soccer ball=Soccer ball and Auto+Soccer ball=Auto. Then, speaking in types:

val a = List(Soccer ball, Soccer ball, Soccer ball, Soccer ball)

a.reduceLeft( (Soccer ball, Soccer ball) => Soccer ball + Soccer ball ) : Soccer ball ==

((Soccer ball+Soccer ball)+Soccer ball)+Soccer ball

a.foldLeft(Auto)( (Auto,Soccer ball) => Auto + Soccer ball ) : Auto ==

(((Auto+Soccer ball)+Soccer ball)+Soccer ball)+Soccer ball

Edge cases:

val b = List[Soccer ball]() //empty list of Soccer ball

b.reduceLeft(_+_) //throw exception

b.foldLeft(Auto)(_+_) //returns Auto

val c = List[](Soccer ball) //list of single Soccer ball

c.reduceLeft(_+_) // returns Soccer ball, the single element in the list

c.foldLeft(Auto)(_+_) //returns Auto, result of operation Auto+Soccer ball

Notice the difference: when list is empty, reduceLeft throws an exception, while foldLeft returns starting value. This can often help to decide which one to use.

Examples:

val a = List(1, 2, 3, 4)

//sum of integers in the list
val sum = a.reduceLeft( (u, v) => u+v )
//integers in the list separated by comma, using reduce
val strRepr1 = a.map( u => u.toString() )
  .reduceLeft( (u, v) => u+","+v ) 
  
//sum of squares, we use Long to support larger values
val sqrSum = a.foldLeft(0L)( (u, v)=> u+v*v )
//number if items in the list
var count = a.foldLeft(0) ( (u, v) => u+1 )
//integers in the list separated by comma, using fold
val strRepr2 = a.tail.foldLeft(a.head.toString())( (u, v) =>
  u+","+v )

2010-09-24

The Right Methodology

Is Lean better than Agile? Scrum vs XP? Of course it all depends, and all methodologies can only be to some degree beneficial for current situation. And in the future new methodologies will appear, which will better address problems of the day. There can be no single “body of knowledge” which is relevant forever.

“But what about math”, you may ask. It seems to have stable body of knowledge (say, axiomatic set theory). Well yes, but we are solving problems on a different level today, taking as a foundation all that has been proven before. It’s not that the foundation is irrelevant, it’s that our focus is elsewhere, as we believe the foundation is stable :).

Also, there is no “single true” foundation. We may take another set of axioms and start building all over from there. It just depends on a problem at hand which theory is better.

That said, i feel Lean and Agile are both very beneficial methodologies for the environment I’m working in, ant it’s really pity how badly they can be misinterpreted. (“You’re not following the plan! Can’t you be a little more agile?”, “Agile is total and thorough crap. Lean, that’s out thing!”)