Big O notation and The Jackal

In the 1973 movie (and 1971 novel) “The Day of the Jackal”, the “good guy” Inspector Lebel, suspects that the only reason the assassin (The Jackal) is always one step ahead of them is because there is a leak within the special council designated to stop the assassin.

Spoiler alert: Lebel weeds out the leak by tapping the phone of a council member, who it turns out, had a female friend who was the mole, perhaps saving the day by preventing such leaks in the future.[1]

Conversation that follows:

Minister (head of council): There is one thing: how did you know whose telephone to tap?
Lebel: I didn’t, so I tapped all of them.

This is fascinating from an algorithmic perspective, because Lebel went with a brute force method. The cost of his approach was probably Big O of O(n^2). However, given that the sample set was small enough, worked perfectly.

However, this type of approach is considered heresy in the programming business. We’re trained from day one, to expect a supersized-mega-garagnatuan-mamma-jamma-ridonculously-large dataset and always optimize our analytical approach to scale to millions, billions or higher. The solution works great when it’s finally fully functional and can be run *infinite* times on scale. However, we’re unable to use this system for data obtained before the solution is fully functional.

My point: Don’t ignore scaling but also don’t start off with trying to prematurely optimize your problem for the biggest dataset in existence. Design your approach to get the proverbial low hanging fruit and then evolve and optimize the solution, if needed.

Keep your eye on the target
Keep your eye on the target

[1] – I don’t condone phone tapping.

On a related note, The Day of the Jackal (1973 movie/1971 novel, not the Bruce Willis abomination from the 90s) – political thriller set in post WW2 mid 20th century Western Europe – is a must watch/read.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s