Archive for August, 2008

When human error isn’t

25 Aug

One of today’s headlines is Human error causes most Predator crashes.

Without reading the article, it should be obvious that this statement is simply not true. If there is a common recurring problem attributed to human error, it is most likely actually a design error.

A caption in that article reads

Research shows that most Predator mishaps are the result of inadequate skills, lack of teamwork and lack of situational awareness.

This could just as easily be rewritten:

Research shows that most Predator mishaps are the result of poor design. The controls as designed to not promote situational awareness, require extensive teamwork and higher than normal levels of skill to operate.

Rather than trying to adapt people to your designs, adapt your designs to the people who will be using them. If there’s a lack of teamwork, don’t require it. If situational awareness is a necessity, make sure your design promotes it, if the people using your design do not possess the necessary skills, don’t require those skills.

I’m not saying that’s easy, but those mishaps cost $1 million to $4 million according to the article, so it’s obviously well worth the effort.

1 Comment

Posted in design


Who will monitor the monitors?

21 Aug

This is the second time that an error in a monitoring/testing tool that we use has caused me to waste a good deal of time trying to solve a non-problem.

The first was when we were using a tool to test the scaling performance of a new feature. According to this tool the performance was absolutely abysmal with only 50 virtual clients, and I spent a couple of days writing alternative algorithms and trying to test them with this tool. The scalability was horrible no matter what I did, so I added some logging to see if something was going on that wasn’t supposed to be. It was. The testing tool was sending completely wrong information, creating completely unrealistic scenarios. The testing implementation was done by another developer who was sure that he had set it up right, and I was not familiar with the tool so I assumed that it was a problem with my code. Having another developer write the tests was supposed to save me time.

The latest monitoring snag was caused by a monitoring tool that, for our convenience, tails the php error log and sends the last couple hundred lines to us whenever there is an error. We kept getting the same error periodically and could not track down the source; everything seemed to be working perfectly, but we still got the error. Usage of the site every day is breaking new records so we assumed that it had something to do with the unprecedented load on servers, but could never pinpoint the source of the errors. Today I looked a bit more closely at the error log only to discover that the log was from 3 days ago!

So the question is, when your testing or monitoring tools report an error, how do you ensure that the error is real and not just in the tool itself?


On being a DJ

18 Aug

When I was in college (oh so long ago…) I was a DJ for our radio station, and then I was a music director. I loved being a DJ: having lots of new, interesting and unreleased music on tap, from Smashing Pumpkins to Underwater Boxer; having a channel to share that music with other people; being able to make a small band’s day by playing their stuff and reporting it to CMJ. Well, there was one part I didn’t care for so much: talking on the radio. I’m a bit shy, which is why although I loved being a DJ and music director at Eckerd College, I knew it wasn’t ever going to be a career path for me.

It’s interesting, then, that I work at Grooveshark where much of that dream is being fulfilled by participating in this movement. The one piece that is missing is having a channel to share music with other people and subsequently helping small bands by making them more discoverable. Well, now with the release of Autoplay in Grooveshark Lite, it’s kind of like I get to be everybody’s DJ. Of course a computer scientist would write a DJing program rather than doing the manual labor of DJing.

As Professor Fishman, the best professor who ever lived, was fond of saying in our classes, a computer scientist isn’t satisfied with just using computers to put other people out of a job, they won’t settle until they manage to put themselves out of a job too. To be fair, he usually talked about that in the context of AI and specifically programming languages such as LISP, where the program can rewrite itself, but I think it applies here as well.

Now I get to be everyone’s DJ, but with everyone’s help too. If the system is currently a bad DJ, keep giving it feedback and it will learn. Imagine if you got to call up your local radio station and yell at them every time they played something you didn’t like, and congratulate them every time they played something you liked. If they didn’t block your phone number, you’d end up with the ultimate radio station for you, and that’s what Grooveshark aims to be, although we admit it will take some time to get there.

Check out Autoplay, and let me know what you think.


Making it easy

16 Aug

There is an effort underway at GS to modify our framework to make writing code easier than ever before (for the second time). It’s a great idea in theory and the results of what I have seen so far certainly make doing certain things more convenient, which is great when you’re writing code.

But I can’t help but feel that we are perhaps barking up the wrong tree a bit.

Peter Hallam points out that programmers spend most of their time reading code, not writing it. So rather than focusing on making code easier to write, we should be sure that we are making it easy to read, understand and modify. Peter surmises that a 10% reduction in the time it takes to understand code is equivalent to a 100% reduction in the time that it takes to write code. That’s very significant.

One of the mantras I have heard a bit too much is “always favor composition over inheritance.” As Phil Haack points out, composition is great sometimes, but it’s not a perfect design either (because there isn’t one). Personally, I think it’s best to keep composition and inheritance in mind and always prefer whichever one going to lead to easy-to-understand code. Many times I find that to be inheritance. Sometimes the solution is even minor code duplication, such as having each page requiring authentication to do an explicit auth check rather than having the framework infer whether or not an auth check is required based on the name or, and I shudder at the thought, comments in the code.



15 Aug

Much like with financial investments, past performance of software is not necessarily indicative of future results.

It’s a good thing to keep in mind when debugging: test even the stuff you know isn’t broken.

No Comments

Posted in Coding


Is that a wig?

14 Aug

This evening, while sitting down with Chris Suter trying to work out the finer details of an algorithm over a pint, I was approached by a woman who asked: “This is probably an offensive question, but is that a wig?” in reference to my hair.

For the record, my hair is real, as unlikely as that may seem at first glance.

No Comments

Posted in life


Disturbing discovery

14 Aug

I came home to find a dying cockroach on my kitchen floor.
That discovery alone is not all that disturbing. I do live in FL, after all, where palmetto bugs are a fact of life and can be found just about anywhere for no good reason at all. No, I quite expect to very occasionally stumble across one in my home, so that was not of much concern to me.

What did concern me was that this cockroach was on its back mostly dead, missing at least two legs on the same side. This raises two questions:
1. If this cockroach came into my house that way, how did it manage to get that far before losing it?
2. If this cockroach came into my house fully intact, how did it end up that way?

I suspect that I would rather not know the answer to the latter question.


Posted in life


Be careful what you return

05 Aug

(and how you handle what has been returned)

Things have been busy at Grooveshark, as usual. These past couple of days I have been hunting a bug both cthulu-like in its scary-strangeness and ninja-like in its stealthy manner. I went through all my code related to this particular project several times with a fine-toothed comb and didn’t catch it until today.

Turns out it wasn’t so strange after all. The fault was definitely mine, but PHP’s quirks certainly didn’t help matters any.
I was using array_search in a straightforward manner, not to find the particular position of an item but to find whether or not the item was in the array at all. The one thing about array_search, especially in the context of PHP’s loose typing, is that if it finds nothing it returns false. Of course, php happily treats false as zero, so how do you check to see if array_search is saying that it wasn’t found, or is saying that it’s the first element in the array? You have to check using strict equivalency, which I remembered, so I wrote my code like this:

$found = array_search('something', $arr);
if ($found === false) {
//handle what you do when it's not in there

then, a little while ago, specs changed and there was another case that had to be handled the exact same way as when ‘something’ wasn’t in $arr. So I did this:

$found = array_search('something', $arr);
$found = $found || (SOME_OTHER_CASE);
if ($found === false) {
//handle what you do when it's not in there

So in other words, when I went back and looked at that code later, I didn’t notice the triple equals instead of the double, so without much thought I assumed that $found was already a boolean, when it was really only a sometimes-boolean. In a strictly typed language this mistake would, of course, not have been possible. More practically, if array_search returned something other than practically-zero, I would have been able to explicitly handle that special case and store the result of that explicit handling as a boolean. If you read the documentation you will see that array_search actually returned NULL before version 4.2.0. I have to wonder why they decided to change it.

The reason this bug was so hard to find was because it was only a bug when the item being searched was the first item in the array, which it turns out is not that often. By the time I found that bug, there were hundreds of newer lines of code to check first.

Now that the bug is solved and now that I am far into this very technical post I think it’s safe to leak a tiny bit of information about what you, dear user, can expect to see in Grooveshark Lite in the near future: autoplay. We have decided that we want to be your personal DJ. Our tack on this feature aims to get around the chicken and egg problem: how do you build recommendations without user feedback, and how do you get user feedback if you don’t have recommendations to make them want to use the system? I’m not going to answer that question directly, but we hope that you will find the autoplay sessions to be enjoyable, and as you provide feedback to the system, we’ll take that data and make it even better.