Coding « Jay Paroline – Grooveshark Dev

RSS

Archive for the ‘Coding’ Category

Making it easy

16 Aug

There is an effort underway at GS to modify our framework to make writing code easier than ever before (for the second time). It’s a great idea in theory and the results of what I have seen so far certainly make doing certain things more convenient, which is great when you’re writing code.

But I can’t help but feel that we are perhaps barking up the wrong tree a bit.

Peter Hallam points out that programmers spend most of their time reading code, not writing it. So rather than focusing on making code easier to write, we should be sure that we are making it easy to read, understand and modify. Peter surmises that a 10% reduction in the time it takes to understand code is equivalent to a 100% reduction in the time that it takes to write code. That’s very significant.

One of the mantras I have heard a bit too much is “always favor composition over inheritance.” As Phil Haack points out, composition is great sometimes, but it’s not a perfect design either (because there isn’t one). Personally, I think it’s best to keep composition and inheritance in mind and always prefer whichever one going to lead to easy-to-understand code. Many times I find that to be inheritance. Sometimes the solution is even minor code duplication, such as having each page requiring authentication to do an explicit auth check rather than having the framework infer whether or not an auth check is required based on the name or, and I shudder at the thought, comments in the code.

1 Comment

Posted in Coding, grooveshark

Quip

15 Aug

Much like with financial investments, past performance of software is not necessarily indicative of future results.

It’s a good thing to keep in mind when debugging: test even the stuff you know isn’t broken.

No Comments

Posted in Coding

Be careful what you return

05 Aug

(and how you handle what has been returned)

Things have been busy at Grooveshark, as usual. These past couple of days I have been hunting a bug both cthulu-like in its scary-strangeness and ninja-like in its stealthy manner. I went through all my code related to this particular project several times with a fine-toothed comb and didn’t catch it until today.

Turns out it wasn’t so strange after all. The fault was definitely mine, but PHP’s quirks certainly didn’t help matters any.
I was using array_search in a straightforward manner, not to find the particular position of an item but to find whether or not the item was in the array at all. The one thing about array_search, especially in the context of PHP’s loose typing, is that if it finds nothing it returns false. Of course, php happily treats false as zero, so how do you check to see if array_search is saying that it wasn’t found, or is saying that it’s the first element in the array? You have to check using strict equivalency, which I remembered, so I wrote my code like this:

$found = array_search('something', $arr); if ($found === false) { //handle what you do when it's not in there }

then, a little while ago, specs changed and there was another case that had to be handled the exact same way as when ‘something’ wasn’t in $arr. So I did this:

$found = array_search('something', $arr); $found = $found || (SOME_OTHER_CASE); if ($found === false) { //handle what you do when it's not in there }

So in other words, when I went back and looked at that code later, I didn’t notice the triple equals instead of the double, so without much thought I assumed that $found was already a boolean, when it was really only a sometimes-boolean. In a strictly typed language this mistake would, of course, not have been possible. More practically, if array_search returned something other than practically-zero, I would have been able to explicitly handle that special case and store the result of that explicit handling as a boolean. If you read the documentation you will see that array_search actually returned NULL before version 4.2.0. I have to wonder why they decided to change it.

The reason this bug was so hard to find was because it was only a bug when the item being searched was the first item in the array, which it turns out is not that often. By the time I found that bug, there were hundreds of newer lines of code to check first.

Now that the bug is solved and now that I am far into this very technical post I think it’s safe to leak a tiny bit of information about what you, dear user, can expect to see in Grooveshark Lite in the near future: autoplay. We have decided that we want to be your personal DJ. Our tack on this feature aims to get around the chicken and egg problem: how do you build recommendations without user feedback, and how do you get user feedback if you don’t have recommendations to make them want to use the system? I’m not going to answer that question directly, but we hope that you will find the autoplay sessions to be enjoyable, and as you provide feedback to the system, we’ll take that data and make it even better.

No Comments

Posted in Coding, grooveshark

SQL Schema and Graphs/Maps

18 Mar

A while ago I wrote about my auto-query generator project. I only just recently got around to finishing it up because other things had higher priority, and also because I wasn’t entirely convinced that I was doing things the bes way, and I wanted to take some time to experiment.

Matt sat down with me and analyzed the problem, and we decided that we could use the schema to create a graph with all of the edges (our ID columns are consistently named in each table), and then use a shortest-path-finding algorithm, and then I could write a SQL generator that works off of the path. Getting all of the IDs in our tables in MySQL is pretty easy:
SELECT DISTINCT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME LIKE '%ID'
Then getting all the tables for a given ID:
SELECT TABLE_NAME, COLUMN_KEY FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME = '$id' AND TABLE_SCHEMA = 'yourschemahere'

From that information, I simply built a graph which I represented with an adjacency map.

Unfortunately, that did not work. The path finding algorithm was basically too good, finding paths that were the shortest, but not necessarily the correct way to get from one table to another. For example, two tables might contain a GenreID, but maybe they are actually linked by ArtistID. Ok, so what about only making an edge when that column is a primary key in one of the nodes representing a table? That wasn’t hard to do, either, but it still gave wrong results in some cases. Sometimes it’s just more efficient (but wrong) to route through the Genres table than go the right way.

I considered making a directed graph so that connections would only be one-way to the table with the ID as a primary key, but I realized that wouldn’t work either, because sometimes you do need to join tables based in IDs that are not primary keys. Essentially, our schema does not completely represent the full complexity of the relationships that it contains.

So I went back to my original method, which was to map out the paths by hand. Tedious though it may have been, it’s still a pretty clever solution, in my opinion.

I created two maps. The first simply says “if you have this ID, and you are trying to get to this table, start at this other table,” for every possible ID, and the next one simply says “if you’re at this table, and you’re trying to get to this other table, here is the next table you need to go through.”

The great thing about this is that most of those steps can be reused, but I only had to create them once. For example, it’s always true that to get from Users to Files you must go through UsersFiles, no matter what your starting point is, although you may be trying to find all of the Songs, Albums or Artists that a user has in their library.

Having spelled things out this way, there is no guesswork for a path finding algorithm, because there is literally only one path. In fact it hardly counts as a path finder or an algorithm; it just iterates through the map until it reaches its destination. And it works. I will post as many details as I’m allowed about exactly how the actual SQL building algorithm works, and about how I am able to merge multiple paths, so for example you can have an ArtistID and a PlaylistID to get all of the artists on a given playlist. Stay tuned.

No Comments

Posted in Coding, SQL

Let me drive

12 Mar

One odd quirk that I have noticed about myself as I have been building a reputation as an^‡ SQL guru at Grooveshark, and therefore being regularly pulled aside to look at queries, is that I have a hard time thinking about an SQL query when I’m just looking at it over someone’s shoulder. It’s even harder for me to think about how I would change the query.

For some reason, I need to be in the proverbial driver’s seat. Let me sit down in front of the screen, give me a gui text editor that I can use easily (vim does not qualify), and my brain is prepared to evaluate the problem. I may not even need to type anything out in the process of solving the problem, but the brain juice just won’t even start to flow if I don’t have that.

I seem to have that problem much less when looking at PHP (although I prefer to look at the code in my IDE of choice), and I’m not entirely sure why that is. The only thing I can think of is that maybe SQL requires a higher level of abstract thought than PHP does most of the time, so I am more dependent on having the right set up before I can get into the right mode?

Do any other coders out there have this problem?

‡ ‘an’ is the correct usage here because I expect you to read that as S Q L, not Sequel.

1 Comment

Posted in Coding

OhHai->I->HasA(UPDAET)

04 Feb

As an update to my previous lolcode post, we are fixing the GetGenre()->GetGenre() issue by calling them names. GetGenre()->GetName()

There is talk of adding __toString() functions to classes like Genres and Tags but I tend to not be a fan of automagic functions. __toString() would enable us to just call GetGenre() and if we treat the resulting object as a string, it will call GetName() behind the scenes, and if we treat it like an object it will still be an object. That is a “neat” language feature, but I believe it leads to obscurity and inconsistent behavior in certain cases.

For example, if the object is not directly treated as a string even though it needs to be a string, it __toString() will not be called, and problems will ensue. Confusing problems, because the object acts like a string, sometimes.

sloppy example code:

class notAString
{
    public $what;
    public function __construct($val)
    {
        $this->what = $val;
    }
    public function __toString()
    {
        return $this->what;
    }
}
$whatIsIt = new notAString("a string");
$isAString = is_string($whatIsIt);
$isAnObject = is_object($whatIsIt);
var_dump(array('isAString' => $isAString, 'isAnObject' => $isAnObject));
echo $whatIsIt;

Output:array 'isAString' => boolean false 'isAnObject' => boolean true a string

It fails the is_string check, so if you pass the object to a function that expects a string, and the function is smart enough to check for a string before doing anything with it, your call is going to fail and you’re going to be scratching your head wondering why.

Now imagine how confusing this would be if you were trying to debug a piece of code that you had no hand in writing, you see this object being used as a string, only you don’t know it’s an object because it’s being used as a string and that part of the cod works. “It should be declared right there, just look and you’ll see it’s an object.” Sure, or it could be passed in from another function and you haven’t looked that far up the ladder yet.

Worse, you finally figure out that it’s an object, and now you can’t figure out why it’s successfully being treated as a string elsewhere. You look at the class and you don’t see a __toString() function. You look at the parent class, no __toString() there either. Ah well, a red herring, time to move on right? Or did you give up before looking at the parent class’s parent? Was there a __toString() there? How much time was wasted trying to find that, compared to how much time the automagic __toString() function might save you as a developer?

I’d wager it’s not worth the lost time, and the added frustration.

No Comments

Posted in Coding

OhHai->IHasASong()->ICanHasGenre()

03 Feb

Sometimes naming conventions can have weird side effects.

Consider this example:
Our tables and fields are named in CamelCase (or StudleyCaps, or StudleyCamels as we like to call them), this is not my preferred way of naming tables/fields, but it’s what we’ve got so we work with it. We* decided that Genres are entities so the Genres table contains a Genre field which is a VARCHAR.
In PHP we are using our own flavor of ORM that avoids the pitfalls of most ORM systems while keeping the benefits. Our database objects all have Get methods for each object property (which may or may not be a field in the table that the object represents). If you want a song’s Genre, you call Song->GetGenre(), which gives you a Genre object. If you then want the text-representation of the Genre, you have to call…GetGenre(). So if you want, from the Song object, the text representation of the Song’s Genre, you call: Song->GetGenre()->GetGenre()
We might as well have made it say Song->GetGenre()->PrettyPlease() or ->NoSeriously()

At least we aren’t writing our code lolcats style (see: lolcode) or it might look more like the title:
OhHai->I->HasASong()->CanHasGenre()->CanHasGenreNoowwwwwwws()

*actually someone else decided before I ever started working here

Kthxbye.

3 Comments

Posted in Coding

Auto-query-generator

02 Feb

They say that Computer Scientists aren’t satisfied with putting other people out of jobs; they want to put themselves out of a job as well. In that vein I am writing a function that, given basic information about our database tables and their relationships, can build efficient queries automatically based on a complex filter consisting of:

The information requested
Any combination of IDs to filter the results based on

For example, you can tell it that you want the names and IDs of Playlists belonging to a user with a certain artist and my function will build the SQL on the fly based on what it knows.
This is extremely useful because I am working on a super secret project* with Katy and her app has no knowledge of hte database but needs to be able to ask for some extremely specific sets of data depending on user behavior. Instead of manually writing N! queries to handle every possible combination of requests, my function does all of the heavy lifting for me.

The code isn’t quite done yet but the code that figures out the join sequence and which key to join on is done, and surprisingly it only takes about 10 lines of code. I still need to add code to handle special cases, and I need to write the code that takes that sequence and turns it into real live SQL so it will surely grow, but that is still far less than I imagined would be required to get this far.

*not actually a secret project, but I haven’t written about it yet and it’s beyond the scope of this post.

1 Comment

Posted in Coding, SQL

The Importance of Being Earnest

27 Jan

One of the facts of life you have to work with when working at a startup is that your code will always be changing. New features are added, old code gets re-factored, the database layer gets overhauled, etc. That, coupled with the fact that there is always more that needs to be done than time to do it, add up to having almost no documentation for anything.

This is why it is critically important to have self-documenting code. If your code is not self-documenting, and I need to interface with it, then I have to wait for you to have time to answer my questions, or spend a lot of time digging through your code trying to figure out what is actually going on.

There is actually something worse than non-self-documenting code, however, and that’s self-documenting code that is incorrect. If you do not have real documentation and the names you use imply one thing but do another, you are going to cause serious problems.

Of course, I have examples of both problems. In the first, we have two entities that I needed to extrapolate information from: Duration and Bitrate. Do you see the problem? No? What if I ask you for the file size of a song given the following: Bitrate: 190000, Duration: 211749000. Can you tell me at a glance what math would be required in order to get the file size? You can’t, because the units are missing.

For the second example we have something that actually caused a major bug in Grooveshark that is just now in the process of being fixed. We have a field called isOnline. Do you think that isOnline tells whether a file is online? It doesn’t. Well, not always. If isOnline is 0, the file is definitely offline. But if isOnline is 1, it might still be offline, because what is really being tracked by this field is whether the file has been removed from a user’s library (or rather, the inverse of that). With a name like isOnline, it’s quite tempting to assume that that is how you check whether a file is online, and so that is what happened. Which is why often times when you click play on a song in Grooveshark, you get a peer offline error. The file isn’t online because the peer isn’t online, but isOnline is set to 1 because the user has not removed the file.

1 Comment

Posted in Coding, SQL