Jay Paroline - Grooveshark Dev

Here’s a fun issue I ran into a little while ago, while doing DB maintenance at Grooveshark.

MySQL has a handy atomic rename function, where you can do something like this:
RENAME TABLE CurrentTable TO OldTable, NewTable TO CurrentTable
Both renames are done as a single atomic action, so that the table CurrentTable is always there, and Nothing Has To Break*

A few days ago, I took advantage of this feature. I needed to make some schema changes to one table that was being read from but not written to.

CREATE TABLE NewTable LIKE CurrentTable;
ALTER TABLE NewTable DROP COLUMN UnnecessaryColumn;
...
INSERT INTO NewTable SELECT (...) FROM CurrentTable;
RENAME TABLE CurrentTable TO OldTable, NewTable TO CurrentTable;
That way we have a live, selectable backup of the table in case I really screwed something up, not to mention that moving the tables this way meant we could finally get that huge table into file per table, as that table had been around since before we had turned on the file per table setting in MySQL.

Several other tables have foreign key constraints pointing to CurrentTable, and unfortunately MySQL is smart enough to notice that CurrentTable is being renamed, so it “helpfully” updates all the foreign key constraints in those tables so that they are now pointing at OldTable. Without mentioning it. The problem didn’t become apparent until CurrentTable was being written to again, and then the other tables with the foreign keys were being written to pointing to the new data in CurrentTable, except the updated constraints meant MySQL tried to find the rows in OldTable, causing the inserts to fail.

This is definitely one of those cases where being smart and helpful can sure lead to a lot of misery. I had to hunt down every table that had referenced CurrentTable to change them back by hand. If MySQL had been just a little bit smarter it would have seen that CurrentTable wasn’t *really* going away and it wouldn’t have made those changes for me. If it was a little bit dumber, it would have done nothing and things would have just kept working, or it would have blocked me from doing my rename, forcing me to deal with the constraints before they broke anything.

* except when it does

Sorry about the new ads on Grooveshark Lite, folks. They certainly don’t make me very happy either, but in these tough economic times, we gotta pay the bills somehow!

The good news is that these ads should free up some resources so we can spend more time making Grooveshark Lite even more awesome.

If you have thoughts on the new ads, Ben’s post on the community page has more information about the ads and a couple of ways to get in touch with us. Feedback is always appreciated.

It’s finally here: the Grooveshark widget! To show it off, here’s a collection of songs by The Bird and the Bee; one of my new favorite bands.

One problem we have had at Grooveshark is needing to verbally distinguish between CamelCase and camelCase. As you can see, they have the same name. Well, according to Wikipedia they are called lower camelCase and upper CamelCase, but those names are both clunky and probably not particularly intuitive.

Here at Grooveshark we’ve tried to come up with something better. For a while we tried calling upper CamelCase StudlyCaps while calling lower camelCase just camelCase. That didn’t sit well with everyone since even this spelling of STuDLeYCAps meets the definition, so we decided to invent a new word for upper CamelCase: StudleyCamels while still calling lower camelCase camelCase. Then another problem arose: how do you let people know that when you say camelCase you aren’t just failing to be specific?

Enter: headlessCamels (or decappedCamels). The visualization fits: you still have the hump(s), but the head (the leading uppercase) is missing, so you have a headlessCamel. As for StudleyCamels, I visualize a camel proudly sporting studded leather duds, and he’s holding his head high because he’s proud of his studs. But then again, I’m pretty weird.

Tonight we added about 400k files to Grooveshark Lite. If there’s anything you haven’t been able to find on Lite, there’s a good chance that it’s on there now. It should also take significantly less time for newly uploaded content to be added to Lite from now on.

Searching Grooveshark Lite just got even easier. At least if you’re a Firefox user. Get the search extension by going to listen.grooveshark.com, click on the search drop down in the upper right-hand corner in Firefox, and click “Add Grooveshark Lite Search”, and you’re done! Now whenever you want to search for a song or an artist, just select Grooveshark Lite from the search drop down and type in what you’re looking for. Couldn’t be easier.

I believe I have already mentioned a couple of times how memcache makes Grooveshark Lite possible.

Well, while implementing memcache for Grooveshark Lite, I ended up making a wrapper class called GMemcache for lack of a better name, which inherits from the PHP Memcache class, adding on just a few cool enhancements:

1. Singleton. Has static variable $instance that is set in the constructor and is also available in a getInstance() method.
2. Disable-able. Integrating with memcache can involve writing a lot of memcache-specific code, which means it can be quite a pain if you then need to temporarily disable memcache for some reason, such as testing. We define MEMCACHE_DISABLED to true when we wish to have our code bypass memcache altogether. The class wraps get, set, etc., and if memcache is disabled returns false for get (equivalent of saying “I couldn’t find it”) and doesn’t do anything when set is called. otherwise it calls parent::get etc.
3. Version-able. We also define an app version, which is then automatically appended to all keys. That way new releases never get stale data (or worse, data in a different structure from what it expects!). For situations where versioning does not make sense, set, get, etc all take an extra boolean parameter which indicates whether versioning should be skipped (defaults to false).

The nice thing about inheriting from Memcache this way (as opposed to having a Memcache instance as a member variable) is that we don’t have to rewrite all of the functionality. Whatever parts we do not need to change or do not care about, do not have to be written and they will continue to work when accessed via GMemcache.

If you are not already familiar with SOAP, this note will mean very little to you, and I don’t feel like explaining it so please feel free to ignore.

We are in the middle of adding a new payment processor to GS, and this particular payment processor, who shall remain unnamed, decided to implement their API in SOAP. I am not a huge fan of SOAP, but it certainly does have some level of usefulness: the entire backend API for GSLite is written in SOAP, after all. Anyway, their implementation of SOAP is the silliest I have seen so far. Every method takes a single parameter. It’s a string named ixml.

Can you guess yet what they are doing?

Yes, every parameter is a piece of xml. The actual structure of the xml message is not defined in the WSDL. In other words, the Web Service Description Language does not describe the web service. Good job, guys.

In all the years I’ve been using Linux just enough to get by, I’ve never stumbled across this handy tip. If you have a script that, say, needs root access, but must be executable by users who do not have root access, rather than setting up sudo and then making everyone have to remember to type sudo before they type the path to the script, you can do this:

chmod +s filename

The script will run as the owner, and the permissions for executing the script will be determined by group membership. If you have a script that must be executable by users in the ‘devs’ group, but it must run as root, all you have to do is:
chown root.devs filename
chmod +s filename
(assuming your file is already group executable)

Obviously this file should not be group writable or you’re giving everyone in that group the power to run anything as root, simply by modifying the script and then running it.

Bad design is everywhere, and it doesn’t have to be that way. I suppose that’s why I make such a fuss when one of our products or features seems to be poorly designed. I want to do better!

Here’s an example of bad design that isn’t our fault:
Everyone in the office has trouble using the coffee pot at one time or another. People get grounds in the water reservoir, overflow the basket, or get burned by steam when they open the lid. I have witnessed people do all of these things, and I have studied the design of the coffee pot. I understand how it works, but I still managed to overflow the basket myself, creating a horrible mess and (worse) delaying my caffeine intake. Why is this coffee pot so hard to use? My cheap coffee pot at home works just fine. I had no difficulties with the coffee pot at my previous job.

It’s the design. The basket and water reservoir are in the same “compartment”, covered by the same lid, and the component that stops the flow of coffee if the carafe has been removed (so you can pour a cup before it’s finished) is a mechanism that is half in the basket (spring loaded stopper) and half in the lid of the carafe (the shape of the lid pushes up the spring, allowing water to flow).

Every element of the design of this coffee pot is directly related to the ways that people manage to mess up their coffee:
If they are not careful about adding coffee to the basket, they will get grounds in the reservoir. The proper way to avoid that problem is to remove the basket, add the coffee and put it back in. The basket does not sit evenly on the counter because of the spring loaded mechanism on the bottom, so it discourages people from adding grounds this way.
Having one lid that covers the reservoir and the basket causes steam to be trapped in the coffee maker, so if people open the lid during the brewing process (say, because something else is messing up) or shortly afterwards (say, to clean up), they will have their fingers burned.
The spring loaded stopper does not stop the brewing process, only the dripping into the carafe process. If the stopper remains closed, water and grounds will eventually overflow the basket into the reservoir, into the carafe, and all over the place. The stopper remains closed whenever it is not in contact with the lid, so if someone accidentally leaves the lid off, or leaves the lid open “to make sure nothing gets in the way” (as one of our interns did), it’s an instant mess complete with ruined coffee.

Compare this to my coffee maker at home, the water reservoir is completely separate from the basket, which swings out on a hinge so that any spilled coffee grounds end up on the counter, not in the water. The mechanism that stops the drip process when the carafe is removed is a small lever that is depressed when the carafe is placed in the coffee maker. It stops the flow of hot water into the basket, thereby halting the brewing process whenever the carafe is removed.

This is an ideal design in every way, especially the lever to stop the brewing process. The only component required to let the water flow is the carafe, which catches the brew. If the carafe is not present, no coffee is brewed and no disaster ensues. Use of a lid is entirely optional.

So why is our coffee pot so horribly designed? It’s not like coffee pots are a new technology, and it’s not like they’ve just been this lousy all along, my coffee pot is older! They could have just copied the design if they couldn’t think of their own. I don’t see how the manufacture of my coffee pot style would be any more expensive than the one we have at Grooveshark. Mine was under $10, after all.

The only thing I can think of is that manufacturers just plain and simply do not care about usability for these essentially disposable appliances. Customers do not pay attention to these details until it’s too late, or they blame themselves for their inability to use them, so good design just does not matter.

But design does matter if your company is not satisfied with mediocrity. One need not look any further than Apple to see the result of putting a heavy emphasis on design and usability. The success of the iPod and iPhone are the direct result of that emphasis. If the iPod worked like a Sansa and the iPhone worked like my old Motorola phone, where would Apple be right now?