Manage Your Library

21 Aug

Coming to Grooveshark 2.0 is the ability for users to add music to their library without uploading, and without even scanning in their library.

That’s right, if you find a song on Grooveshark that you like, you can add it to your library with the simple click of a button. If you have a song in your library that you wish would just go away, you can also remove it from your library with the simple click of a button.


You can click this button on any song row to add it to your library (note: icon subject to change)

You can also drag as many songs to your library at once as you’d like.

If you toggle this button (icon subject to change) when it is highlighted, the song will be removed from your library.

Reminder: We will be releasing a preview (in other words, still buggy) version of Grooveshark 2.0 to VIP subscribers on the 24th, this upcoming Monday. Subsequent changes to Grooveshark 2.0 will depend on feedback we get from the VIP community, so we can’t predict when 2.0 will be available to the general public.


New RPC Server

20 Aug

Among the many backend enhancements coming together for Grooveshark 2.0 is a brand new RPC Server that I wrote. The format it follows is the same as the last incarnation, JSON, but it’s considerably more efficient.

First, a history lesson. Our first stab at RPC (codename tambourine) was SOAP. We chose SOAP because it did what we needed (theoretically), it was XML (so super easy for Flash to use), and it’s also super enterprisey, which appeals to certain people here, myself not included. We used NuSOAP because it supports WSDL generation, which is so annoying we did not ever, ever want to have to do that by hand. Using SOAP turned out to be a huge mistake (anyone familiar with SOAP probably could have told us that). Because it’s XML, it was super easy to break the service by returning data with, say, an unsupported character in it, or by having PHP accidentally output a newline character (say, by following the PEAR standard and having an extra newline at the end of every PHP file). Further, because browsers eat most error codes and don’t pass the codes or the accompanying data on to plugins, we had to hack NuSOAP to violate SOAP standards and return a 200 OK when generating SOAP Faults. On top of all that, and most importantly, it turns out that generating all of that XML is very expensive. So expensive that I could not test code locally on my machine before putting it on a server. So expensive that we actually got a huge performance boost by caching the generated XML in memcache.

After the SOAP fiasco, we wanted to try using JSON, because it’s much more terse and far less expensive to generate than XML, while still being human readable, which we value for debugging purposes. We looked at official JSON-RPC stuff, but we were spoiled by SOAP in one regard: support for headers. We got used to using headers to pass around status information that didn’t exactly belong to the method calls, and we didn’t want to give that up so I wrote my own JSON-RPC server (codenamed cowbell) modeled largely after how SOAP worked, but more efficient. My custom JSON server class was only 275 lines long to NuSOAP’s 7,994 lines for just the core file. We gave up the elaborate data type declarations (we just verbally communicated what type everything would be), but beyond that method registration worked the same way as SOAP and every other RPC implementation I’ve seen: you need a service file which registers all available methods with the RPC server, and then invokes the RPC class, passing through the raw POST data. Although I did not save benchmark information, the speed difference between my custom RPC class and NuSOAP was astronomical, so we left the RPC optimizations at that.

I recently came back and did some profiling out of curiosity to see how much overhead our current RPC implementation was creating, and it turned out to be about 15% plus about an extra 1.5MB of memory used. I thought about how things were working with the current implementation and realized that it’s a bit silly to be registering hundreds of methods every time the client makes a request, when you really only need information about whatever method is currently being requested. All the other information is superfluous. So in our new JSON-RPC implementation (codename more because it’s still cowbell in a way), 162 lines long, there is no ‘registration’ of methods. Instead, the service class that would normally do the registration simply has a collection of “meta methods” which return information about the methods that make up the service when called. So the RPC server parses the request, does a little bit of reflection magic to determine which meta method to call, and calls that method to get all the information it needs in order to handle the request. What’s the overhead of this approach? Well, it hardly even registers in the profiling I’ve done. :)

In summary, Grooveshark 2.0 method calls should be nearly 15% faster on average than Grooveshark 1.0 calls. :)

P.S. On the topic of cowbell, there’s an old easter egg that we added and never got around to telling anyone about. Before Grooveshark 2.0 is lost and it’s forgotten forever, you can get to it by typing about:cowbell into the search bar.


All /sorts/ of Improvements

19 Aug

I mentioned in Bypassing Magic that Grooveshark 2.0 will be loading much larger data sets, and one consequence of this is that the application can do more with that data.

As you might have guessed from the punny title of this post, one of those cool things is sorting! When looking at a list of songs, artists, albums, etc., in Grooveshark 2.0, you will be able to sort that data however you like.

Yes, this is another excuse for me to leak a screenshot. ;)

Grooveshark 2.0 Sorting

Another feature it enables which won’t make it into the VIP release of the 24th, but will be available to VIP users shortly thereafter, is live search. You’ll be able to narrow down any list of songs you’re looking at by searching within the result set.


Bypassing Magic

18 Aug

In my post about how we are adding client-side caching to Grooveshark 2.0, I mentioned one of the ways we are taking advantage of the fact that thanks to using Flash, we have a full-blown stateful application.

As Grooveshark evolves and the application becomes more sophisticated, the PHP layer is more and more becoming just an interface to the DB. The application just needs the data; it knows exactly what to do with it from there. It also only needs to ask for one type of data at a time, whereas a traditional webpage would need to load dozens of different pieces of information at the same time. So for our type of application, magic methods and ORM can really just get in the way when all we really need is to run a query, fill up an array with the results of that query, and return it.

Our old libraries employing ORM, magic methods and collections, were designed to meet the needs of a typical website and don’t necessarily make sense for a full-fledged application. On a webpage, you might only show 20 results at a time, so the overhead of having a bunch of getters and setters automatically fire whenever you load up your data is probably not noticeable. But in an application, you often load far more results than can be displayed, and allow the user to interact with them more richly. When you’re loading 500, or 5000 results as opposed to 20, the overhead of ORM and magic can start to really bog you down. I first noticed the overhead issue when testing new method calls for lite2, when in some cases fetching the data would take over 30 seconds, triggering my locally defined maximum execution time, even when the data was already cached.

Like any responsible developer considering making changes to code for performance reasons, I profiled our collections code using XDebug and KCachegrind, and then I rewrote the code to bypass collections, magic and all that stuff, loading data from the DB (or memcache) into an array and returning it. The difference? In the worst case, bypassing magic was an order of magnitude less work, often times far better than that. My > 30 second example took less than 1 second in the new code.

For Grooveshark 2.0 code, wherever makes sense, we are bypassing magic, ORM and collections and loading data directly. This of course means that Grooveshark is faster, but it also means that we can load more data at once. In most cases we can now afford to load up entire lists of songs without having to paginate the results, which in turn means fewer calls to the backend *and* much less work for the database. Whenever you must LIMIT results, you must also ORDER BY the results so they come back in an order that makes sense. Not having to ORDER results means in many cases we save an expensive filesort which often requires a temporary table in MySQL. Returning the full data set also allows the client to do more with the data, like decide how the results should actually be sorted and displayed to the user. But that’s another post…


Ghost Glacier

18 Aug

Some of my favorite music also happens to be made by some of my friends. :) Please enjoy Ghost Glacier.


Better Playlist Editing

16 Aug

Ugh, we know, in Grooveshark 1.0 it is a royal pain in the ass to edit playlists. For various reasons the playlist editing system in 1.0 is a hack on top of a hack. For Grooveshark 2.0 the playlist editing system has finally been rewritten. Aside from a brand new, far less annoying interface, the actual mechanisms for modifying a playlist are vastly improved.

In Grooveshark 1.0, the way most people edit a playlist is to load it up into their queue, most songs around in the queue, add new songs to the queue, etc., and then save the queue, overwriting their existing playlist. If you just want to change the order of your playlist, technically you can do so by opening the playlist page, and clicking the up or down arrow to move a song up or down one position at a time. That means if you have a song in position 10 and you want it to be the first song on the list, you have to click up 9 times.

In Grooveshark 2.0, playlists fully support drag and drop actions. You can still save your queue as a playlist if you like, but you can also edit a playlist by dragging songs into it, dragging songs to the trash to delete them, and moving songs up and down on the list by, you guessed it, dragging them!

Here’s a screenshot with drag and drop in action.
Grooveshark 2.0 Drag and Drop playlists
Click to see full size


Client-side Caching

15 Aug

One of the exciting features coming to Grooveshark 2.0 (yes, VIP only at first) starting on the 24th is client-side caching.

What does that mean?
We are finally taking advantage of the fact that because we use flash, we have a stateful application capable of remembering even dynamic data. In other words, if the client already knows something, it doesn’t have to ask the server again.

For our users this is exciting because navigating back and forth on pages they have already seen should be almost instantaneous. For the backend it’s exciting because the client now acts as another layer of cache in front of the database, and can keep data in memory even when it must be flushed from memcache. Example:
When a user edits their playlist, that data is deleted from memcache (not overwritten due to potential race conditions). The next time that playlist data is requested, it must be loaded from the database. In 2.0, the chances of the client asking for that playlist data again are very slim, because in most cases it will remain in memory, so we should still be saved a round trip to the database in most cases.

In the slightly longer term (before Grooveshark 2.0 is available to the public), we will be taking advantage of Flash LSOs to even remember certain data between reloads, so things like browsing your library will be lightning fast and won’t require loading data from the server at all, unless you changed your library from another computer.

Client-side caching is just one of the many ways in which we are working to improve the user experience while hopefully reducing load on our servers, and I will be posting more details over the next few days, so stay tuned.


2.0 Screenshots

14 Aug

So far the VIP launch has been a resounding success, but as I mentioned previously, the real gravy for VIP users is yet to come: the ability to try out 2.0 first.

The official date that 2.0 becomes available for VIP users is the 24th. Here’s some of what they have to look forward to:

Select from themes
Choose from cool themes. My favorite is Greenify.

New home screen
Here you can see the home screen, redesigned player and brand new sidebar in action. You’ll notice the little dot on the player. Yes, you can click and drag that to skip ahead and back while a song is playing (finally)!

Song list view
Lists of songs are redesigned to be faster and more usable at the same time. No more slidey panels and awkward navigation heirarchies! Also notice that in 2.0 you can favorite artists and albums in addition to songs.


However you succeed, you will die

13 Aug

Mark Cuban recently wrote an article titled When you succeed with Free, you are going to die by Free.

Lets look at the rule that eventually KILLS all freemium based content plays:

There will always be a company that replaces you. At some point your BlackSwan competitor will appear and they will kick your ass. Their product will be better or more interesting or just better marketed than yours, and it also will be free. They will be Facebook to your Myspace, or Myspace to your Friendster or Google to your Yahoo. You get the point. Someone out there with a better idea will raise a bunch of money, give it away for free, build scale and charge less to reach the audience. Or will be differentiated enough, and important enough to the audience to maybe even charge more. Who knows. But they will kick your ass and you will be in trouble.

What’s the difference in this equation between free and non-free services? Does being non-free insulate one from competition?

Let’s imagine a world where search isn’t free. Google search is so good that they have killed off all of their competitors, and they decide to start charging $5 a month for everyone to use it. People pay because hey, Google always finds what they want and what else can they do? In this very imaginary scenario, Google’s black swan would be any other remotely decent search technology provided that it is offered for free. The point of this extreme example is that Google is more insulated from competition by already being free, so the above argument is silly.

The rest of the article goes on to say that no matter what type of service you offer, you should be on the lookout for your black swan and when it arrives, assuming you can’t purchase it, you should take all the money you’ve earned and run, rather than stay and fight an expensive losing battle. (I’m oversimplifying quite a bit, mostly because this isn’t the part that I care about)

That’s…fine, although I don’t necessarily agree on that point either, but Mark attacks some of the commenters in his blog for picking on the “free” aspect of his blog post like I am doing here because they miss the point at the end about how cutting and running. Perhaps if he had stuck with his original intended title “The Freemium Company Lifecycle Challenge” he wouldn’t have caught so many peoples attention and it would have just blown over. But if he really didn’t want these disputes popping up, a better title would have been “Everyone has a Black Swan, the trick is recognizing and reacting,” or something, you know, actually related to the true premise of the article. ;)


VIP live!

12 Aug

Grooveshark has launched VIP subscriptions.

Sign up for $3/month or $30/year.
In exchange you get:
-No ads ever (and more screen real estate because of it)
-Access to the latest and greatest features before anyone else.
-More to be announced later
-Locked in at the low rate. If the price goes up, your cost doesn’t.
-A badge next to your name that says you’re VIP.

On the 24th we will be launching a beta version of 2.0 to our VIP users, and trust me, that’s where the real value lies.

Grooveshark Launches Subscription VIP Service