RSS
 

Archive for January, 2009

Indexes Matter (or: Memcache Will Only Take You So Far)

28 Jan

About a week ago, I was doing some work on the DB in the middle of the night and noticed that my simple queries were running a bit sluggish. I dropped out of mysql and ran top, and noticed that load averages were way higher than I was used to seeing. I ran SHOW FULL PROCESSLIST a bunch of times, and noticed two queries popping up frequently, one was a backend processing query which did not belong on the production database, and the other was the query used to build Widget objects. My first suspect was the backend process, since it did not belong, so we took that off and moved it to a more appropriate server, which brought down the load average by 1; a significant improvement, but the load averages were still pretty high, however the server was usable and responsive enough again, so I forgot about it.

A couple of days later, I noticed our load averages were still pretty high and the main recurring query was still the widget one, so I ran an explain on it, and although the query looked innocent enough, it was missing an index, so instead of a quick lookup it was a full table scan across millions of rows. Ouch.

I knew we wouldn’t have a chance to have some downtime to run the necessary ALTERs to get the indexes in there until after the weekend, so I asked Chanel to put in memcache support so that widgets would only need to be loaded once from SQL. Chanel got that done on Sunday, and on Monday night we were able to get the proper indexes added.

Because of the time span involved, combined with the fact that we monitor server metrics with Zabbix, means that we can look back at a nice little graph of our performance before and after each of the changes.

The days with the grey background are Saturday and Sunday, before memcache was added. The next day, with memcache added the peak load is cut in half. The day after that, with proper indexes, the peak load is barely perceptible, roughly 1/4 of what the load was with just memecache.

The lesson to be learned from this is that while memcache can help quite a bit, there’s a lot to be said for making sure your SQL queries are optimized.

 
1 Comment

Posted in grooveshark, SQL

 

Why I Drove a 19-Year-Old Car

23 Jan

Get Rich Slowly has a blog post entitled why I drive a 13-year-old car, which I recommend reading.

I followed a similar strategy until recently. I bought a 1989 Pontiac 6000 LE station wagon a few years ago, paid cash, somewhere between $600-800 for it, and vowed to drive it until it would not go anymore. It was hideously ugly, did not have heat or air conditioning, the electric windows sometimes stuck, and the electric locks would sometimes fail to unlock, but it was super cheap, and insurance was even cheaper. I was able to set aside $250/month as a “car payment” to myself.

When I got rid of that car, it wasn’t even because it needed repairs, it was just as junky as ever, no better, no worse. The problem was that I needed to be able to drive to Orlando frequently, and although the car was great for my in town driving, I didn’t feel entirely safe taking it on the highway that frequently, not to mention the relatively lousy gas mileage of an older heavy car. By then I had saved up enough to buy a “newish” car, but my need wasn’t urgent so I waited until I spotted a great deal. I found a solar yellow 2005 Scion xB that the owner had been having difficulty selling due to the nature of how ugly solar yellow is, but that type of ugly was still a few steps up from my old car, and I was able to talk them down to $2,000 below blue book.

I’m happy with my decision, it’s an extremely utilitarian car which gets very good gas mileage, and it’s a tolerable sort of ugly. I still put aside “car payments” even though I don’t have any, so I’ll be set if anything happens. I think the main thing to take away from the article is that you can save a lot of money by simply owning your car, so that’s my goal: to never have a car loan again. If you start out with a car you can pay cash for and make the same payments as you would otherwise, at the end you have the cash to pay for a new car (plus interest). If you do it the traditional way and buy a car with a loan, at the end you have an old car that may not be worth much anymore. If the circumstances allow it, I highly recommend buying no more car than you can afford with cash.

Then, when you have cash and wheels, you’ll have the ability to save even more money because you have more negotiating power with cash, and the ability walk away from any deal that’s not an awesome one.

 
No Comments

Posted in life

 

Sharing Firefox and Thunderbird profiles across a dual-boot

18 Jan

I’m currently trying to see if WinXP x64 will meet all of my hardware and software compatability needs so that I can fully utilize the 4GB of RAM I got for Christmas, so I am dual-booting between XP x32 and x64, with each install on a different drive. I set up Firefox and Thunderbird, but wanted access to my settings and more importantly, my saved passwords that I have configured on the x32 install.

I could have just copied the profiles over, but the profiles won’t stay in sync without a considerable amount of effort, if I am constantly switching back and forth. What if I bookmark something in x64? When I run FireFox on x32, I want that to show up in my bookmarks. Same goes for my extensions and any other settings. Turns out, it’s not very difficult to do, but it does take a little bit of work:

First boot into the newer of the two installs. In my case that’s XP x64.

Get ntfslink, a handy shell extension that lets you create and manage hard links and junction points through explorer. What we care about is the ability to create a junction point, which you can read about if you are truly interested. The general idea is that a junction point acts like a sort of shortcut, but it makes it look like the thing your shortcut is pointing to is actually there.

Find your original profile. Something like c:\Documents and Settings\username\Application Data\Mozilla\Firefox\profiles\somestring.default
Your new firefox appdata should be pretty similar. In my case it’s exactly the same except with the drive letter d: instead of c:, because that’s where I installed x64.

In d:\Documents and Settings\username\Application Data\Mozilla\Firefox\profiles\ (or wherever your newer OS install lives) create a NTFS junction point to c:\Documents and Settings\username\Application Data\Mozilla\Firefox\profiles\somestring.default (or wherever your older OS install lives) by navigating to the new profile directory, right-clicking in a blank space, and choosing New | NTFS Junction Point. Ntfslink names your junction “link to somestring.default” by default. Rename that to just somestring.default for simplicity.

For completeness, you should do the same thing between d:\Documents and Settings\username\Local Settings\Application Data\Mozilla\Firefox\profiles\ and c:\Documents and Settings\username\Local Settings\Application Data\Mozilla\Firefox\profiles\somestring.default

In d:\Documents and Settings\username\Application Data\Mozilla\Firefox\profiles\profiles.ini you should see something like this:
[Profile0]
Name=default
IsRelative=1
Path=Profiles/25wwypw7.default
Default=1
Copy that and paste it at the end of the file. Change Profile0 to Profile1 (assuming you only had one profile previously). Change Name=default to something more useful. In my case I went with Name=jay Change Path=Profiles/… to Path=Profiles/somestring.default

Save profiles.ini, and then start firefox from the run menu with firefox -p — this should prompt you for which profile you would like to use. Choose the name that matches what you set in the previous step, in my case “jay.”

If you followed these steps carefully then you should now have the same set of extensions, preferences and bookmarks that was on your old install, and changes made on one OS should carry across to the other.

The steps I spelled out here for Firefox profiles are identical for Thunderbird profiles, just substitute Thunderbird wherever you see the word Firefox.

 
9 Comments

Posted in Uncategorized

 

Grooveshark is growing!

14 Jan

I’m probably not at liberty to speak about specific numbers, but I have to share my elation at Grooveshark’s current growth rate.
Last month I predicted that given our current growth rate, we should double our total number of users every 3.6 months.

We are one day away from hitting a big round number, so I thought I’d look back and see how long ago we were at half of that. What do you know, 3.5 months ago. A growth rate slightly better than I had projected.

This, my friends, is exponential growth, and it turns out that when you have exponential growth it is insanely easy to calculate how long it will take to double your numbers. It’s called the rule of 72 and it basically states that you simply divide 72 by your growth rate to get your doubling time. For example, 72/(20% per month) = 3.6 months.

I must credit Dr. Albert Bartlett for teaching me about the rule of 72. According to Dr. Albert Bartlett, “the greatest shortcoming of the human race is our inability to understand the exponential function,” and I highly recommend watching his lecture on the topic. (sorry, it’s a .ram file. here is an alternate version on google video, which I haven’t watched)

 
1 Comment

Posted in grooveshark, life