Comments on: ORDER BY SLOW()

By: Ash

Ash — Thu, 30 Jun 2011 05:46:12 +0000

Robert, your solution always returns the same results whenever I execute the query. It’s only random once but not until a new row has been inserted. I prefer Gabe’s explanation and Frederick’s solution. Sorry for the grammar.

By: augusto

augusto — Thu, 31 Mar 2011 18:49:41 +0000

well done!
thanks for sharing

By: Jay

Jay — Thu, 23 Dec 2010 12:22:55 +0000

Robert, I don’t see how that solution could be fast unless you are dealing with small sets of data, it’s going to apply the md5 function for every row in the table, store all of those results, and then sort them. What does the EXPLAIN look like?

By: Robert Goodyear

Robert Goodyear — Thu, 23 Dec 2010 02:14:52 +0000

I wound up simply determining a column with non-sequential data in it, and then using ORDER BY md5(‘column’) LIMIT n

Was super fast, and subsequent pulls could add the offset method for LIMIT to grab more random chunks.

By: Matthew Montgomery

Matthew Montgomery — Wed, 28 Jan 2009 19:05:55 +0000

See: solutions at http://jan.kneschke.de/projects/mysql/order-by-rand/

By: Frederick R

Frederick R — Fri, 31 Oct 2008 11:30:26 +0000

It is different from ORDER BY RAND() in sense that you actually take only 10 rows given by mysql then order it by the randomize number ran.

Unlike ORDER BY RAND(), mysql will take all 6m tuples and randomize it.

One idea to improve the code would be to increase the samples and do another limit. Even better if you play with the result a little more…

SELECT * FROM (SELECT id,name,rand() as ran FROM table WHERE id like '345' OR name like 'exa' LIMIT 100) AS x ORDER BY x.ran LIMIT 10

Feeding SQL another random number or random string from your script will give an even more mixed-up result. Still a lot faster than RAND().

By: Jay

Jay — Fri, 31 Oct 2008 10:14:53 +0000

What that query will do is grab the first 10 rows in the table, and order them by rand. If you only want the first 10 rows to be considered, and your table is much larger, then yes this is faster. Otherwise you are just doing explicitly what MySQL does implicitly when you do an ORDER BY RAND().

By: Frederick R

Frederick R — Fri, 31 Oct 2008 06:41:45 +0000

Alternative random result; I think more flexible and faster result.

SELECT * FROM (SELECT id,name,rand() as ran FROM table LIMIT 10) AS x ORDER BY x.ran

Cheers!

By: Gabe da Silveira

Gabe da Silveira — Tue, 21 Oct 2008 01:27:14 +0000

A few notes:

First, if you are limiting the data set via a where clause, if the set is small enough, then ORDER BY RAND() may be fast enough for you.

Another thing to keep in mind is that:

SELECT id FROM table ORDER BY rand();

Can be orders of magnitude faster than:

SELECT * FROM table ORDER BY rand();

Due to the temporary table creation. Granted you probably will then need to do another query to pull the data, but…

Regarding multiple queries to the database, you should run benchmarks with your particular setup to find out if multiple queries will actually hurt you. In some cases what you’re trying to do can be more efficient in code than SQL even considering connection latency. On my setup even though the database is on a separate cluster, it is still very low latency and multiple queries have not been a problem.

With that in mind, I think the original fast solution you post is pretty good (and no less elegant to me than a weird subselect). When I started database programming 10 years ago I gravitated in the direction of as many efficient joins as possible to minimize queries. It worked pretty well, but after working on more complex applications I found the benefits of a nice ORM system and simple queries to ultimately be more beneficial for productivity without hurting performance (ie. knowing where to optimize).

Finally, another thing to watch out for in your solution is that in sparse tables the randomness can suffer severely. Imagine a table with rows 1-100. Delete 2-99 . Now you have a 99% chance of getting 100 and a 1% chance of getting 1.

By: Jay

Jay — Thu, 21 Aug 2008 06:42:23 +0000

Hi Andreas, unfortunately that’s where this solution falls apart. As soon as you’re dealing with a partial data set, there aren’t any elegant solutions to the problem (that I know of). If you do find something that works, is quick and elegant, please do share!