beer planet

• Tom Hanlon of MySQL presents
• Benchmarking tools
  • mysqlslap (with MySQL 5.1)
  • sql-bench
  • supersmack – Jeremy Zawodny's tool
  • Apache Bench (combined with some sample PHP scripts)
  • MySQL's benchmark() function
  • mybench
  • WAST
  • JMeter
• sql-bench
  • pros
    • ubiquitous
    • long history of use
  • cons
    • single thread
    • Perl
    • not always real-life test cases (create 10k tables?)
  • list of tests follows
• supersmack
  • configurable, flexible
  • 1000 queries, 50 users
    • super-smack -d mysql select-key-smack 50 1000
  • can modify queries to be closer to what your own application uses
  • pros
    • benches concurrent connections
    • well documented
  • cons
    • test language sucks
• Apache Bench
  • webserver benchmarking tool
  • point to a webserver, utilizes concurrent users
  • siege, httperf, httpload are similar
  • 404 errors deliver really quickly, so make sure to check for those
• benchmark()
  • tests MySQL's internal functions
  • SELECT BENCHMARK(10000, SHA1(MD5(rand())));
  • pros
    • simple to use
  • cons
    • only benchmarks functions, can be overcome (see next)
  • creative use: write your own queries as functions and then run benchmark() on them
• MyBench
  • simple
• WAST
  • pros
    • flexible
    • easy to use (GUI)
    • replay example click trail from browsing
  • cons
    • Windows only
• Jmeter
  • Java based app for testing web applications
  • hard to built tests but once you do, they run really well
  • configuration done using test plans, which is more involved than just typing a cmd line
  • can output tables, graphs
  • can be configured to run endlessly, as background noise
  • good documentation
  • flexible
• mysqlslap
  • internally developed benchmark tool
  • ships with 5.1
  • feature rich
  • mysqlslap –user=john
  • –auto-generate-sql
    or
    –query="select blabla …" or –query=file
  • –concurrency=100
  • –iterations=5
  • –engine=myisam
• audience members additionally mention grinder, openSTA, yslow (ties in with FireBug), selenium

Artem Russakovskii is a San Francisco programmer, blogger, and future millionaire (that last part is in the works). Follow Artem on Twitter (@ArtemR) or subscribe to the RSS feed.

In the meantime, if you found this article useful, feel free to buy me a cup of coffee below.