MySQL Conference Liveblogging: Benchmarking Tools (Wednesday 4:25PM)

Posted by Artem Russakovskii on April 16th, 2008 in Databases

  • Tom Hanlon of MySQL presents
  • Benchmarking tools
  • sql-bench
    • pros
      • ubiquitous
      • long history of use
    • cons
      • single thread
      • Perl
      • not always real-life test cases (create 10k tables?)
    • list of tests follows
  • supersmack
    • configurable, flexible
    • 1000 queries, 50 users
      • super-smack -d mysql select-key-smack 50 1000
    • can modify queries to be closer to what your own application uses
    • pros
      • benches concurrent connections
      • well documented
    • cons
      • test language sucks
  • Apache Bench
    • webserver benchmarking tool
    • point to a webserver, utilizes concurrent users
    • siege, httperf, httpload are similar
    • 404 errors deliver really quickly, so make sure to check for those
  • benchmark()
    • tests MySQL's internal functions
    • SELECT BENCHMARK(10000, SHA1(MD5(rand())));
    • pros
      • simple to use
    • cons
      • only benchmarks functions, can be overcome (see next)
    • creative use: write your own queries as functions and then run benchmark() on them
  • MyBench
    • simple
  • WAST
    • pros
      • flexible
      • easy to use (GUI)
      • replay example click trail from browsing
    • cons
      • Windows only
  • Jmeter
    • Java based app for testing web applications
    • hard to built tests but once you do, they run really well
    • configuration done using test plans, which is more involved than just typing a cmd line
    • can output tables, graphs
    • can be configured to run endlessly, as background noise
    • good documentation
    • flexible
  • mysqlslap
    • internally developed benchmark tool
    • ships with 5.1
    • feature rich
    • mysqlslap –user=john
    • –auto-generate-sql
      –query="select blabla …" or –query=file
    • –concurrency=100
    • –iterations=5
    • –engine=myisam
  • audience members additionally mention grinder, openSTA, yslow (ties in with FireBug), selenium
● ● ●
Artem Russakovskii is a San Francisco programmer and blogger. Follow Artem on Twitter (@ArtemR) or subscribe to the RSS feed.

In the meantime, if you found this article useful, feel free to buy me a cup of coffee below.

  • mp4todvd

    the benchmark() function is extremely useful for me. 🙂

  • Jamsheer Ak

    I tried to install onther benchmark super-smack-1.2
    while i am giving the make fuction i am geting the error message as following
    And also I tried to install onther benchmark super-smack-1.2,if am i am
    giveing the commnd ./configure it wll configure successfully and for make it is giving the following error
    dictionary.h: In member function ?void

    Unique_dictionary::set_template(const char*)?:
    dictionary.h:93: error: ?strlen? was not declared in this scope
    super-smack.cc: In function ?void set_parse_file(char*)?:
    super-smack.cc:65: warning: deprecated conversion from string constant to
    super-smack.cc: In function ?void set_db_type(const char*)?:
    super-smack.cc:72: warning: deprecated conversion from string constant to
    super-smack.cc: In function ?void print_engines()?:
    super-smack.cc:126: error: ?strlen? was not declared in this scope
    make[2]: *** [super-smack.o] Error 1
    make[2]:Leaving directory `/home/jamsheer/Downloadssuper-smack-1.2/src'
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/home/jamsheer/Downloads
    make: *** [all-recursive-am] Error 2
    Or can i use any other benchmarch in with i can perform various test of
    obuntu 9.04?
    Tr please give me the the clarification on which i am mention above so
    that i can perform the test either sybench or super-smack-1.2
    yours sincerely.


  • Got around to trying mysqlslap – it's great.