beer planet

I just got back from the StackOverflow's DevDays conference in the rainy (at least today) San Francisco.

I was really glad to see Joel Spolsky, Jeff Atwood, and the whole StackOverflow team in person, as well as listen to great talks in the following topics:

9:00 – 9:50    Joel Spolsky Opening Keynote
9:50…

Updated: June 1st, 2008

I think this is going to be really neat: you walk around the streets of San Francisco, for example, with your Android powered phone, en route to your destination 20 blocks away.

You whip out your phone, go to Google Maps, pull up the StreetView (remember this?), which zeroes in on your location using a built-in GPS, and then changes as you move the phone…

• http://en.oreilly.com/mysql2008/public/schedule/detail/874
• presented by Charles Lee of Hyperic
• Hyperic has the best performance with MySQL out of MySQL, Oracle, and Postgres in their application
• I suddenly remember hyperic was highly recommended above nagios in MySQL Conference Liveblogging: Monitoring Tools (Wednesday 5:15PM)
• performance bottleneck

• the database

• CPU
• memory

• disk latency
• network latency

…

• Damien Seguy of Nexen Services presents
• easiest session of all (phew, that's a relief)
• clever SQL recipes
• tweaking SQL queries
• shows an example where SELECT is ORDERED by a column that is actually an enum.

• an enum is both a string and a number
• sorted by number
• displayed as string
• can be sorted by string if it's cast as string

• compacts storage
• faster to

…

Updated: April 18th, 2008

• Tom Hanlon of MySQL presents
• monitoring tool basics
  • SHOW FULL PROCESSLIST
  • SHOW GLOBAL STATUS
  • SHOW GLOBAL VARIABLES
• basic tools
  • mysqladmin is provided with the server
    • mysqladmin -i 10 extended status: will repeat the same command every 10 seconds. Pipe through grep "and smoke it" (bad pun, hah hah)
    • -r: show only changed values
  • MySQL Administrator
• cacti
  • rrdtool based network

…

• Tom Hanlon of MySQL presents
• Benchmarking tools
  • mysqlslap (with MySQL 5.1)
  • sql-bench
  • supersmack – Jeremy Zawodny's tool
  • Apache Bench (combined with some sample PHP scripts)
  • MySQL's benchmark() function
  • mybench
  • WAST
  • JMeter
• sql-bench
  • pros
    • ubiquitous
    • long history of use
  • cons
    • single thread
    • Perl
    • not always real-life test cases (create 10k tables?)
  • list of tests follows
• supersmack
  • configurable, flexible

…

• Paul McCullagh presents
• BLOB

• invented by Jim Starkey
• Basic Large OBject
• Binary Large OBject
• photos, films, mp4 files, pdfs, etc

• mysql client send buffer -> receive buffer on the server (max_allowed_packet)
• streaming a BLOB

• continuous data stream
• stream BLOB data directly in and out of the database
• store BLOBs of any size (>4GB) in the database
• create a scalable back-end that

…

• Jay Pipes, Tobias Asplund
• Finding out the number of rows that would have been returned (MyISAM and InnoDB)

• SQL_CALC_FOUND_ROWS and FOUND_ROWS()
• COUNT(*)
• MEMORY table
• if query cache is on, then it makes no difference
• if it's off

• Memory MyISAM is fastest
• FOUND_ROWS() is slightly slower than count(*)

…

• Phil Hilderbrand of thePlatform for Media, Inc presents
• classic partitioning

• old school – union in the archive tables
• auto partitioning and partition pruning
• great for data warehousing
• query performance improved
• maintenance is clearly improved

• often id driven access vs date driven access
• 1 big clients could be 80% of the whole database, so

…

• Robert Hodges from Continuent presents
• About Continuent

• leading provider of open source database availability and scaling solutions

• uni/cluster – multi-master database clustering that replicates data across multiple databases and load balances reads
• uses "database virtualization"

• protection from db and site failures
• continuous operation during upgrades

…

• Suicide

• having no backups
• depending on slaves for backup
• keeping backups on same SAN
• having a single DBA – Frank didn't like this one at all
• not keeping binlogs

• how much time?
• uncompressed backup ready to mount?
• separate network for recovery?

• first problem: backup was highly compressed (tar.gz)
• uncompressing took hours
• so keep

…

Updated: April 17th, 2008

I had a chance to visit the Kickfire booth after the keynotes and before the first presentation. They gave me a kicking t-shirt, followed by a presentation on the newly announced Kickfire appliance (now in beta, shipping in Fall 2008). Here are some notes I jotted down:

• von Neumann bottleneck
• SQL chip (SQC), packs the power of 10s of conventional CPUs
• Query parallelization on the chip

…

• Baron Schwartz presents
• only works for SELECTs
• nobody dares admit if they've never seen EXPLAIN
• MySQL actually executes the query
• at each JOIN, instead of executing the query, it fills the EXPLAIN result set
• everything is a JOIN (even SELECT 1)
• Columns in EXPLAIN

• id: which SELECT the row belongs to
• select_type

• simple
• subquery
• derived
• union
• union result

…

• Robin Schumacher
• gives overview of MySQL products
• MySQL Enterprise
• MySQL 5.1 announced
  • table/index partitioning -> great for data warehouses, range, cache, key, list, composite, subpartitioning. Partition pruning. Response time greatly improved with proper partitioning.
  • row-based/hybrid replication -> safer and smarter
  • disk-based cluster -> supports bigger DBs
  • built-in job scheduler -> simplified task management
  • problem SQL identification -> easier troubleshooting. Dynamic query

…

Were there too many "my"'s in that title? Anyway… this week's MySQL conference is promising to be really busy and exciting. I can't wait to finally be there and experience it in all its glory. Thanks to the O'Reilly personal conference planner and scheduler and the advice of my fellow conference goers, I was able to easily (not really) pick out the speeches…