For example, my regular PHP memory allocation is 32MB or so, but if I load up Windows Live Writer, my favorite publishing tool, and ask it to load 1000 of the latest blog posts, I will undoubtedly get a server error back.

One solution would be to increase the memory allocated to PHP to something higher, like 256MB, which is how I used to get around the issue. I say "get around" because it's not a good solution – if someone finds a page that uses a lot of memory on your site, they could easily kill not only your web server but the whole machine due to swap death. Keeping a lower memory limit allows you to run your web server, such as Apache, with more children, thus serving more requests without getting overloaded.

So, I've looked into the WordPress core and came up with what I think is a proper fix – dynamic memory limit tweaking when dealing with XML-RPC only. Here is the code – add it to your functions.php and you should be golden:

1
2
3
4
5
6
7
8
9
10
11

/**
* Dynamically increase allowed memory limit for XML-RPC only.
*
* @param array $methods
* @return array
*/
function my_xmlrpc_methods($methods) {
  ini_set('memory_limit', '256M');
  return $methods;
}
add_action('xmlrpc_methods', 'my_xmlrpc_methods');

The code is pretty self-explanatory: it hooks into a WordPress hook that fires only for XML-RPC requests and adjusts the memory limit on the fly. No more out of memory errors and I'm able to load 1000 posts in Windows Live Writer without problems.

• How To: Disable/Block External HTTP Requests In WordPress
• Follow-up To Loading CSS And JS Conditionally
• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• How To Add Shortcode Support To WordPress Comments And Sidebar Widgets
• [Solr] How To Fix java.io.IOException: directory FOO exists and is a directory, but cannot be listed: list() returned null

Introduction

Whether you are a web developer or a self-hosting business owner, the only excuse for not activating compression capabilities of your web server can be that you didn't know about it. And now that you are reading this, there is no excuse left at all.

Here is how big a single page of this blog was before compression was enabled on CSS and Javascript files (computed by YSlow):

And here it is after compression:

As you see, the difference is quite substantial – almost 30% savings.

Compressing your HTML, XML, Javascript, CSS, etc pages will mean less data transferred between the server and the client which:

• reduces the bandwidth usage.
• provides faster page rendering which in turn leads to less user frustration, higher conversion rates, lower bounce rate, etc etc etc.

Compression is especially important for users with slow connections as every kilobyte of your code is that much more painful to them.

Compression can be very effective – you can easily shrink your text, code (HTML, XML, Javascript, CSS, etc) to 10% of the original size (of course, your mileage may vary). 100KB page that needs only 10KB to transfer? Sign me up!

So, before I talk about the solution, let me describe what exactly happens when compression is turned on and how it affects older browsers that don't support it.

Compression Mechanism Explained

In order for compression to work in the first place, the web server (Apache in my example) needs to support it. This is achieved by enabling one of Apache modules called mod_deflate. The server will then be able to compress the data to the DEFLATE standard using either the zlib (also known as deflate) or gzip implementations. Yeah, DEFLATE is both the standard the one of its implementations, for those confused. I know I was. This is best described in this Wikipedia article.

The following mechanism is used:

• the server with a compression extension enabled is able to serve either compressed (smaller) or uncompressed (larger) pages, depending on what the client supports.
• the client (that is, your browser) sends a special header called "Accept-Encoding" listing the DEFLATE implementations it's capable of decompressing. For example "gzip,deflate".
• the server picks the best compression supported by the client (if any), compresses the files, and sends them over to the client.
• the client receives the compressed files and decompresses them.

Are Your Pages Already Compressed? Test Them!

If you are not sure whether you are already serving compressed pages or not, test them! My favorite way is by using Charles HTTP Debugger. Another option is by downloading Firebug for Firefox and installing Yahoo's YSlow or Google's Page Speed. Just look at the response headers to see if compression is on (look for the Content-Encoding header). Here are some before and after examples:

Theme CSS

Before:

After:

jQuery

Before:

After:

Solution

Create a .htaccess file in the top directory of your site with the following contents:

1
2
3
4
5
6
7
8

# DEFLATE by type - html, text, css, xml
AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml
 
# DEFLATE by type - javascript
AddOutputFilterByType DEFLATE application/x-javascript application/javascript text/javascript text/x-js text/x-javascript
 
# DEFLATE by extension
AddOutputFilter DEFLATE js css htm html xml

Alternatively, you could put these lines into your Apache config within the Directory directive.

The AddOutputFilterByType directive adds DEFLATE filters to certain MIME types. I tried to assemble some of the common ones but feel free to add more, as each server may be configured differently and give out MIME types different from mine.

The AddOutputFilter directive binds the DEFLATE filter to specific file extensions, just in case they are not served with a proper MIME type. Feel free to add to this list as well.

Caveats

1. In order to use this whole compression/deflate/gzip business, your Apache server must first have mod_deflate enabled. Without it, you will get the HTTP 500 error (Internal Server error). You can check which mods you already have enabled by checking with the output of phpinfo() function.

In order to enable mod_deflate, uncomment the line with "deflate_module" in your Apache config file. The location of this config file is highly dependant on your system. Some examples include

• /etc/apache2/httpd.conf
• /etc/httpd/conf/httpd.conf
• c:\wamp\bin\apache\Apache2.2.11\conf\httpd.conf
• some other place where your system stores Apache config files (read the special note below for OpenSUSE).

Here's what you should have:

LoadModule deflate_module modules/mod_deflate.so

On OpenSUSE, you actually enable modules a bit differently. Go to /etc/sysconfig/apache2 and look for APACHE_MODULES=. Then add "deflate" to the list, if it's not already there.

Now, restart Apache and check the output of phpinfo() again.

2. Adding AddOutputFilter and AddOutputFilterByType to .htaccess requires such overrides to be authorized by the main Apache configuration for that directory, otherwise it will return error 500 as well. The option you are looking for is called "AllowOverride" and mine was set to "AllowOverride AuthConfig" which wasn't enough. Changing it to

AllowOverride AuthConfig FileInfo

or just

AllowOverride All

fixes the problem. You can find more info about AllowOverride here.

3. In WordPress, if you are using Google Gears (Turbo mode) for caching some core WordPress files, they will not show up compressed. That is because they're not served by the remote server but rather reside locally (think of it as permanent cache). I was very confused at first when I didn't see jQuery.js in the HTTP log and YSlow reported it uncompressed.

Are you a WordPress user?

If you are a WordPress user, don't assume WordPress is going to automatically compress your pages. In fact, as you install more and more plugins, the payload becomes larger and larger with those additional CSS and Javascript files.

You owe it to yourself and to your users to immediately enable compression on your blog.

Here is what happened after I enabled compression on this blog.

Before:

After:

Bonus – WP Minify

For even better results, I suggest you have a look at my good friend and talented WordPress master Thaya's plugin called WP Minify. It preprocesses and aggregates all or most of your CSS and Javascript into just 2 files, thus saving on the number of HTTP requests. It also minifies content to achieve smaller size.

My blog before WP Minify:

After WP Minify:

That's all folks. Let me know if something was unclear and I'll be glad to clarify it.

A few references that pointed me in the right direction and allowed me to provide a more complete solution:

• http://brightscape.net/compress-jquery-even-further/
• http://brightscape.net/compress-your-web-pages-with-mod_deflate/

• Beer Planet Upgraded To WordPress 2.5
• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• Hidden Features Of Perl, PHP, Javascript, C, C++, C#, Java, Ruby, Python, And Others [Collection Of Incredibly Useful Lists]
• Rendr – a Live Awesome CSS/HTML Rendering Tool
• How To View A Specific SVN Revision In Your Browser

1
2
3
4
5
6
7

rcapache2 restart
/usr/sbin/httpd2-prefork: symbol lookup error: /usr/sbin/httpd2-prefork: undefined symbol: apr_ldap_ssl_init
Starting httpd2 (prefork) /usr/sbin/httpd2-prefork: symbol lookup error: /usr/sbin/httpd2-prefork: undefined symbol: apr_ldap_ssl_init
 
The command line was:
/usr/sbin/httpd2-prefork -f /etc/apache2/httpd.conf -DSSL
                                                                      failed

So I tried reinstalling libapr and apache2-utils related stuff with no luck. ldconfig didn't help either. It's not until I looked at /usr/lib and relinked a few things that it started working. What the hell, SUSE?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

/usr/lib # l libapr*
-rw-r--r-- 1 root root 239306 2007-09-21 23:41 libapr-1.a
-rw-r--r-- 1 root root    821 2007-09-21 23:41 libapr-1.la
lrwxrwxrwx 1 root root     17 2008-07-30 00:21 libapr-1.so -> libapr-1.so.0.2.9*
lrwxrwxrwx 1 root root     18 2008-07-30 00:21 libapr-1.so.0 -> libapr-1.so.0.2.12*
-rwxr-xr-x 1 root root 543841 2008-07-29 23:47 libapr-1.so.0.2.12*
-rwxr-xr-x 1 root root 148372 2007-09-21 23:41 libapr-1.so.0.2.9*
-rw-r--r-- 1 root root 156300 2007-09-23 13:33 libaprutil-1.a
-rw-r--r-- 1 root root    912 2007-09-23 13:33 libaprutil-1.la
lrwxrwxrwx 1 root root     21 2008-07-30 00:22 libaprutil-1.so -> libaprutil-1.so.0.2.9*
lrwxrwxrwx 1 root root     22 2008-07-30 00:21 libaprutil-1.so.0 -> libaprutil-1.so.0.2.12*
-rwxr-xr-x 1 root root 291520 2008-07-29 23:47 libaprutil-1.so.0.2.12*
-rwxr-xr-x 1 root root  98148 2007-09-23 13:33 libaprutil-1.so.0.2.9*
 
/usr/lib # ln -sf libaprutil-1.so.0.2.9 libaprutil-1.so.0
/usr/lib # ln -sf libapr-1.so.0.2.9 libapr-1.so.0
 
/usr/lib # l libapr*
-rw-r--r-- 1 root root 239306 2007-09-21 23:41 libapr-1.a
-rw-r--r-- 1 root root    821 2007-09-21 23:41 libapr-1.la
lrwxrwxrwx 1 root root     18 2008-07-30 00:34 libapr-1.so -> libapr-1.so.0.2.9*
lrwxrwxrwx 1 root root     18 2008-07-30 00:21 libapr-1.so.0 -> libapr-1.so.0.2.9*
-rwxr-xr-x 1 root root 543841 2008-07-29 23:47 libapr-1.so.0.2.12*
-rwxr-xr-x 1 root root 148372 2007-09-21 23:41 libapr-1.so.0.2.9*
-rw-r--r-- 1 root root 156300 2007-09-23 13:33 libaprutil-1.a
-rw-r--r-- 1 root root    912 2007-09-23 13:33 libaprutil-1.la
lrwxrwxrwx 1 root root     21 2008-07-30 00:35 libaprutil-1.so -> libaprutil-1.so.0.2.9*
lrwxrwxrwx 1 root root     21 2008-07-30 00:35 libaprutil-1.so.0 -> libaprutil-1.so.0.2.9*
-rwxr-xr-x 1 root root 291520 2008-07-29 23:47 libaprutil-1.so.0.2.12*
-rwxr-xr-x 1 root root  98148 2007-09-23 13:33 libaprutil-1.so.0.2.9*
 
/usr/lib # rcapache2 restart
Syntax OK
Shutting down httpd2 (waiting for all children to terminate)          done
Starting httpd2 (prefork)                                             done

Voila, apache starts now. Grrr, why linker, why??

• How To Make Your Site Lightning Fast* By Compressing (deflate/gzip) Your HTML, Javascript, CSS, XML, etc In Apache
• How To Resize/Grow VMware Linux Disks and Partitions
• How To Install The Latest SOAP::Lite Using Perl CPAN
• Hadoop Primer – Yet Another Hadoop Introduction
• How To View A Specific SVN Revision In Your Browser