The Problem

I wanted to export all the data from WordPress using its native export mechanism (located at http://YOURBLOG/wp-admin/export.php), but since the blog I was working on was pretty large (6k posts, 120k comments), I kept getting XML files that ended prematurely and for which xmllint spit out this error:

Premature end of data in tag channel

Upon closer inspection, I saw the XML file ended with a random, yet always fully closed, </item> tag, but was missing the closing </channel> and </rss> tags, as well as a whole bunch of data.

My immediate theory was that PHP was running out of memory. It's also interesting that the number of <item> elements was always divisible by 20, but after looking at the code in export.php, I saw the loop grabbed 20 posts at a time. This revelation made it obvious that the code crashed while processing one of the batches, which only made the out-of-memory theory stronger.

The Solution

After raising the memory in the export.php file itself and verifying the fix, I came up with the following solution that removes the need to modify core WordPress files. Just add this somewhere in functions.php:

1
2
3
4
5
6
7
8

/**
* Dynamically increase allowed memory limit for export.
*
*/
function my_export_wp() {
  ini_set('memory_limit', '1024M');
}
add_action('export_wp', 'my_export_wp');

The code above will dynamically set the memory limit high enough for all but unimaginably large jobs to complete. Feel free to adjust this limit if you have, for instance, millions of comments and lots of RAM on the web server.

And there you have it – full exports. Much better, isn't it?

P.S. Some shared hosting providers have PHP set to ignore the directive above, in which case this solution, or no other solution but upgrading your hosting, will do anything.

P.P.S. There is a bug (#15203) in WordPress <=3.3.1, which doesn't properly escape posts that contain CDATA and breaks XML. It is slated for a fix in WordPress 3.4, but if you haven't upgraded yet (it's not even out at the time of this writing), you can still apply the fix manually, like so. My dumps wouldn't validate before the fix, but I've confirmed that they now fully validate after.

• How To Dynamically Increase Memory Limits When Interfacing With WordPress Using XML-RPC (Windows Live Writer, Etc)
• How To Remove Inline Hardcoded Recent Comments Sidebar Widget Style From Your WordPress Theme
• How To Add Shortcode Support To WordPress Comments And Sidebar Widgets
• Follow-up To Loading CSS And JS Conditionally
• [Android] Auto Formatting Android XML Files With Eclipse

For example, my regular PHP memory allocation is 32MB or so, but if I load up Windows Live Writer, my favorite publishing tool, and ask it to load 1000 of the latest blog posts, I will undoubtedly get a server error back.

One solution would be to increase the memory allocated to PHP to something higher, like 256MB, which is how I used to get around the issue. I say "get around" because it's not a good solution – if someone finds a page that uses a lot of memory on your site, they could easily kill not only your web server but the whole machine due to swap death. Keeping a lower memory limit allows you to run your web server, such as Apache, with more children, thus serving more requests without getting overloaded.

So, I've looked into the WordPress core and came up with what I think is a proper fix – dynamic memory limit tweaking when dealing with XML-RPC only. Here is the code – add it to your functions.php and you should be golden:

1
2
3
4
5
6
7
8
9
10
11

/**
* Dynamically increase allowed memory limit for XML-RPC only.
*
* @param array $methods
* @return array
*/
function my_xmlrpc_methods($methods) {
  ini_set('memory_limit', '256M');
  return $methods;
}
add_action('xmlrpc_methods', 'my_xmlrpc_methods');

The code is pretty self-explanatory: it hooks into a WordPress hook that fires only for XML-RPC requests and adjusts the memory limit on the fly. No more out of memory errors and I'm able to load 1000 posts in Windows Live Writer without problems.

• How To Fix Incomplete WordPress (WXR) Exports
• How To: Disable/Block External HTTP Requests In WordPress
• Follow-up To Loading CSS And JS Conditionally
• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• How To Add Shortcode Support To WordPress Comments And Sidebar Widgets

One glaring omission in the way shortcodes are set up by default is that they only get triggered in the content of your post, leaving the sidebar and comments out. I'm sure this is done for security, so that your readers can't screw something up by posting shortcodes they're not supposed to – after all, shortcodes are PHP snippets on the backend.

However, let's assume you really know what you're doing and you want, for example, to allow certain shortcodes in the comments and all shortcodes in the sidebar. It's your site and you should be able to do whatever you want with it. For instance, I wanted to provide a way to include QR codes using a [qr] shortcode syntax that I'd written both in the sidebar and in the comments, except in the comments that was the only shortcode I wanted to allow my readers to use.

Since these cases can get complicated, let's start simple and then progressively get deeper.

Allow all shortcodes in the sidebar

Assuming you want to enable shortcodes in the PHP Code Widget widgets (I believe this is the most common way to add PHP support to the sidebar), use this code:

1

add_filter('widget_execphp', 'do_shortcode');

If you have some other widgets in mind, you have to find out the right filter and use it just like above.

Allow only whitelisted shortcodes in the comments

Since I don't want all shortcodes to fire here, things get slightly trickier.

1
2
3
4
5
6
7
8
9
10
11
12

function init_common_shortcodes() {
  add_shortcode('qr', 'my_shortcode_qr');
}
 
function init_comment_shortcodes() {
  remove_all_shortcodes();
  init_common_shortcodes();
  add_filter('comment_text', 'do_shortcode');
}
 
init_common_shortcodes();
add_filter('comments_template', 'init_comment_shortcodes');

The code above will remove all shortcodes when it gets to comments and reinitialize only the whitelisted one called "[qr]", which is exactly what I wanted.

Allow all shortcodes in the sidebar AND only whitelisted ones in the comments

At this point, things get slightly tricky because if I combine the previous 2 solutions into 1, by the time WordPress gets to the sidebar, it'll be left without any shortcodes at all – they've been removed by the code dealing with comments. Therefore, I need to save the shortcodes before dealing with comments and then restore them after the comments and before the sidebar.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

function my_remove_all_shortcodes() {
  global $shortcode_tags;
  global $temp_shortcode_tags;
  $temp_shortcode_tags = $shortcode_tags;
  remove_all_shortcodes();
}
 
function my_restore_all_shortcodes() {
  global $shortcode_tags;
  global $temp_shortcode_tags;
  if(!empty($temp_shortcode_tags)) {
    $shortcode_tags = $temp_shortcode_tags;
  }
}
 
function init_common_shortcodes() {
  add_shortcode('qr', 'my_shortcode_qr');
}
 
function init_comment_shortcodes() {
  my_remove_all_shortcodes();
  init_common_shortcodes();
  add_filter('comment_text', 'do_shortcode');
}
 
add_filter('comments_template', 'init_comment_shortcodes');
add_filter('dynamic_sidebar', 'my_restore_all_shortcodes');
add_filter('widget_execphp', 'do_shortcode');
init_common_shortcodes();

Here is what the code does:

1. Initialize custom shortcodes that I want to add all over the place.
2. When WP gets to the comment block, it removes all shortcodes (again, for security) and re-initializes only the ones I want, but this time remembers them so that they can be restored later.
3. After the comments, when WP gets to the dynamic sidebar, it restores all the shortcodes and applies them to each PHP Code Widget.

And there you have it. Of course, the code may need tweaking for your own scenario but the ideas are all there. If you spot an error or know of an easier/better way, please let me know.

• How To Remove Inline Hardcoded Recent Comments Sidebar Widget Style From Your WordPress Theme
• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• How To Fix Incomplete WordPress (WXR) Exports
• How To Diagnose And Fix Incorrect Post Comment Counts In WordPress
• How To: Disable/Block External HTTP Requests In WordPress

One of the biggest problems with Eclipse is that there is no clear upgrade path from major versions, for example 3.5->3.6. What I ended up having to do for years is back up the old release, download and unpack the new release, and then try to migrate all the settings by importing and exporting left and right. Not so pleasant.

Turns out, as of Eclipse 3.3 (though I've only tried it with Eclipse 3.5), it's possible to successfully complete an in-place upgrade without doing any wiping or voodoo hackery. I've successfully completed this on 2 of my machines and now need to share it with the rest of the world.

Step 1. Go to Help > Install New Software… > Available Software Sites or Windows > Preferences > Install/Update > Available Software Sites. I personally prefer the former way because it results in fewer clicks but both end up in the same place.

Step 2. Everywhere you see 3.5, press Edit and change it to 3.6. Similarly, change "galileo" to "helios". For example, you should end up with at least these (if you don't have these, just add them):

http://download.eclipse.org/eclipse/updates/3.6

http://download.eclipse.org/releases/helios

Note: you may also need to update other repositories (for example, Galileo was compatible with CDT 6, while Helios needed CDT 7).

Step 3. Press OK, then go to Help > Check for Updates. You should see the progress in the Progress tab – let the magic happen, restart when prompted, and after a few minutes, you should have a working upgraded version of Eclipse.

• [Android] Auto Formatting Android XML Files With Eclipse
• How To Download And Install Windows Media Player 11 Bypassing WGA
• Supercharge Your GNU Screen With A Power "Taskbar" And Never Feel Lost Again
• How To Fix ERROR_NOT_FOUND 0×80070490 During Windows 7 SP1 Installation
• [Updated for 2012] Here's An Exclusive 10% Off NuSphere PHPEd Discount Coupon Code (Also Includes NuCoder And PHPDoc)

Introduction

I don't know about you, but I can't imagine doing my PHP development without an IDE with a debugger anymore.

It autocompletes for me, it lets me step through each line of code, jumping around the project, execute and change the code flow on the fly, and does many other things that make me feel cozy, comfortable, and efficient at PHP development (as opposed to, say, CPP which makes me feel cold and lonely).

There are many PHP IDEs out there and I've tried most of them (including the free PHPEclipse and PDT for Eclipse) but kept coming back to NuSphere's PHPEd every time. The other ones just don't do as good of a job and don't have the array of features PHPEd does. Did you know Dmitri, the author of PHPEd, actually wrote the PHP debugger and profiler for PHPEd from scratch?

But this post is not about PHPEd's features – I'm planning on doing a few detailed ones later on – this post is a quick note about a coupon code for 10% off PHPEd or any of their other products.

The Coupon

Here it is for your immediate enjoyment:

Just add the products you want to your cart and enter the coupon in the Coupon code field:

Observe the discount applied immediately:

Edit: it seems that they're running some sort of a promotion – PHPEd Pro temporarily has an additional $80 off the regular price. The discount was set to expire on January 15th but it seems to be still active:

The 10% off should work on top of the $80 discount no problem.

Enjoy!

• Hidden Features Of Perl, PHP, Javascript, C, C++, C#, Java, Ruby, Python, And Others [Collection Of Incredibly Useful Lists]
• How To Make Firebug's JavaScript Debugger Break Inside Dynamic JavaScript Using The 'debugger' Keyword (IE & Chrome Too)
• WordPress Developers – How Do You Make A Living [Poll + Discussion]?
• How To Update Eclipse From Galileo (3.5) To Helios (3.6) In-Place Without Reinstalling
• How To Check If The Local SVN Revision Is Up-To-Date

Here are some discussions and mentions around the web:

• an article on Weblog Tools Collection
• a forum post on WP Tavern
• twitter retweets

Sure, there are drawbacks to this method and it does require some more processing on the backend and it's not for everyone, which is why we should always strive for an even better solution.

I stand by my point of view that, for instance, my dedicated gallery shouldn't load for people who will never even go see my photos.

I think an ideal solution would be for WP core developers, who had a lot of experience designing WordPress' internals and who know what can work and what can't, perhaps Matt included, to get together and think about a better solution really hard.

Conditional loading similar to the one discussed here is already possible in the admin area, which creates dynamic hooks by appending page ids to 'admin_print_styles' and 'admin_print_scripts'. It is, however, still not good enough to be used more generically because those hooks rely on the page you are on, rather than the content of that page.

Another possibility is using a PHP technique called output buffering (ob_start(), ob_flush(), etc) that grants second chances to data that had already been echoed. It intercepts all echo and print calls and redirects them into a memory buffer, so instead of printing data to the page right away, you can now modify the header even if it had already been processed. It's like giving it a second chance.

Would it work for WordPress? I am not sure but I sure could use your feedback, devs.

• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• Swapping Column Values in MySQL
• How To Dynamically Increase Memory Limits When Interfacing With WordPress Using XML-RPC (Windows Live Writer, Etc)
• How To Remove Inline Hardcoded Recent Comments Sidebar Widget Style From Your WordPress Theme
• How To Fix Incomplete WordPress (WXR) Exports

It's always important to know for developers what browsers they are developing for, who dominates the market, and what the current trends are.

I have gotten my hands on the Plaxo.com visitors' browser stats for December of 2009.

This information is valuable because Plaxo has a relatively general demographics, as it's not a site only geeks or only moms visit, and the statistics tends to not be skewed. Therefore, as you can see, Firefox doesn't occupy the same share as you might see on a techy site (on this site, more than 50% of users visit in Firefox).

Also, since Plaxo has a couple million monthly visitors and therefore a couple million data points, statistically speaking these numbers are relatively accurate.

Without further ado, here is the data, with some spiffy charts.

Overall Browser Breakdown For December 2009

Internet Explorer Version Breakdown For December 2009

Some numbers here have a value of 0.00% which means they've been rounded down and have really low numbers. The order is still correct though.

Firefox Version Breakdown For December 2009

Thoughts

• Internet Explorer is doing remarkably well – it is still most definitely the primary browser, so testing it should be planned accordingly.
• Internet Explorer 7 is by far the most popular version with 65.9% of all IE versions, followed by IE6 at 17.2% (wtf??), and only then IE8 at 16.9%. This definitely came as a surprise to me – both in how popular IE7 still is and how pathetic Microsoft's efforts to upgrade Windows users up from IE6 have been.
• Safari and Chrome are neck and neck at 6.20% and 4.6% respectively, with Chrome gaining rapidly.
• Opera is doing undeservedly badly – 0.6%. That makes me sad.
• IE 999.1? Interesting. I wonder if it's some sort of an uber hacked version.
• And, instead of conclusion: damn you, IE6! Why can't you die already?

• Firefox Being Slow, Especially Switching Tabs, High CPU Load, Memory Problems? Are You Using Firecookie For Firebug?
• [Updated x3] Goodbye Outlook, I Don't Need You Anymore – Gmail Now Lets You Paste Images Directly From Clipboard
• Meet Firefox For Mobile [Video + Feature Highlights + More Info]
• Modern-Day Frame Busting With X-FRAME-OPTIONS And "This content cannot be displayed in a frame" Warnings
• Converting from CVS to SVN: Developer's Notes And Why SVN Is Better

The Problem

I've had MySQL on my Windows 7 laptop for a bit (as part of wampserver), mostly for local offline WordPress development.

However, even though MySQL is relatively stable, I've been observing a vast quantity of intermittent MySQL errors, as reported by WordPress in the PHP error log (C:\wamp\logs\php_error.log). Here are some examples:

[05-Jan-2010 09:47:51] WordPress database error Error on delete of
'C:\Windows\TEMP\#sql17e0_1a2_6.MYD' (Errcode: 13) for query SELECT t.*, tt.*
FROM wp_terms AS t INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id
INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id =
tt.term_taxonomy_id WHERE tt.taxonomy IN ('category') AND tr.object_id IN (3)
ORDER BY t.name ASC made by require, require_once, include, get_footer,
locate_template, load_template, require_once, dynamic_sidebar,
call_user_func_array, widget_rrm_recent_posts, RecentPosts->execute,
ppl_expand_template, otf_categorylinks, get_the_category, wp_get_object_terms
 
[05-Jan-2010 09:50:42] WordPress database error Error on delete of
'C:\Windows\TEMP\#sql17e0_1b0_0.MYD' (Errcode: 13) for query  SELECT
SQL_CALC_FOUND_ROWS  wp_posts.* FROM wp_posts  INNER JOIN wp_term_relationships
ON (wp_posts.ID = wp_term_relationships.object_id) INNER JOIN wp_term_taxonomy ON
(wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id)
WHERE 1=1  AND wp_term_taxonomy.taxonomy = 'category' AND
wp_term_taxonomy.term_id IN ('3') AND wp_posts.post_type = 'post' AND
(wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private') GROUP BY
wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 0, 10 made by require, wp,
WP->main, WP->query_posts, WP_Query->query, WP_Query->get_posts

The important part here is "Errcode: 13", which is a file access error. The MySQL daemon process (mysqld.exe) randomly cannot access temporary tables it itself creates, which causes these errors and failed queries.

Digging Around

After looking around and finding nothing obvious, I tracked down a few forum posts mentioning the same issue.

Here is the gist – the problem is caused by an anti-virus program that is clearly not working properly.

The one mentioned on the forums is McAfee (big surprise, right? McAfee is a piece of junk – probably the worst anti-virus I've ever tried). In my case, however, it was the recently installed freeware security program from Microsoft called Microsoft Security Essentials, highly praised but problematic in this case nonetheless.

After thinking about it, I am confident that these security programs don't actually purposely prohibit access to the files MySQL creates. Instead, they lock these files for the duration of the check, so that the system doesn't get infected before they are approved. If you notice, the problems are related to temporary tables, created by MySQL on the fly. MySQL probably has a very short access timeout for these files, for performance reasons, and because it doesn't get this access fast enough, it considers it a failure (file deletions are failing, as you can see in the log).

The Solution

Now onto fixing the problem. The solution is to have Microsoft Security Essentials, in my case, or whatever your security program may be ignore these files.

You can block the whole Temp directory from being checked. It is not a good idea, as viruses can trickle down to that location and bypass your anti-virus:

Instead, you should just ignore by file extension: *.MYI and *.MYD. MySQL uses files with these extensions for its MyISAM table types. Using this approach is obviously safer as it doesn't single out a directory and instead targets specific files:

After I applied either of these exclusions, all MySQL Error 13 problems went away immediately.

Conclusion

The intermittent Error 13 problem on Windows is caused by 2 otherwise legitimate processes which, when mixed together, end up breaking MySQL.

Is this entirely the fault of the antivirus programs? Inadvertently, perhaps so.

Could MySQL be a bit smarter in this scenario? Perhaps so as well.

What do you, MySQL pros, think?

• How To Diagnose And Fix Incorrect Post Comment Counts In WordPress
• MySQL Conference Liveblogging: EXPLAIN Demystified (Tuesday 2:00PM)
• Swapping Column Values in MySQL
• [MySQL] Deleting/Updating Rows Common To 2 Tables – Speed And Slave Lag Considerations
• MySQL Conference Liveblogging: Benchmarking Tools (Wednesday 4:25PM)

Here's where Firebug comes in with its JavaScript debugger. I'm used to using a debugger in every language I deal with, so using Firebug is a no brainer. Since it supports breakpoints, stopping execution and inspecting local variables and the rest of the scope generally beats alerts and console.logs for me.

Here's what a typical breakpoint looks like in Firebug:

It's easy to set breakpoints in static scripts – just open the Scripts tab, select a JavaScript file from the dropdown menu, and click to the left of the wanted line number.

Then, when the page is reloaded, if your breakpoints are triggered, Firebug will pause script execution and transfer the control to you.

In most cases, the method above is the only method of setting breakpoints you will ever need to use.

The Problem With Dynamic JavaScript

However, what if the JavaScript file where you need to set breakpoints is not static but instead dynamic (generated on the fly). If you set a breakpoint in this case and reload the page, the breakpoint will most likely disappear, especially if the JavaScript url is generated uniquely every time.

The Solution

If you have access to the source, the solution comes in the form of the

debugger;

keyword. Just add it to your dynamic JavaScript generator or into any JavaScript file you have access to exactly where you want Firebug to break, and voila – it does.

More so, this method also works in Google Chrome and IE (if you have Microsoft Script Debugger). Here is a screenshot of my Chrome Beta 4.0.266.0 triggering:

I consider this feature relatively unpublished and therefore awesome because:

• it's hard to search for this specific meaning of the keyword "debugger" when Firebug itself is a debugger and it's a very popular word
• nobody really reads documentation for Firebug, and even if they do, I haven't actually seen the debugger keyword mentioned
• I didn't know about it until recently, even though I've been using Firebug for years

Of course, you need access to the code for this to work, so it's not going to work if you're trying to debug someone else's JavaScript.

And finally, don't forget to remove any traces of 'debugger' from your code when you go live or your users will rightfully hunt you down.

Credits And References:

• suggested by a co-worker from Plaxo – Russ. Thanks Russ!
• some more interesting info at StackOverflow and Infragistics.

Happy debugging!

(*) – if your project works 100% on the first pass, you must be either a magician or Jon Skeet (Jon Skeet Facts – a-la Chuck Norris, a must read).

• [WordPress Plugin Development] How To Include CSS and JavaScript Conditionally And Only When Needed By The Posts
• [Updated for 2012] Here's An Exclusive 10% Off NuSphere PHPEd Discount Coupon Code (Also Includes NuCoder And PHPDoc)
• How To Make Your Site Lightning Fast* By Compressing (deflate/gzip) Your HTML, Javascript, CSS, XML, etc In Apache
• How To Dynamically Increase Memory Limits When Interfacing With WordPress Using XML-RPC (Windows Live Writer, Etc)
• How To Fix Incomplete WordPress (WXR) Exports

Today, however, we know a lot more. Firefox for Mobile, codenamed Fennec, is coming next month (see More Info below) and looks very promising.

Here is a recent video of the Firefox's mobile and design teams discussing and showing the features of Mobile Firefox:

Feature Highlights

For the busy folks, here are the highlights from the video:

• Mobile Firefox uses as much screen real estate as possible to show the page
• swipe screen left to see the open tab list
• swipe right to see navigation buttons
• awesome bar knows what you're looking for as you start typing, just like in your desktop Firefox
• sync using Mozilla Weave – seamless sync desktop and mobile, including history and open tabs
• location based browsing – allows sharing your location with websites automatically (obviously, with your approval only – excellent!)
• security and privacy are the #1 priority
• mobile add-ons – they're trying to create the same development platform that the desktop Firefox already uses. Yup, that means AdBlock is coming.
• HTML 5 support

And some more from Mozilla and GigaOM's interview with Mozilla CEO John Lilly:

• support for Javascript, CSS, Flash, SVG, video, and audio
• based on Firefox 3.6 engine which is not even available for desktops yet
• Firefox for Mobile will initially support Windows Mobile and Maemo, followed by Android and Symbian
• surely, iPhone owners are out of luck as Apple will never allow a competing browser app on their market. Great – another reason to switch to Android
• Blackberry users are also out of luck as BB uses Java
• Android also uses Java and initially Mozilla ignored it. However, later the Android NDK came out, which allows building of applications in C/C++ and this fact changed everything
• the first public releases should be available in December 2009, according to this article.

Some useful linkage:

• Mozilla's own Mobile wiki page
• Mobile features
• Supported platforms (looks slightly out of date)
• GigaOM: On Mobile Phones, Firefox’s Big Bet Is Nokia & Android
• Firefox Headed to Android Phones

Conclusion

Firefox for Mobile is going to be a serious contender in the mobile web space – we know how serious these guys can get, don't we? After all, they've turned a barely known browser into a #2 in the last few years (soon to be #1, right?). Surely, they can do it again, this time in mobile.

I will be trying out Mobile Firefox (for Android) as soon as I get my hands on it, so expect a detailed review then. I have little doubt that it will replace my plain mobile browser.

What do you guys think?

• HTC Hero Coming To Sprint October 11th! $179.99, Powerful, And Sexy. Here Is Why You Need To Own It
• More On Android – A Mobile OS With A Clue
• Sun Definitely Developing A Phone This Year
• How To Fix Redrawing Problems In Your VNC
• Skype Extension For Firefox Is A Piece Of Crap – Leaks Memory, Hangs Firefox, Clubs Baby Seals

I use Eclipse to do my Android development for a few reasons:

• it's the only IDE fully supported by the Android dev team
• it has a visual Layout/Resources builder that transforms XML files into corresponding visual representations
• it's free and open source
• I've been using Eclipse for many years and am very familiar with it

The Problem

However, one thing about Eclipse Android development has bothered me for a while – and that is XML formatting by the visual tools. See, if you open an XML file in Eclipse and use the Layout/Resources tab (which functions as either a visual UI builder or a visual frontend to resource management), Eclipse creates the XML representation of what you're building and dumps it into the file you're editing. The problem is, this XML is not formatted in any way – Eclipse just writes it in a single line, which looks absolutely horrible and is essentially unusable.

For example, I had this beautifully formatted XML file with a sample Android menu. After using the Layout tab to add another item, I ended up with an single ugly unindented line for it.

The Solution

Here's how to solve this in an almost fully automated fashion, using formatting rules that I think result in the most readable file. There is no need for external tools – Eclipse provides everything needed.

• open up Window->Preferences->XML->XML Files->Editor
  
• check the "Split multiple attributes each on a new line" and adjust other options according to your liking (such as Indent using spaces and Indentation size)
• press OK to save the options
• all you have to do now is press Ctrl-Shift-F or select Source->Format

Let's see what this does to the XML I showed above:

It's a beauty, isn't it? Eclipse's Auto Format is one of its best features (yes, you can apply it to your Android Java code too as well as pretty much any other document format).

Bonus – XML Cleanup

As an added bonus, Eclipse also offers a Cleanup XML option, which can do the following:

• compress empty element tags
• insert required attributes
• insert missing tags
• quote attribute values
• format source (as above)
• convert line delimiters to Windows, UNIX, or Mac

You can access the Cleanup option via Source->Cleanup

Running Cleanup on my already pretty clean XML file turned the last <item> into this:

The first option compressed the <item></item> declaration to simply <item />. Neat, isn't it?

Final Words

+ Clean is good.

+ Uniform is good.

+ Use Ctrl-Shift-F and make it a habit.

+ Create a keyboard shortcut to the Cleanup function by going to Window->Preferences->General->Keys and use it instead of Ctrl-Shift-F (since it already formats as part of the Cleanup).

? I'd like to know if there is a way to apply auto formatting upon save, automatically. Do you?

• Some Useful vim Commands – My vim Cheatsheet
• Installing The Android Plugin For Eclipse
• How To Update Eclipse From Galileo (3.5) To Helios (3.6) In-Place Without Reinstalling
• WordPress Developers – How Do You Make A Living [Poll + Discussion]?
• Mastering The Linux Shell – Bash Shortcuts Explained (Now With Cheat Sheets)

I was working with a site that was using frames. Suddenly, one of the frames (which was hosted on a domain that differed from the one it was embedded in) displayed the following message (in Firefox 3.5.4):

This content cannot be displayed in a frame
 
To protect your security, the publisher of this content does
not allow it to be displayed in a frame.
 
Click here to open this content in a new window

Notice how this is a native Firefox window and not a web page rendering. Quite stumped, I started looking at the frame response and finally found that it included this little header:

X-FRAME-OPTIONS: DENY

Turns out that modern browsers like Firefox 3.5 (turns out it's the NoScript addon that does this and not Firefox itself) and IE8 treat this header as a precautionary measure and display a generic "warning" to the user instead of the page content in certain conditions described below. This effort, led, surprisingly, by Microsoft, was really to protect users from clickjacking (I wrote about clickjacking here earlier) but can be viewed as an alternative to framebusting.

Microsoft introduced the new X-FRAME-OPTIONS header with the following possible values:

• DENY – prevents the page from being rendered if it is contained in a frame
• SAMEORIGIN – same as above, unless the page belongs to the same domain as the top-level frameset holder.

Firefox adopted this technique a few months later (again, I was wrong here – it was NoScript that did it), and I expect other browsers to follow.

So what does it mean to you, the developer?

• setting such a header will essentially render frame busting code unnecessary in modern browsers
• but it has a downside of displaying a relatively ugly warning to the user
• no automatic redirect is done as your page (including any framebusting code) is not loaded
• thus requiring an extra click
• and popping up a new tab or window
• it will work even if the user has Javascript disabled, which is more secure

Some useful discussion on the issue can also be found in this post on Hackademix.net.

So is it good practice to use this new X-FRAME-OPTIONS header instead of the traditional framebusting code? I definitely think so, especially if Firefox and other browsers start supporting it. What do you say?

• How To Fight Clickjacking (Using The Recent Twitter Hijacking As An Example)
• Meet Firefox For Mobile [Video + Feature Highlights + More Info]
• Watch – A Useful Linux Command You May Have Never Heard Of
• How To Make Your Site Lightning Fast* By Compressing (deflate/gzip) Your HTML, Javascript, CSS, XML, etc In Apache
• Rendr – a Live Awesome CSS/HTML Rendering Tool

If you, like me, love StackOverflow, SuperUser, ServerFault – the community programming/software/sysadmin Q&A sites built by Joel Spolsky and Jeff Atwood and you are an active member of these sites, you owe it to yourself to install this underappreciated and unadvertised greasemonkey script: StackOverflow – User Info Aggregate. I really think it should get more attention. All credit goes to Jon Erickson.

(What? You've never heard of the sites mentioned above and you call yourself a programmer? Shame on you – go check them out immediately!)

Once you install the script, the top bar that normally shows only single site statistics, like so:

will turn into a multi-site bar, with all your stats at a glance, like so:

Each icon is hyperlinked to your account on the respective site, which makes navigating between all of them very easy.

Installation

Here are the steps required to get the script up and running:

1. Install greasemonkey for Firefox.
2. Install the StackOverflow – User Info Aggregate script.
3. Right click on the greasemonkey icon and select Manage User Scripts…
   
4. Select the script from the list and click the Edit button. This should open up your favorite editor or prompt you to select one.
   
5. Find the section with userIds and replace them with ones of your own (by default the IDs belong to the script author). You can get your own IDs from the user info page (click your name at the top of each site you have an account on).
    
6. Now save the script file and reload the page – you should have all the stats displayed in one place.

Credits

• Big thanks to the creator of this script Jon Erickson.
• Orginal Meta.StackOverflow post by Jon.

Enjoy.

• Enable A Twitter Retweet (RT) Button That Lets You Add Comments Before Retweeting
• Twitter.com Autocomplete, Auto URL Expansion, Auto URL Shortener, RT Button, Nested Replies, Inline Media Embed, Search Tabs, And More
• How To Fight Clickjacking (Using The Recent Twitter Hijacking As An Example)
• How To Make Firebug's JavaScript Debugger Break Inside Dynamic JavaScript Using The 'debugger' Keyword (IE & Chrome Too)
• How To Export/Import Your ExpanDrive/SFTPDrive Drives And Settings

I was really glad to see Joel Spolsky, Jeff Atwood, and the whole StackOverflow team in person, as well as listen to great talks in the following topics:

9:00 – 9:50    Joel Spolsky Opening Keynote
9:50 – 10:45    Mark Harrison Python
11:00 – 11:55    Rory Blyth iPhone
11:55 – 12:25    Joel Spolsky Fogbugz
13:30 – 14:25    Scott Hanselman ASP.NET-MVC
14:25 – 14:45    Jeff Atwood Stack Overflow
14:45 – 15:40    Daniel Rocha Qt
16:10 – 17:05    James Yum Android
17:05 – 18:00    Yehuda Katz jQuery

My own favorite topics were in the following order of fun/usefulness level:

• iPhone (though I'm interested in Android development myself but it's always good to check out the competition) – it was fun and not boring. Quirky, jittery, OCD all come to mind.
• Joel's keynote (Hi, Joel, I'm @ArtemR ) – I welcome the discussion of simple vs feature rich, though Joel has talked about this on his blog multiple times.
• jQuery – nice intro into history of jQuery and competition with other frameworks. The technical level of specific examples was quite basic.
• Android – this is the topic I'm mostly interested in and unfortunately the main speaker got substituted by a guy who has obviously not presented in front of a large crowd before (sorry, I have to criticize – you'll do better and better each time! Just don't worry so much and know your stuff) and was pretty much a recent college grad from the looks of it. He did a pretty decent job at making the code run but a really poor job of marketing and explaining Android's weaknesses and strengths, as some people on twitter pointed out.
• StackOverflow review – quite brief but fun. Thanks Jeff!
• Python – very odd that it was even there – just a look at a few lines of code from the infamous Python google-like suggest algorithm.
• Qt – didn't really care for it but sat through it. Nokia, meh. Next!
• ASP.NET, Fogbugz – I skipped as I didn't care much for those

Overall, I feel like the information presented in all sessions was pretty basic but I guess that was the idea – a little bit of everything for everyone.

The absence of WiFi was pretty disappointing as I think it was advertised but I didn't care much for it. Kept everyone more concentrated anyway.

The free memory upgrades by Microsoft was an AWESOME move (though mine is already at 4GB and they didn't give out 4GB sticks)!

Lunch was decent – Boudin. Lots of drinks and snacks.

Well worth $99.

Twitter #devdays results.

Cross-posted to Meta-StackOverflow.

• Hidden Features Of Perl, PHP, Javascript, C, C++, C#, Java, Ruby, Python, And Others [Collection Of Incredibly Useful Lists]
• StackOverflow.com, SuperUser.com, ServerFault.com Fan? Here's A Greasemonkey Script To Keep Track Of All Your Accounts
• Monitor All Your Domains From One Location
• The Real Reasons To Use Twitter (Get Over Your Prejudice Already)
• Minority Report Becomes Reality

The plugin install page provides initial instructions and the location of the plugin to give Eclipse (https://dl-ssl.google.com/android/eclipse/). This is very standard stuff and I've installed many plugins exactly the same way. However, here's when problems started.

Problem #1: Error while loading manipulator

Eclipse just shows a cryptic Install failed box. Here are the highlights:

• !MESSAGE Error while loading manipulator
• java.lang.IllegalStateException: !fwConfigLocation.equals(fwPersistentDataLocation)

It looks like for whatever reason, Eclipse's fwConfigLocation variable does not match fwPersistentDataLocation. I have no idea where those come from and searching for a solution (such as this crazy Japanese sourceforge and this Eclipse bug), restarting Eclipse, and beating my head against the wall took the majority of my the 2 hours I spent on this.

Here's the full log, which will help people with the same problem find it in search engines:

!ENTRY org.eclipse.equinox.p2.touchpoint.eclipse 4 0 2009-10-15 13:21:11.224
!MESSAGE Error while loading manipulator.
!STACK 0
java.lang.IllegalStateException: !fwConfigLocation.equals(fwPersistentDataLocation)
	!fwConfigLocation=C:\eclipse\configuration
	,fwPersistentDataLocation=C:\eclipse\plugins\configuration
	at org.eclipse.equinox.internal.frameworkadmin.equinox.EquinoxManipulatorImpl.checkConsistencyOfFwConfigLocAndFwPersistentDataLoc(EquinoxManipulatorImpl.java:65)
	at org.eclipse.equinox.internal.frameworkadmin.equinox.EquinoxManipulatorImpl.loadWithoutFwPersistentData(EquinoxManipulatorImpl.java:360)
	at org.eclipse.equinox.internal.frameworkadmin.equinox.EquinoxManipulatorImpl.load(EquinoxManipulatorImpl.java:331)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.LazyManipulator.loadDelegate(LazyManipulator.java:50)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.LazyManipulator.getConfigData(LazyManipulator.java:108)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallBundleAction.uninstallBundle(UninstallBundleAction.java:74)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallBundleAction.execute(UninstallBundleAction.java:29)
	at org.eclipse.equinox.internal.p2.engine.ParameterizedProvisioningAction.execute(ParameterizedProvisioningAction.java:35)
	at org.eclipse.equinox.internal.provisional.p2.engine.Phase.mainPerform(Phase.java:129)
	at org.eclipse.equinox.internal.provisional.p2.engine.Phase.perform(Phase.java:72)
	at org.eclipse.equinox.internal.provisional.p2.engine.PhaseSet.perform(PhaseSet.java:44)
	at org.eclipse.equinox.internal.provisional.p2.engine.Engine.perform(Engine.java:54)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProvisioningUtil.performProvisioningPlan(ProvisioningUtil.java:389)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProfileModificationOperation.doExecute(ProfileModificationOperation.java:61)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProvisioningOperation.execute(ProvisioningOperation.java:37)
	at org.eclipse.equinox.internal.provisional.p2.ui.ProvisioningOperationRunner$1.run(ProvisioningOperationRunner.java:94)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
 
!ENTRY org.eclipse.equinox.p2.engine 4 4 2009-10-15 13:21:11.240
!MESSAGE An error occurred while uninstalling
!SUBENTRY 1 org.eclipse.equinox.p2.engine 4 0 2009-10-15 13:21:11.240
!MESSAGE session context was:(profile=epp.package.php, phase=org.eclipse.equinox.internal.provisional.p2.engine.phases.Uninstall, operand=[R]javax.activation 1.1.0.v200905021805 --> [R]javax.activation 1.1.0.v200906290531, action=org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallBundleAction).
!SUBENTRY 1 org.eclipse.equinox.p2.engine 4 0 2009-10-15 13:21:11.240
!MESSAGE Error while loading manipulator.
!STACK 0
java.lang.IllegalStateException: Error while loading manipulator.
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.LazyManipulator.loadDelegate(LazyManipulator.java:54)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.LazyManipulator.getConfigData(LazyManipulator.java:108)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallBundleAction.uninstallBundle(UninstallBundleAction.java:74)
	at org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallBundleAction.execute(UninstallBundleAction.java:29)
	at org.eclipse.equinox.internal.p2.engine.ParameterizedProvisioningAction.execute(ParameterizedProvisioningAction.java:35)
	at org.eclipse.equinox.internal.provisional.p2.engine.Phase.mainPerform(Phase.java:129)
	at org.eclipse.equinox.internal.provisional.p2.engine.Phase.perform(Phase.java:72)
	at org.eclipse.equinox.internal.provisional.p2.engine.PhaseSet.perform(PhaseSet.java:44)
	at org.eclipse.equinox.internal.provisional.p2.engine.Engine.perform(Engine.java:54)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProvisioningUtil.performProvisioningPlan(ProvisioningUtil.java:389)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProfileModificationOperation.doExecute(ProfileModificationOperation.java:61)
	at org.eclipse.equinox.internal.provisional.p2.ui.operations.ProvisioningOperation.execute(ProvisioningOperation.java:37)
	at org.eclipse.equinox.internal.provisional.p2.ui.ProvisioningOperationRunner$1.run(ProvisioningOperationRunner.java:94)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
!SESSION 2009-10-15 13:31:40.853 -----------------------------------------------
eclipse.buildId=I20090611-1540
java.version=1.6.0_16
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86

Finally, I found a solution that worked – modify eclipse.ini (the one that sits in elipse's install directory) and add the following:

1
2
3
4

-startup
file:/C:/eclipse/plugins/org.eclipse.equinox.launcher_1.0.200.v20090520.jar
--launcher.library
file:/C:/eclipse/plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.0.200.v20090519

Note that the exact locations may differ for you, so figure out what is installed on your machine and replace accordingly.

Also, it seems that the location of the file HAS to be on a new line – otherwise it is not recognized. What kind of shit is that, Eclipse? Some values in eclipse.ini are specified on the same line separated by space, some separated by '=', and some have to be on a new line? This fun trivia fact wasted about 45 minutes of my time.

And yes, startup has 1 dash and launcher.library has 2. /sigh

Edit: it looks like by default Eclipse does come with these options. I somehow managed to lose them, so this is probably my fault. The error couldn't be more cryptic though.

Problem #2: java.io.IOException: The file "C:\eclipse\features\com.android.ide.eclipse.adt_0.9.3.v200909031112-12945" does not exist

Great. Now what…

After examining the features directory, I indeed found that directory com.android.ide.eclipse.adt_0.9.3.v200909031112-12945 does not exist. What I did find though is com.android.ide.eclipse.adt_0.9.3.v200909031112-12945.jar, which is simply an archive. Why didn't you unpack this archive, Eclipse? Or Android plugin devs? Whoever is responsible for this – argh!

After quickly creating the directories and unpacking the .jar contents into them, Eclipse was finally able to install the Android plugin.

If installing a plugin is this friendly, I can only imagine what awaits me in Android app development. Am I the only one with these problems? You let me know.

• [Android] Auto Formatting Android XML Files With Eclipse
• How To Update Eclipse From Galileo (3.5) To Helios (3.6) In-Place Without Reinstalling
• A Short Note On eAccelerator – The PHP Accelerator
• My Notes On Learning Python Coming From Perl
• Meet Firefox For Mobile [Video + Feature Highlights + More Info]

Screen is awesome. Once you become comfortable navigating around it, you start using it ALL the time. No more dropped sessions, no having 10 Putty windows open at the same time, no more nohup.

However, with default screen settings I've always felt a bit lost and out of place, mostly because there was no "taskbar" with a bird's eye view of all windows. Pressing ctrl-a, " really does get annoying fast (that's the command that brings up the window selector – screenshot below).

So instead I modded my screen to have a "taskbar" which sits at the bottom of screen and adds:

• the name of each window
• a clear marking of window states, such as
  • the currently active window (* and different background)
  • last used window (-)
  • if you're sharing the current window with someone else (&)
• the current incoming/outgoing bandwidth in bytes/sec, updating every 5 seconds
• the current time
• the load on the server, as reported by the uptime or w commands

Of course, all of the above is customizable to your liking, but this is what I picked for myself.

Without further ado, here's a screenshot:

Doesn't that make you feel much more homey?

Now using a shared screen session for pair programming, as suggested in a recent article posted on Hacker News, suddenly sounds even more appealing.

Installation

Here's what you need to get this bar on your screen:

• download my .screenrc (0 B, downloaded 963 times) and put it in your home directory. Make sure the location of wormulon (installed in the next step) is correct. Also, specify the right network interface (eth0 in most cases).
  Here is the file, for reference:

1
2
3
4
5
6
7
8
9
10
11

# Monitor eth0 bandwidth and update every 5 s
# Assign the output of this cmd to '%1`' and use it in the below hardstatus line:
backtick 1 0 0 /usr/local/bin/wormulon -d 5 -i eth0
 
hardstatus alwayslastline "%{bw}[%H] [%?%-Lw%?%{wb}%n*%f %t%{bw}%?%+Lw%?]%=%{bw} [%1`] [%c:%s] [%l]" # modified from http://lists.gnu.org/archive/html/screen-users/2007-08/msg00008.html
 
# Set the scrollback length:
defscrollback 10000
 
# Select whether you want to see the copyright notice during startup:
startup_message off

• download and compile wormulon (wormulon-0.1.4.tar.gz 45.3 kB, downloaded 473 times or from a mirror) – a nifty little program that shows incoming and outgoing bandwidth and is perfect for integrating with screen. You can omit this step if you don't want the bandwidth monitor. I know the name is a bit odd but it's a legit little open source utility – feel free to check out the code.

tar xvzf wormulon-0.1.4.tar.gz
cd wormulon-0.1.4/
./configure && make
sudo make install # this installs into /usr/local/bin/;
                  # alternatively, copy the wormulon binary wherever you
                  # want and update the path in .screenrc

Now restart screen and you're good to go.

• How To Download And Install Windows Media Player 11 Bypassing WGA
• Watch – A Useful Linux Command You May Have Never Heard Of
• How To Update Eclipse From Galileo (3.5) To Helios (3.6) In-Place Without Reinstalling
• Mastering The Linux Shell – Bash Shortcuts Explained (Now With Cheat Sheets)
• How To Resize/Grow VMware Linux Disks and Partitions

I am throwing up a quick post about a relatively cryptic error that Solr started throwing the other day here at Plaxo. After happily running for a few days, I suddenly started getting pages about failed Solr indexing.

Upon closer examination, I saw the following repeatedly in the log file:

catalina.2009-09-18.log:SEVERE: java.io.IOException: directory 'DATADIR/index'
exists and is a directory, but cannot be listed: list() returned null

I tried to see if sending an OPTIMIZE command would help but the server returned the same response.

Digging Deeper

The reason was these errors was quite simple – Solr was running into the system level limit on allowed number of open files (ulimit). This limit can be seen by running

ulimit -n
1024

or simply

ulimit -a | grep 'open files'
open files                      (-n) 1024

This means that if a process tries to open that many files at the same time, the kernel will prohibit opening any more, which in my case caused the Java IOException.

In my case, I haven't been using the Solr OPTIMIZE command for a while, so after a lot of COMMITs, the Solr data got pretty fragmented, thus hitting the open files limit.

The Solution

There are 2 things to be done here:

1. OPTIMIZE Solr more often. When an OPTIMIZE occurs, multiple index files are merged into 1, thus reducing the number of files that need to be opened. However, before you can OPTIMIZE, you have to raise the allowed number of open files (see the next bullet).

2. Set a higher open files limit for the user that runs Solr (in my case, the solr user) – for example to 4096 instead of 1024. One way to do it is by adding a file /etc/security/limits.d/solr.conf with the following contents:

   solr hard nofile 4096
   solr soft nofile 4096

   and then logging out and back in. The file should be automatically loaded, which you can verify by running the ulimit commands from the section above.

Happy Solring!

By the way, here's a really good resource for Solr 1.4 that just came out: Solr 1.4 Enterprise Search. I have this book and it's quite helpful in explaining such topics as multicore setup, search methods, replication, etc.

• How To Show Hidden/Invisible Files In Total Commander, Both Locally And On An FTP Server
• Comparison Between Solr And Sphinx Search Servers (Solr Vs Sphinx – Fight!)
• Watch – A Useful Linux Command You May Have Never Heard Of
• [Perl] Finding Files, The Fun And Elegant Way
• How To Sort Folders The Same Way As Files In Total Commander

Similarities

• Both Solr and Sphinx satisfy all of your requirements. They're fast and designed to index and search large bodies of data efficiently.
• Both have a long list of high-traffic sites using them (Solr, Sphinx)
• Both offer commercial support. (Solr, Sphinx)
• Both offer client API bindings for several platforms/languages (Sphinx, Solr)
• Both can be distributed to increase speed and capacity (Sphinx, Solr)

Here are some differences

• Solr, being an Apache project, is obviously is Apache2-licensed. Sphinx is GPLv2. This means that if you ever need to embed or extend (not just "use") Sphinx in a commercial application, you'll have to buy a commercial license.
• Solr is easily embeddable in Java applications.
• Solr is built on top of Lucene, which is a proven technology over 7 years old with a huge user base (this is only a small part). Whenever Lucene gets a new feature or speedup, Solr gets it too. Many of the devs committing to Solr are also Lucene committers.
• Sphinx integrates more tightly with RDBMSs, especially MySQL.
• Solr can be integrated with Hadoop to build distributed applications
• Solr can be integrated with Nutch to quickly build a fully-fledged web search engine with crawler.
• Solr can index proprietary formats like Microsoft Word, PDF, etc. Sphinx can't.
• Solr comes with a spell-checker out of the box.
• Solr comes with facet support out of the box. Faceting in Sphinx takes more work.
• Sphinx doesn't allow partial index updates for field data.
• In Sphinx, all document ids must be unique unsigned non-zero integer numbers. Solr doesn't even require a unique key for many operations, and unique keys can be either integers or strings.
• Solr supports field collapsing to avoid duplicating similar results. Sphinx doesn't seem to provide any feature like this.

Related questions

• http://stackoverflow.com/questions/1284083/choosing-a-stand-alone-full-text-search-server-sphinx-or-solr
• http://stackoverflow.com/questions/1132284/full-text-searching-with-rails
• http://stackoverflow.com/questions/737275/pros-cons-of-full-text-search-engine-lucene-sphinx-postgresql-full-text-searc

Conclusion

In my experience, Solr is very-very fast on the query side. It is also very powerful. The indexing side is very CPU and memory intensive and is an unfortunate side effect of having such a feature-rich, fast application. Nevertheless, I highly recommend Solr.

For disclaimer purposes, I have not had much experience with Sphinx and, again, all credit for this comparison goes to mausch.

By the way, here's a really good resource for Solr 1.4 that just came out: Solr 1.4 Enterprise Search. I have this book and it's quite helpful in explaining such topics as multicore setup, search methods, replication, etc.

• [Solr] How To Fix java.io.IOException: directory FOO exists and is a directory, but cannot be listed: list() returned null
• Delicious.com [Quietly] Rolls Out Domain And Url Searching/Filtering. Finally!
• Hidden Features Of Perl, PHP, Javascript, C, C++, C#, Java, Ruby, Python, And Others [Collection Of Incredibly Useful Lists]
• My MySQL Conference Schedule
• [Updated x3] Goodbye Outlook, I Don't Need You Anymore – Gmail Now Lets You Paste Images Directly From Clipboard

StackOverflow is an amazing site for coding questions. It was created by Joel Spolsky of joelonsoftware.com, Jeff Atwood of codinghorror.com, and some other incredibly smart guys who truly care about user experience. I have been a total fan of SO since it went mainstream and it's now a borderline addiction (you can see my StackOverflow badge on the right sidebar).

The Story

So, one day someone at StackOverflow started a "Hidden features of" post about a famous language (I don't feel like finding out which one was first exactly), and it turned out to be so popular that other posts in the same series started popping up.

Such questions were quickly turned into community wikis, for the purposes of harvesting and organizing information coming from the best developers on the planet and voted by users of the site. There are literally hundreds of answers, sorted by votes.

Hidden Features Of

Programming Languages

Hidden features of ActionScript3 / Flex

Hidden features of ASP.NET

Hidden features of x86 assembly

Hidden features of C

Hidden features of C++

Hidden features of C#

Hidden features of ColdFusion

Hidden features of D

Hidden features of Delphi

Hidden features of Erlang

Hidden features of F#

Hidden features of Flash development, Flash language (AS2/3), and Flash IDE

Hidden features of Java

Hidden features of JavaScript

Hidden features of Haskell

Hidden features of Lisp

Hidden features of Lua

Hidden features of Objective-C

Hidden features of Perl

Hidden features of PHP

Hidden features of Python

Hidden features of R

Hidden features of Ruby

Hidden features of Ruby on Rails

Hidden features of Scala

Hidden features of Silverlight

Hidden features and Dark Corners of STL?

Hidden features of TCL/TK

Hidden features of VB.Net

Hidden features of VBA

Databases

Hidden features of MySQL

Hidden features of Oracle

Hidden features of PL/SQL

Hidden features of PostgreSQL

Hidden features of SQL Server

Mobile

Hidden features of Android development

Other

Hidden features of Bash – also see my bash cheatsheet.

Hidden features of CSS

Hidden features of Django

Hidden features of Eclipse

Hidden features of Emacs

Hidden features of Grails

Hidden features of Greasemonkey

Hidden features of HTML

Hidden features of HTTP

Hidden features of jQuery

Hidden features of mod_rewrite

Hidden features of Qt

Hidden features of RegEx

Hidden features of Spring framework

Hidden features of VHDL

Hidden features of Visual Studio (2005-2008)

Hidden features of Windows.Forms

Hidden features of WPF and XAML

Hidden features of Xpath+Xslt

I will try to maintain this list, adding new languages that join the series as I find them. Now go learn something new!

• StackOverflow.com, SuperUser.com, ServerFault.com Fan? Here's A Greasemonkey Script To Keep Track Of All Your Accounts
• Comparison Between Solr And Sphinx Search Servers (Solr Vs Sphinx – Fight!)
• Twitter.com Autocomplete, Auto URL Expansion, Auto URL Shortener, RT Button, Nested Replies, Inline Media Embed, Search Tabs, And More
• WordPress Developers – How Do You Make A Living [Poll + Discussion]?
• Delicious.com [Quietly] Rolls Out Domain And Url Searching/Filtering. Finally!

Every time I have to deal with svn:externals in SVN, I forget the command line syntax. Every single damn time. Normally, I use SVN GUI clients, such as SmartSVN, which make it very simple to add an svn:externals property. But for command line, it always takes looking at 25 different sites on google, which are all incredibly unhelpful for this question for some reason. Trying "svn help propset" on the command line was bloated and equally useless.

So this time I needed to write it down and make sure everyone who needed help with svn:externals would find exactly what they need here. I hope this page will soon come up on top of all the unhelpful results on google for "propset svn:externals" and other related queries.

The Problem

I want to set a simple svn:externals property in one of my project's directories, lets say 'plugins' (talking about WordPress here). The outcome would be a directory called 'akismet' within 'plugins' that points to a remote svn url.

Various combinations of trying to do it produced pathetic results, like

svn propset svn:externals akismet http://plugins.svn.wordpress.org/akismet/trunk
svn: Setting property on non-local target 'http://plugins.svn.wordpress.org/akismet/trunk' needs a base revision

svn propset svn:externals . akismet http://plugins.svn.wordpress.org/akismet/trunk
svn: Error parsing svn:externals property on 'akismet': '.'

svn propset svn:externals akismet http://plugins.svn.wordpress.org/akismet/trunk akismet
svn: Setting property on non-local target 'http://plugins.svn.wordpress.org/akismet/trunk' needs a base revision

The Solution

Finally, thanks to this post, I found the right command:

svn propset svn:externals 'akismet http://plugins.svn.wordpress.org/akismet/trunk' .
property 'svn:externals' set on '.'

Note that dot at the end of the command and the quotes around the directory name and url.

Now commit via

svn commit

and then

svn up
Fetching external item into 'akismet'
A    akismet/akismet.gif
A    akismet/akismet.php
A    akismet/readme.txt
Updated external to revision 127962.
 
Updated to revision 16.

There, was it that hard, forum gurus and blog fiends?

Edit: in order to set multiple directory/url pairs in a single svn:externals property, you should put the individual dir/url pairs into a file (let's call it 'svn.externals'), like so

akismet http://svn.wp-plugins.org/akismet/trunk
all-in-one-seo-pack http://svn.wp-plugins.org/all-in-one-seo-pack/trunk

and then apply the property using

svn propset svn:externals -F svn.externals .

You should also just check in 'svn.externals' to easily keep track of it.