Automattic
Mullenweg: Scale WordPress to 20,000,000 Views per Day for $100 p/month
An interesting article appeared on the front page of the drupal.org website, detailing the migration of the popular “crooks and liars” blog from WordPress to Drupal. According to the developers, when the site was averaging around the “200,000 hits per day mark, we started experiencing a lot of down time from server overloads. We were utilizing the famous wp-cache plugin for Wordpress, as well as hosting the database on a single master and two slaves, using the HyperDB class for Wordpress to handle the replication.” After experiencing a high degree of server downtime from the massive number of comments on the site, “crooks and liars” began to consider porting the site to Drupal for performance issues.
According to the site development team, benchmark tests showed that a Drupal 5.x installation was able to serve more than 8 times the number of pages per second vs. a standard WordPress 2.3 set up:
“I setup default installations of Wordpress 2.3 and Drupal 5. I only enabled the core caching mechanisms in both setups and populated them with the exact same data and display options. Both systems also used the default themes and features. After running a series of tests through JMeter, I quickly confirmed my beliefs and even exceeded them as I saw Drupal was able to handle about eight times the requests per second as Wordpress, both on the front page and the same single post view with 157 comments.”
An interesting overview of the migration, and custom modules used in the development of the “crooks and liars” site can be found online here:
What is more interesting, is after the post was published, WordPress / Automattic founder Matt Mullenweg weighed in personally on the issue, by commenting on the post and listing ways to configure WordPress to scale to 20 million hits per day – at a cost of only $100 per month. He writes:
“Always sorry to see someone leave WordPress, but you ended up pretty much the other best place I could think of. Features are a great reason to switch, but scaling doesn’t need to be. We host some of the largest poltical blogs like all of CNN’s which regularly get thousands of comments per day and we do about a billion pageviews a month on WordPress.com, so here are some tips for future people who may come across this post (some which may be useful to the Drupal community as well):
1. Every release of WP gets faster, so upgrading can get you sometimes significant boosts depending on your bottleneck.
2. Use the memcached object cache backend.
3. If memcached is set up, use Batcache instead of wp-cache.
4. If you get a lot of comments, consider using InnoDB as your storage engine instead of MyISAM inside of MySQL.
5. Double-check that your webserver is set up properly for static requests, this is the cause of 90%+ of the problems we see.
With the above and a single $100/month server from LT you can get around 20,000,000 pageviews a day. With shared Batcache and HyperDB (which you already used, nice) it’s a lot easier to scale out both the web and database tier independently as needed. We haven’t found the upper limit of this strategy yet.”
Included are links to Quantcast’s statistics proving 1 billion page hits per month on wordpress.com (globally): http://www.quantcast.com/p-18-mFEk4J448M/traffic
Link to the Memcache Plugin: http://plugins.trac.wordpress.org/browser/memcached/trunk
Link to the Batcache Plugin: http://wordpress.org/extend/plugins/batcache/
According to the site:
“Development testing showed a 40x reduction in page generation times: pages generated in 200ms were served from the cache in 5ms. Traffic simulations with Siege demonstrate that WordPress can handle up to twenty times more traffic with Batcache installed.”
Based on Quantcast statistics, Drupal.org ranks 13,298 overall while WordPress.org ranks #11. Global tracking statistics are not available for drupal.org on the site.
Automattic Acquires Poll Daddy - Integrated Polls & Surveys for WordPress
Matt Mullenweg, the founding developer of WordPress, announced on his blog that Automattic has just acquired the company Poll Daddy, to shore up a lack of integrated polling functionality on the popular Open Source blogging platform. According to Mullenweg, polling has become very popular with WordPress users, and there are over a dozen different companies developing polling plugins for the CMS. Yet, when looking at the top sites, the “Poll Daddy” functionality was most favored. Upon investigation, it turns out the company was run by two programmers in Ireland – Mullenweg traveled there, had a few beers with the crew, and acquired the company for Automattic.
Looking at the functionality of “Poll Daddy,” the main advantages are in its flexibility and ability to be customized. There are a number of skins available for the plugin, and developers can modify the display easily through CSS. HTML customizations can be used to add image, video, and multimedia files. Poll Daddy adds realtime data reporting from polls and surveys to a WordPress site, and the data can be exported through XML, RSS, or CSV. In setting up questions, there are over ten different formats to select from, which can be further configured in the control panel.
The use of conditional branching allows you to split surveys into different paths depending on the answers given by the user. Thus Poll Daddy allows you to create highly complex surveys on a WordPress site, and the pages themselves can be styled individually. Repeat visitors can be automatically blocked from retaking the poll using IP analysis tools, and with the “Pro” version, there are a lot more options for tracking and vote counting. Interestingly, Poll Daddy isn’t limited only to WordPress – it can also be integrated in other blogging software like Blogger and TypePad, as well as on social networking platforms.
There are three versions of Poll Daddy – Free, Pro, and Pro2. The biggest difference is when you pay for the software, the proprietary link to the developer’s website is removed. Other than that, you get support for more responses and questions per survey. The “Free” version only allows for 10 questions and 100 responses per month, which may be limiting to many. For reference, the “Pro” version upgrade is $200 per year and includes 1,000 survey responses per month plus unlimited questions, while the “Pro2” version is $899 per year with 10,000 survey responses. The two “Pro” versions also include tech support by phone.
The highest profile users of Poll Daddy are Wired, Fox, PC World, TechCrunch, and Read/Write Web. According to Mullenweg, the service has just been enabled for an additional 4.4 million blogs on wordpress.com. If you want to try out the plugin on a WordPress site hosted on your own servers, the download is available at http://wordpress.org/extend/plugins/polldaddy/ and more information can be found at the Poll Daddy site.
Looking at the announcement on wordpress.com, there doesn’t seem to be a single negative comment from the community on this:
http://en.blog.wordpress.com/2008/10/15/polldaddy/
Where other Open Source projects are meandering currently, the constant improvement of WordPress through version upgrades and 3rd party acquisitions such as IntenseDebate and Poll Daddy show Automattic leading the CMS project in a great, new direction, building on the success and popularity of the platform to make it even better.
Automattic Acquires IntenseDebate - Multisite Threaded Comments
Automattic, the company behind the development of the popular WordPress content management system, announced today that they acquired IntenseDebate, an online service that provides a system for threaded commenting and tracks user activity across multiple websites. The financial terms related to the agreement were not released, but the announcement comes after several moves by WordPress to acquire strategic companies whose services expand the core functionality of the CMS. Examples of this are the previous acquisitions of bbPress (a forum software company), Gravatar (cross site avatar support), and Ping-O-Matic (pingbacks). IntenseDebate announced on their blog that they were happy to be joining the Automattic team, and would be going back into development to build new features for the system. Some of IntenseDebate’s functionality is slated to become part of the WordPress code base, with a plugin available for integrating its more advanced services into the CMS.
Jon Fox, a 23 year old Computer Science graduate student at Washington University in St. Louis and co-founder of IntenseDebate, was interviewed last month by Center Networks, where he described the service:
“IntenseDebate is a replacement to your stock comment system on blogs and websites. We provide a tool to better facilitate community and interaction by adding loads of new features. Some of my favorites include Reply-by-email (reply to a comment by replying in email to the email notification), threading, voting, universal profiles, reputation, and the ability to track a user and/or topic across blogs.”
According to the interview, the IntenseDebate team consists of only six people:
• CEO - Tom Keller
• CTO – Jon Fox
• Graphics/Design - Isaac Keyet
• Voice of the users - Michael Koenig
• Software Engineer - Austin Hallock
• Software Engineer - Mehmet Alkanlar
The company has seen a regular increase in user statistics since the service was launched, according to Compete going from around 25,000 views per month in April to 350,000 views in August. Overall, the site boasted a 2505% increase in traffic in 2008.
The main competition to IntenseDebate is Disqus, which Compete ranks at #1,797 compared to IntenseDebate at #5,000. In comparison, Disqus went from about 25,000 views per month in August of 2007 to nearly 900,000 views last month, a rise of 3042% over the year. The surge in popularity driving both sites is definitely something WordPress is looking to tap into.
Don Dodge, part of the Microsoft Emerging Business Team, writes on his blog of an email interchange with IntenseDebate CEO Tom Keller. He writes:
“Earlier this week I sent a note to Tom Keller, CEO of IntenseDebate, and asked him for reasons why I should use IntenseDebate. The three reasons he gave me were almost the same three reasons that Fred Wilson cites in his post; (‘Three Reasons to Use Disqus’)
1) Threading comments makes them easier to read,
2) Better user interaction and community,
3) More comments, maybe 5X more comments.”
Based on this analysis, which seems to be correct, WordPress users can expect more comment activity on their posts following the addition of the new services from IntenseDebate. It appears that Automattic has made a great choice in solidifying the platform to ride this continuing internet trend.
Announcement:
http://www.intensedebate.com/blog/2008/09/23/automattic-acquires-intensedebate/