Mullenweg: Biggest WordPress.com Mistake – “Buying Servers”

WordPress Guru & Automattic founder Matt Mullenweg gave a speech at the recent GigaOm’s Structure 09 conference in San Francisco, where he spoke on cloud computing and wordpress.com’s server infrastructure. The speech was reported on by Rich Miller at datacenterknowledge.com. According to the report, Mullenweg said, “The biggest mistake we made with the WordPress.com infrastructure was actually buying servers… (buying servers was ) not a utility. Now we lease them all on a month-to-month basis.” According to the presentation, WordPress.com runs “about 5 million sites serving more than 1 billion page views a month. Automattic uses two data center providers, the dedicated hosting specialists ServerBeach (PEER 1) and Layered Technologies.”

Mullenweg went on to say he viewed WordPress.com’s use of Amazon’s S3 storage system “a failure” because it represented a lack of an open source alternative. “When I have to go to the cloud, I consider that a failure. The thing that’s been most exciting to me is how the open source tools have evolved.” The summary, for those who would like to learn from the experience of running one of the most highly trafficked, multi-user blog sites on the internet? Use leased servers rather than investing in your own data center, that way you can keep on the latest and most up to date boxes without having to cover the cost of the perpetual upgrades, management staff for the data center, and infrastructure / backup facilities.

Month to month leasing also keeps open the negotiation options for a large hosting account like WordPress when dealing with service providers, rather than getting locked into a long term contract or HR staffing issue. This makes a lot of sense. For example, I recently looked into opening a data center in India based around Virident Eco Ram servers, 4 at around $5000 each. Add office space in an Indian IT Park ($400-$500 p/month), backup power supply and power synchronizers ($5000), cooling system ($2500), 24 hour sys admin staff ($36-$40,000 p/year), internet backbone connection ($1250-$2000 p/year?), utilities ($100-$150 p/month), etc. and the total quickly adds up. Granted you likely do not need 24 hour sys admin for 4 servers, but how to maintain a data center without on-site security and maintenance? Compare this total cost to what you would receive over a similar time period with leased servers. Easy math.

This is why Mullenweg ended up advising start-ups, small development companies, and media groups not to try to compete with what the major tech companies are doing with web infrastructure. “My challenge to everyone competing with Amazon, Google and Microsoft is to remember that you’re competing with Amazon, Google and Microsoft,” he said. “These are strong technology companies, and if you’re going to compete with them, open source is the only way to do that. Otherwise, you have no leverage.”

What then is the “open source equivalent” of a cloud server, when one of the main aspects to the cloud is “utility banks” of grid servers that can expand and contract around the traffic needs of an individual site? With shared, leased, or dedicated servers you are renting a fixed amount of disk space or a set number of machines. It is the ability to scale to meet the highest peak demand, digg-effect, etc. that the cloud is delivering through mass, corporate data banks. Unless someone develops a “distributed computing” model of sharing cpu resources across a network to freely scale in times of peak traffic, and many “open source” servers joined together to share resources in this way, I don’t see an exact “open source equivalent” to the grid. Maybe it is Bittorrent, LimeWire P2P networks, but the performance there is a lot different from Akamai or RackSpace.

Mullenweg recommended the nginx web server for load balancing:

“nginx has been running for more than four years on many heavily loaded Russian sites including Rambler (RamblerMedia.com).
In March 2007 about 20% of all Russian virtual hosts were served or proxied by nginx.
According to Google Online Security Blog year ago nginx served or proxied about 4% of all Internet virtual hosts.
2 of Alexa US Top100 sites use nginx.
According to Netcraft in December 2008 nginx served or proxied 3.5 millions virtual hosts. And now it is on 3rd place (not counting in-house Google server) and ahead of lighttpd.
According to Netcraft in March 2009 nginx served or proxied 3.06% busiest sites.
According to Netcraft in May 2009 nginx served or proxied 3.25% busiest sites.
Here are some of success stories: FastMail.FM, WordPress.com.”

Source: http://nginx.net/

Optimizing a dedicated server for a high traffic site and “cloud hosting” are very different undertakings. An open source alternative to this aspect of “the cloud” would involve users sharing their CPU cycles with other users around the world or locally on the same network during peak traffic or spinning off virtual clones of the site during overflow to another machine to handle the load. There would seem to be a number of security issues that would arise, and ultimately, somewhere there has to be charity – people giving up their processing power and bandwidth when it is not being used, sharing empty, allocated disk space – entire websites at the end of a torrent, mirrored on different servers. Right now it is basically the SETI Screensaver as a model of the open cloud.

If you have your own data center or dedicated cluster, Apache Hadoop is an open source distributed option: http://hadoop.apache.org/core/

Related: http://www.morganclaypool.com/doi/abs/10.2200/S00193ED1V01Y200905CAC006

Read the full Article: http://www.datacenterknowledge.com/archives/2009/06/25/mullenweg-open-source-trumps-the-cloud/

Automattic releases BuddyPress – Official “Sister Project” to WordPress

Matt Mullenweg, the founder of WordPress and Automattic, announced the release of BuddyPress last week on the official WordPress site. The BuddyPress site is live, with free downloads and installation instructions for BuddyPress 1.0 – which expands a typical WordPress blog installation into a full social network with most of the features of mySpace, Facebook, Ning, and other popular sites. My first reaction on this is… amazing, massive, incredible, exactly what was needed, soon to be huge, and really nice design over all. Congratulations to all involved in the development and publication of this release, it looks like a very important move in the future evolution of the WordPress platform and something that will encourage many social network developers to build with the CMS.

“What if there was software with the elegance and extensibility of WordPress but all the features you’ve come to expect from social networks like Facebook? Now there is: check out BuddyPress. BuddyPress is an official sister project of WordPress. The idea behind it was to see what would happen to the web if it was as easy for anyone to create a social network as it is to create a blog today. There’s been an explosion of social activity on the web, it’s probably the most important trend of the past few years, but there’s been a dearth of Open Source tools that enable the social web. In WordPress we have a robust and extensible base that can scale to many millions of users, and BuddyPress is essentially a set of plugins on top of WordPress that add private messaging, profiles, friends, groups, activity streams, and everything else you’ve come to expect from your favorite social network, like a Facebook-in-a-box.”

To take a look at the BuddyPress demo site, visit: http://testbp.org/

BuddyPress includes user profiles, private messaging, friends / buddylists, groups, activity streams, a wall / stream like section called “the wire” for status updates and tweet-like on-site micro-blogging, in addition to multi-user blogs and forums. I use this same profile quite a lot in building social networks with Drupal using Panels, Advanced Profile Kit, Buddylist, Private Message, Flag, Activity Stream, Views, CCK, Content Profile / Bio, Organic Groups, and other modules. Because of the multiple development teams managing the combination of modules needed to build the working equivalent of this in Drupal, and the 5.x / 6.x / 7.x development cycle variations + all the time assembling, theming, and debugging a social network install in Drupal… the out of the box offering from BuddyPress will be a strong challenge.

For examples of sites that have been built with BuddyPress, see:

Sample demo profile page: http://testbp.org/members/galen/

WannaNetwork – Online Real Estate Community: http://wannanetwork.com/

Flokka – Women in Business: http://flokka.com/

GrungePress – Online Music Community: http://grungepress.com/

Working daily with both WordPress and Drupal both for web publishing and building social networks for clients, I have long felt WordPress had many advantages for single user blogs (really nice themes, for example) vs. Drupal, but lacked the module expandability to allow the construction of social networks. BuddyPress completely changes that and offers out of the box what is very challenging to build in Drupal. It could save 2 to 3 weeks development time on a complex social network site, and allowing the designer to focus work on the theme and content rather than building the module architecture.

My hope is that Acquia (or another company or developer) will release a “social network” installation profile for Drupal that is similar to this. Despite the large number of social networks built with Drupal, I don’t think there has been anything close to an “official” social networking profile like BuddyPress. One reason for this – and it may be related to the release of BuddyPress as a “sister project” rather than just a collection of modules that plugin to WordPress, is to create a complex social network site that deploys on an installation profile you need to install in a way that the database is pre-populated with all the correct settings, permissions, and everything is automatically positioned in the site by block, section, menu, etc. To do this in Drupal, you basically have to include a mySQL database map with the installation profile – something that I also haven’t seen often, but we are working on at TypeHost. Then you have to have a GUI layer that makes it easy for the user to transform the archetypal site structure into a personal site. From the way it looks, Automattic has done this perfectly with BuddyPress.

In terms of branding, the name is not the absolute best choice here in my opinion, but there must have been a clear reasoning behind not just releasing it as another version of WordPress, like “WordPress – SN (Social Network)” vs. “WordPress (Blog or Standard version).” Also it is interesting that the projects seem to be on independent / co-dependent / inter-related development paths, but that BuddyPress is not being considered “the next” version of WordPress. Again, similar to Drupal, there is some decision making that sees these not as “core” modules – despite the fact that many people see them as core to the functionality of the CMS. WordPress functioning as a full social network may not be needed by the majority of single user blog publishers who use the CMS as a platform, but this release is going to make a big difference on the web. It will be interesting to chart the usership statistics of BuddyPress vs. WordPress over the next year to see how many sites adopt the new changes.

Summary: combined with WordPress themes and publishing ease, the addition of full social network functionality to the platform with the release of BuddyPress 1.0 is a slam dunk / home run for Automattic, bloggers, traditional WordPress users, and social network developers. Look to see this on even more websites than WordPress in the future, and to pull a lot of development away from Drupal, which still lacks a unified offering that builds a social network as simply and easily as BuddyPress.

Mullenweg: Scale WordPress to 20,000,000 Views per Day for $100 p/month

An interesting article appeared on the front page of the drupal.org website, detailing the migration of the popular “crooks and liars” blog from WordPress to Drupal. According to the developers, when the site was averaging around the “200,000 hits per day mark, we started experiencing a lot of down time from server overloads. We were utilizing the famous wp-cache plugin for WordPress, as well as hosting the database on a single master and two slaves, using the HyperDB class for WordPress to handle the replication.” After experiencing a high degree of server downtime from the massive number of comments on the site, “crooks and liars” began to consider porting the site to Drupal for performance issues.

According to the site development team, benchmark tests showed that a Drupal 5.x installation was able to serve more than 8 times the number of pages per second vs. a standard WordPress 2.3 set up:

“I setup default installations of WordPress 2.3 and Drupal 5. I only enabled the core caching mechanisms in both setups and populated them with the exact same data and display options. Both systems also used the default themes and features. After running a series of tests through JMeter, I quickly confirmed my beliefs and even exceeded them as I saw Drupal was able to handle about eight times the requests per second as WordPress, both on the front page and the same single post view with 157 comments.”

Read the rest of Mullenweg: Scale WordPress to 20,000,000 Views per Day for $100 p/month »

Automattic Acquires Poll Daddy – Integrated Polls & Surveys for WordPress

Matt Mullenweg, the founding developer of WordPress, announced on his blog that Automattic has just acquired the company Poll Daddy, to shore up a lack of integrated polling functionality on the popular Open Source blogging platform. According to Mullenweg, polling has become very popular with WordPress users, and there are over a dozen different companies developing polling plugins for the CMS. Yet, when looking at the top sites, the “Poll Daddy” functionality was most favored. Upon investigation, it turns out the company was run by two programmers in Ireland – Mullenweg traveled there, had a few beers with the crew, and acquired the company for Automattic.

Looking at the functionality of “Poll Daddy,” the main advantages are in its flexibility and ability to be customized. There are a number of skins available for the plugin, and developers can modify the display easily through CSS. HTML customizations can be used to add image, video, and multimedia files. Poll Daddy adds realtime data reporting from polls and surveys to a WordPress site, and the data can be exported through XML, RSS, or CSV. In setting up questions, there are over ten different formats to select from, which can be further configured in the control panel.

Read the rest of Automattic Acquires Poll Daddy – Integrated Polls & Surveys for WordPress »

Automattic Acquires IntenseDebate – Multisite Threaded Comments

Automattic, the company behind the development of the popular WordPress content management system, announced today that they acquired IntenseDebate, an online service that provides a system for threaded commenting and tracks user activity across multiple websites. The financial terms related to the agreement were not released, but the announcement comes after several moves by WordPress to acquire strategic companies whose services expand the core functionality of the CMS. Examples of this are the previous acquisitions of bbPress (a forum software company), Gravatar (cross site avatar support), and Ping-O-Matic (pingbacks). IntenseDebate announced on their blog that they were happy to be joining the Automattic team, and would be going back into development to build new features for the system. Some of IntenseDebate’s functionality is slated to become part of the WordPress code base, with a plugin available for integrating its more advanced services into the CMS.

Jon Fox, a 23 year old Computer Science graduate student at Washington University in St. Louis and co-founder of IntenseDebate, was interviewed last month by Center Networks, where he described the service:

Read the rest of Automattic Acquires IntenseDebate – Multisite Threaded Comments »