Mullenweg went on to say he viewed WordPress.com’s use of Amazon’s S3 storage system “a failure” because it represented a lack of an open source alternative. “When I have to go to the cloud, I consider that a failure. The thing that’s been most exciting to me is how the open source tools have evolved.” The summary, for those who would like to learn from the experience of running one of the most highly trafficked, multi-user blog sites on the internet? Use leased servers rather than investing in your own data center, that way you can keep on the latest and most up to date boxes without having to cover the cost of the perpetual upgrades, management staff for the data center, and infrastructure / backup facilities.

Month to month leasing also keeps open the negotiation options for a large hosting account like WordPress when dealing with service providers, rather than getting locked into a long term contract or HR staffing issue. This makes a lot of sense. For example, I recently looked into opening a data center in India based around Virident Eco Ram servers, 4 at around $5000 each. Add office space in an Indian IT Park ($400-$500 p/month), backup power supply and power synchronizers ($5000), cooling system ($2500), 24 hour sys admin staff ($36-$40,000 p/year), internet backbone connection ($1250-$2000 p/year?), utilities ($100-$150 p/month), etc. and the total quickly adds up. Granted you likely do not need 24 hour sys admin for 4 servers, but how to maintain a data center without on-site security and maintenance? Compare this total cost to what you would receive over a similar time period with leased servers. Easy math.

This is why Mullenweg ended up advising start-ups, small development companies, and media groups not to try to compete with what the major tech companies are doing with web infrastructure. “My challenge to everyone competing with Amazon, Google and Microsoft is to remember that you’re competing with Amazon, Google and Microsoft,” he said. “These are strong technology companies, and if you’re going to compete with them, open source is the only way to do that. Otherwise, you have no leverage.”

What then is the “open source equivalent” of a cloud server, when one of the main aspects to the cloud is “utility banks” of grid servers that can expand and contract around the traffic needs of an individual site? With shared, leased, or dedicated servers you are renting a fixed amount of disk space or a set number of machines. It is the ability to scale to meet the highest peak demand, digg-effect, etc. that the cloud is delivering through mass, corporate data banks. Unless someone develops a “distributed computing” model of sharing cpu resources across a network to freely scale in times of peak traffic, and many “open source” servers joined together to share resources in this way, I don’t see an exact “open source equivalent” to the grid. Maybe it is Bittorrent, LimeWire P2P networks, but the performance there is a lot different from Akamai or RackSpace.

Mullenweg recommended the nginx web server for load balancing:

“nginx has been running for more than four years on many heavily loaded Russian sites including Rambler (RamblerMedia.com).
In March 2007 about 20% of all Russian virtual hosts were served or proxied by nginx.
According to Google Online Security Blog year ago nginx served or proxied about 4% of all Internet virtual hosts.
2 of Alexa US Top100 sites use nginx.
According to Netcraft in December 2008 nginx served or proxied 3.5 millions virtual hosts. And now it is on 3rd place (not counting in-house Google server) and ahead of lighttpd.
According to Netcraft in March 2009 nginx served or proxied 3.06% busiest sites.
According to Netcraft in May 2009 nginx served or proxied 3.25% busiest sites.
Here are some of success stories: FastMail.FM, WordPress.com.”

Source: http://nginx.net/

Optimizing a dedicated server for a high traffic site and “cloud hosting” are very different undertakings. An open source alternative to this aspect of “the cloud” would involve users sharing their CPU cycles with other users around the world or locally on the same network during peak traffic or spinning off virtual clones of the site during overflow to another machine to handle the load. There would seem to be a number of security issues that would arise, and ultimately, somewhere there has to be charity – people giving up their processing power and bandwidth when it is not being used, sharing empty, allocated disk space – entire websites at the end of a torrent, mirrored on different servers. Right now it is basically the SETI Screensaver as a model of the open cloud.

If you have your own data center or dedicated cluster, Apache Hadoop is an open source distributed option: http://hadoop.apache.org/core/

Related: http://www.morganclaypool.com/doi/abs/10.2200/S00193ED1V01Y200905CAC006

Read the full Article: http://www.datacenterknowledge.com/archives/2009/06/25/mullenweg-open-source-trumps-the-cloud/

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Mullenweg: Biggest WordPress.com Mistake – “Buying Servers”

“What if there was software with the elegance and extensibility of WordPress but all the features you’ve come to expect from social networks like Facebook? Now there is: check out BuddyPress. BuddyPress is an official sister project of WordPress. The idea behind it was to see what would happen to the web if it was as easy for anyone to create a social network as it is to create a blog today. There’s been an explosion of social activity on the web, it’s probably the most important trend of the past few years, but there’s been a dearth of Open Source tools that enable the social web. In WordPress we have a robust and extensible base that can scale to many millions of users, and BuddyPress is essentially a set of plugins on top of WordPress that add private messaging, profiles, friends, groups, activity streams, and everything else you’ve come to expect from your favorite social network, like a Facebook-in-a-box.”

To take a look at the BuddyPress demo site, visit: http://testbp.org/

BuddyPress includes user profiles, private messaging, friends / buddylists, groups, activity streams, a wall / stream like section called “the wire” for status updates and tweet-like on-site micro-blogging, in addition to multi-user blogs and forums. I use this same profile quite a lot in building social networks with Drupal using Panels, Advanced Profile Kit, Buddylist, Private Message, Flag, Activity Stream, Views, CCK, Content Profile / Bio, Organic Groups, and other modules. Because of the multiple development teams managing the combination of modules needed to build the working equivalent of this in Drupal, and the 5.x / 6.x / 7.x development cycle variations + all the time assembling, theming, and debugging a social network install in Drupal… the out of the box offering from BuddyPress will be a strong challenge.

For examples of sites that have been built with BuddyPress, see:

Sample demo profile page: http://testbp.org/members/galen/

WannaNetwork – Online Real Estate Community: http://wannanetwork.com/

Flokka – Women in Business: http://flokka.com/

GrungePress – Online Music Community: http://grungepress.com/

Working daily with both WordPress and Drupal both for web publishing and building social networks for clients, I have long felt WordPress had many advantages for single user blogs (really nice themes, for example) vs. Drupal, but lacked the module expandability to allow the construction of social networks. BuddyPress completely changes that and offers out of the box what is very challenging to build in Drupal. It could save 2 to 3 weeks development time on a complex social network site, and allowing the designer to focus work on the theme and content rather than building the module architecture.

My hope is that Acquia (or another company or developer) will release a “social network” installation profile for Drupal that is similar to this. Despite the large number of social networks built with Drupal, I don’t think there has been anything close to an “official” social networking profile like BuddyPress. One reason for this – and it may be related to the release of BuddyPress as a “sister project” rather than just a collection of modules that plugin to WordPress, is to create a complex social network site that deploys on an installation profile you need to install in a way that the database is pre-populated with all the correct settings, permissions, and everything is automatically positioned in the site by block, section, menu, etc. To do this in Drupal, you basically have to include a mySQL database map with the installation profile – something that I also haven’t seen often, but we are working on at TypeHost. Then you have to have a GUI layer that makes it easy for the user to transform the archetypal site structure into a personal site. From the way it looks, Automattic has done this perfectly with BuddyPress.

In terms of branding, the name is not the absolute best choice here in my opinion, but there must have been a clear reasoning behind not just releasing it as another version of WordPress, like “WordPress – SN (Social Network)” vs. “WordPress (Blog or Standard version).” Also it is interesting that the projects seem to be on independent / co-dependent / inter-related development paths, but that BuddyPress is not being considered “the next” version of WordPress. Again, similar to Drupal, there is some decision making that sees these not as “core” modules – despite the fact that many people see them as core to the functionality of the CMS. WordPress functioning as a full social network may not be needed by the majority of single user blog publishers who use the CMS as a platform, but this release is going to make a big difference on the web. It will be interesting to chart the usership statistics of BuddyPress vs. WordPress over the next year to see how many sites adopt the new changes.

Summary: combined with WordPress themes and publishing ease, the addition of full social network functionality to the platform with the release of BuddyPress 1.0 is a slam dunk / home run for Automattic, bloggers, traditional WordPress users, and social network developers. Look to see this on even more websites than WordPress in the future, and to pull a lot of development away from Drupal, which still lacks a unified offering that builds a social network as simply and easily as BuddyPress.

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Automattic releases BuddyPress – Official “Sister Project” to WordPress

According to the site development team, benchmark tests showed that a Drupal 5.x installation was able to serve more than 8 times the number of pages per second vs. a standard WordPress 2.3 set up:

“I setup default installations of WordPress 2.3 and Drupal 5. I only enabled the core caching mechanisms in both setups and populated them with the exact same data and display options. Both systems also used the default themes and features. After running a series of tests through JMeter, I quickly confirmed my beliefs and even exceeded them as I saw Drupal was able to handle about eight times the requests per second as WordPress, both on the front page and the same single post view with 157 comments.”

An interesting overview of the migration, and custom modules used in the development of the “crooks and liars” site can be found online here:

http://drupal.org/node/341231

What is more interesting, is after the post was published, WordPress / Automattic founder Matt Mullenweg weighed in personally on the issue, by commenting on the post and listing ways to configure WordPress to scale to 20 million hits per day – at a cost of only $100 per month. He writes:

“Always sorry to see someone leave WordPress, but you ended up pretty much the other best place I could think of. Features are a great reason to switch, but scaling doesn’t need to be. We host some of the largest poltical blogs like all of CNN’s which regularly get thousands of comments per day and we do about a billion pageviews a month on WordPress.com, so here are some tips for future people who may come across this post (some which may be useful to the Drupal community as well):

1. Every release of WP gets faster, so upgrading can get you sometimes significant boosts depending on your bottleneck.
2. Use the memcached object cache backend.
3. If memcached is set up, use Batcache instead of wp-cache.
4. If you get a lot of comments, consider using InnoDB as your storage engine instead of MyISAM inside of MySQL.
5. Double-check that your webserver is set up properly for static requests, this is the cause of 90%+ of the problems we see.

With the above and a single $100/month server from LT you can get around 20,000,000 pageviews a day. With shared Batcache and HyperDB (which you already used, nice) it’s a lot easier to scale out both the web and database tier independently as needed. We haven’t found the upper limit of this strategy yet.”

Included are links to Quantcast’s statistics proving 1 billion page hits per month on wordpress.com (globally): http://www.quantcast.com/p-18-mFEk4J448M/traffic

Link to the Memcache Plugin: http://plugins.trac.wordpress.org/browser/memcached/trunk

Link to the Batcache Plugin: http://wordpress.org/extend/plugins/batcache/

According to the site:

“Development testing showed a 40x reduction in page generation times: pages generated in 200ms were served from the cache in 5ms. Traffic simulations with Siege demonstrate that WordPress can handle up to twenty times more traffic with Batcache installed.”

Based on Quantcast statistics, Drupal.org ranks 13,298 overall while WordPress.org ranks #11. Global tracking statistics are not available for drupal.org on the site.

http://www.quantcast.com/drupal.org

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Mullenweg: Scale WordPress to 20,000,000 Views per Day for $100 p/month

Looking at the functionality of “Poll Daddy,” the main advantages are in its flexibility and ability to be customized. There are a number of skins available for the plugin, and developers can modify the display easily through CSS. HTML customizations can be used to add image, video, and multimedia files. Poll Daddy adds realtime data reporting from polls and surveys to a WordPress site, and the data can be exported through XML, RSS, or CSV. In setting up questions, there are over ten different formats to select from, which can be further configured in the control panel.

The use of conditional branching allows you to split surveys into different paths depending on the answers given by the user. Thus Poll Daddy allows you to create highly complex surveys on a WordPress site, and the pages themselves can be styled individually. Repeat visitors can be automatically blocked from retaking the poll using IP analysis tools, and with the “Pro” version, there are a lot more options for tracking and vote counting. Interestingly, Poll Daddy isn’t limited only to WordPress – it can also be integrated in other blogging software like Blogger and TypePad, as well as on social networking platforms.

There are three versions of Poll Daddy – Free, Pro, and Pro2. The biggest difference is when you pay for the software, the proprietary link to the developer’s website is removed. Other than that, you get support for more responses and questions per survey. The “Free” version only allows for 10 questions and 100 responses per month, which may be limiting to many. For reference, the “Pro” version upgrade is $200 per year and includes 1,000 survey responses per month plus unlimited questions, while the “Pro2” version is $899 per year with 10,000 survey responses. The two “Pro” versions also include tech support by phone.

The highest profile users of Poll Daddy are Wired, Fox, PC World, TechCrunch, and Read/Write Web. According to Mullenweg, the service has just been enabled for an additional 4.4 million blogs on wordpress.com. If you want to try out the plugin on a WordPress site hosted on your own servers, the download is available at http://wordpress.org/extend/plugins/polldaddy/ and more information can be found at the Poll Daddy site.

Looking at the announcement on wordpress.com, there doesn’t seem to be a single negative comment from the community on this:

http://en.blog.wordpress.com/2008/10/15/polldaddy/

Where other Open Source projects are meandering currently, the constant improvement of WordPress through version upgrades and 3rd party acquisitions such as IntenseDebate and Poll Daddy show Automattic leading the CMS project in a great, new direction, building on the success and popularity of the platform to make it even better.

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Automattic Acquires Poll Daddy – Integrated Polls & Surveys for WordPress

Jon Fox, a 23 year old Computer Science graduate student at Washington University in St. Louis and co-founder of IntenseDebate, was interviewed last month by Center Networks, where he described the service:

“IntenseDebate is a replacement to your stock comment system on blogs and websites. We provide a tool to better facilitate community and interaction by adding loads of new features. Some of my favorites include Reply-by-email (reply to a comment by replying in email to the email notification), threading, voting, universal profiles, reputation, and the ability to track a user and/or topic across blogs.”

According to the interview, the IntenseDebate team consists of only six people:
• CEO – Tom Keller
• CTO – Jon Fox
• Graphics/Design – Isaac Keyet
• Voice of the users – Michael Koenig
• Software Engineer – Austin Hallock
• Software Engineer – Mehmet Alkanlar

The company has seen a regular increase in user statistics since the service was launched, according to Compete going from around 25,000 views per month in April to 350,000 views in August. Overall, the site boasted a 2505% increase in traffic in 2008.

The main competition to IntenseDebate is Disqus, which Compete ranks at #1,797 compared to IntenseDebate at #5,000. In comparison, Disqus went from about 25,000 views per month in August of 2007 to nearly 900,000 views last month, a rise of 3042% over the year. The surge in popularity driving both sites is definitely something WordPress is looking to tap into.

Don Dodge, part of the Microsoft Emerging Business Team, writes on his blog of an email interchange with IntenseDebate CEO Tom Keller. He writes:

“Earlier this week I sent a note to Tom Keller, CEO of IntenseDebate, and asked him for reasons why I should use IntenseDebate. The three reasons he gave me were almost the same three reasons that Fred Wilson cites in his post; (‘Three Reasons to Use Disqus’)
1) Threading comments makes them easier to read,
2) Better user interaction and community,
3) More comments, maybe 5X more comments.”

Based on this analysis, which seems to be correct, WordPress users can expect more comment activity on their posts following the addition of the new services from IntenseDebate. It appears that Automattic has made a great choice in solidifying the platform to ride this continuing internet trend.

Announcement:

http://www.intensedebate.com/blog/2008/09/23/automattic-acquires-intensedebate/

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Automattic Acquires IntenseDebate – Multisite Threaded Comments