One of the most powerful and innovative modules to be developed for Drupal this year is Memetracker. Written by Kyle Mathews in the Google Summer of Code 2008, Memetracker intelligently filters content on a website to synthesize the most popular and important topics and conversations into memes. Similar to the way Google News and TechMeme operate, Memetracker can work with RSS and Atom feeds to track issues, current events, and internet “buzz” across numerous channels simultaneously. The module is a “smart aggregator” that can analyze the actual text of RSS feeds and group posts with related content into categories that can be displayed on a website. The Memetracker module uses click tracking to rank the popularity of sources over time and sorts the display automatically based on these results. There is a public demo for Memetracker available online.

Kyle Mathews’ blog contains some great insights on developing social communities with Drupal, and working with Web 2.0 technologies. For example, he writes:

In researching ways that online communities help participants find the most interesting content/conversations, I’ve found three patterns which help.

1. First is the small-world pattern. Via Organic Groups, you split conversations by topic. Members congregate around only the groups they are interested in.
2. Second is the Twitter pattern. Using buddylist or user relationships, members follow friends or people who’s ideas they find interesting.
3. The third pattern is for members to read the most interesting memes as they are somehow determined by the community (this is where my module will help out)…

My proposal then is to write a memetracker module that will fulfill the same role as Techmeme does to the tech community for any community for which it is installed. My module will intelligently filter and group community generated content to display to the community in real time the most interesting conversations and memes as they emerge.

Memetracker is based on another module called Machine Learning API, also written by Kyle, which uses algorithms in a similar manner to a search engine to sort and rank the data into memes. The complexity of the application is such that it requires the processing power of Python to run, using the Python Numeric and Pycluster extensions that must be installed on the server along with Drupal and the standard LAMP configuration. While this is excellent in theory, in practice it is going to make it difficult for many people using shared hosting to use the module. Plans are being made for a future version of Memetracker that runs without the Python requirement, but it is not clear when that release will be made public.

Memetracker is currently the only Open Source meme generating software available for integration with popular Content Management System websites. As such, it is really a breakthrough, following upon the success of TechMeme and other sites that use this technology to rank the most popular news sources on the web, sifting through millions of pages of blog posts and RSS feeds. Another site that has been developed with Drupal and custom code using the Open Calais platform is PolyMeme. The site has just been upgraded with a new design, and is an excellent source of information about meme tracking on the web.

Note: The “meme” is a term first introduced by philosopher Richard Dawkins, who sought to apply scientific methods to the study of human ideas and cultural evolution in a Darwinian context. The meme is an autonomous conceptual entity like a gene or a virus, which can be transmitted, combining in new forms as it replicates, changing over time.

Links:
Memetracker Module
Machine Learning API Module
Kyle Mathews’ Blog
Memetracker Group
Drupalcon 2008 Szeged

Python:
Installing Python for Memetracker
Download Python Numeric
Download Pycluster

This is a post from Web Dev News, a site brought to you by Xavisys Web Development.

Memetracker Module Released for Drupal