The State of Drupal: Building the Future

Half dozen of the other

Has it been six years already? I'm astounded. Drupal in 2004 was really really really different. It's been quite a ride. And it's getting better and more fun and more interesting with every passing moment. And that's because of the stellar contributions of the dozens of core developers on Drupal 7 (and 6 and 5 and 4.7 and 4.6 and 4.5), the leadership of people and companies in the Drupal community, and all the people who contribute back — whether it's out of the notion of "giving back" or out of the self-interest of inviting the rest of the community to help them on their project.

Always kind of an open secret living in the obscurity of web geekdom, Drupal has had some heady mainstream buzz these past few months. As adoption of Drupal continues to increase, and open source gains "acceptance" [read: understanding] in corporate, educational and governmental realms, this trend has some legs, I think. Expect the buzz to bump up louder a notch when Drupal 7 comes out in the next few weeks (or so). Back in 2006, Dries Buytaert noted that the Drupal community doubles with every major release. Will the community double again this year? I guess we'll find out soon enough.

But there are a few really good reasons that it will. Here are a few of them.

Not a conventional convention (or maybe more conventional)?

DrupalCon this spring, the biggest Drupal event ever (so far), simply rocked. 3000 attendees, a Tim O'Reilly keynote, White House-generated buzz, trainings, code sprints....

Photo of Dries Buytaert giving keynote at DrupalCon San FranciscoThe State of Drupal keynote that Dries Buytaert gives at every DrupalCon this time focused on Drupal's role in CMS market disruptions ... and RDFa. Where is the web going? Will Drupal lead the way?

DrupalCons have changed quite a bit since my first "major" Drupal event: the 300-person OSCMS gathering back in 2007. Things have gotten pretty sharp, corporate and professional in the Drupal conference world. For some, "corporate" and "professional" are dirty words. But I feel that this professionalism in the Drupal ranks has helped Drupal disrupt some dominant players in the enterprise web market.

Without a doubt, Drupal is changing the web world. And it's happening ... openly.

Profile Tracings

From the kind of swiss-army-knife-like hosted Drupal services like Buzzr and Drupal Gardens to the extremely specialized distributions like Open Publish and Managing News, Drupal is being packaged in new ways, offering new affordances for target markets.

The real power, though, is in Installation Profiles, which are like coded, self-cooking recipes for fully realized Drupal sites.

Combined with Drush Make, which can go and get various modules, themes and libraries from a variety of sources (not just Drupal.org), Drupal Installation Profiles can empower site builders to create complex, configured sites in minutes. The challenge is not in using Profiles for site building: it's in creating the Profiles in the first place. It takes hand coding. There have been some "profile builder" projects spun up here and there, but none seems to have traction today.

Which is why distributions, which already contain all the modules needed, can hold such appeal. (Of course, many distributions are installed and configured via an Installation Profile.)

When I first got into Drupal in 2004, CivicSpace was a popular Drupal distribution designed primarily for non-profit communities. While it rocked if it already did what you wanted, I ultimately grew tired of the large integrated nature of maintaining a distributed version of Drupal plus modules. Inevitably the distribution lagged behind the various module updates and even security releases from the Drupal community, leaving me with the question, "Do I go ahead and update the modules myself? Should I wait for the next full release of the distribution?" As a designer and site builder, I ended up moving away from dabbling in CivicSpace, preferring to work with Drupal a la carte. It wasn't because I disliked CivicSpace; rather I preferred the flexibility of solving my own use cases through my own module selections. But the experience left me with this caveat: a distribution is only as good as its maintained state. It can be awfully difficult to keep up with the Drupal community. So I always recommend caution and consideration before adopting a distribution for a site. [See Dries' thoughts on this from 2006.]

Nevertheless the power of Profiles is real. And the number of Installation Profiles is growing. And Drupal.org can now build Installation Profiles into distributions automatically. Despite having been around since Drupal 5, this is a new frontier for Drupal. Expect this to go places.

And go there openly.

Features, Spaces … Chaos in a Kit

All the amazing site administration power you have in Drupal means that a lot of what makes your Drupal-powered site do what it does is not written in code but stored in the database. And that means it can be very hard to deploy a set of site functionality from one server to another.

Features changes all of that by providing a way to write configuration settings in various Drupal modules into one custom-generated "feature" module that can be distributed as code.

A feature is a Drupal module that provides a collection of Drupal entities which taken together satisfy a certain use case. A feature module differs from a standard Drupal module in several ways:

- A feature is built to target one or more use cases - concrete, task-specific functionality - not address a general problem space.
- The bulk of a feature's work is to provide exported configuration, either in the form of exportables or scripted configuration of components.
- A feature provides no API of its own. Any custom code and hooks it may implement are targeted at addressing user stories.

KIT Feature Specification 1.0

What does this mean? It means, ultimately, the ability to install new features feature by feature, rather than having to install and configure several modules for each feature you want.

The advent of Features has brought about a nascent scattered network of "features servers" hosted here and there. What's needed is some kind of peer-to-peer way to make these findable. Meanwhile there are manually compiled attempts here and there.

It's a new way of sharing in the Drupal community. So expect this, too, to go places. And go there openly.

Build in a Drush

What can I say? Drush is for everyone. This command line tool (don't look way, you designers) saves a ton of time in basic site-building chore of downloading and installing modules. But really that's just the start. Drush (Drupal shell) affords new power in streamlining the Drupal development process, from helping to make (as in Drush Make … okay, sorry) those Features and Installation Profiles to updating modules and even deploying them on remote servers.

If you can type, you can save time.

(My recent post (with video!) walks through the easy steps to install Drush on Mac.)

Of course the code is open sourced. (Thank you, Moshe Weitzman, Greg Anderson, Owen Barton, and Adrian Rossouw [who is truly in fact 1337]!)

Drops in Aegir's Sea

As someone who finds Drupal deployment chores something of a bore, Aegir is the bee's knees.

Point. Click. Voila! A new site is deployed! This isn't site building — it's site provisioning. And it's just about as easy as creating a node on a Drupal site. In fact, that's pretty much what you're doing, in a way, because Aegir itself is built in Drupal.

So when you have your Installation Profiles ready (or found the ones you want), with Drush and Drush Make at hand, Aegir provides the GUI for managing those deployments.

Of course the code is open sourced. (Thank you, Miguel Jacq, Antoine Beaupré, Lior Kesos and the 1337 Adrian (again)!)

As Fast as Mercury

Now I am going to talk about some Distributions.

The whole Pantheon Mercury stack is pretty impressive. I'm not quite sure which is what between Pantheon and Mercury, so I just call it Pantheon Mercury … or simply Pantheon for short.

What it is is what counts.

  • Pressflow, the distribution of Drupal with some nice code optimizations to improve performance across the board.
  • Memcached, the tried and true caching system to reduce strain on the database.
  • Varnish, the kick-ass http accelerator.
  • Solr, the awesome search engine, powered by the awesome Apache Lucene project, that gives us fuzzy search, faceted search, and generally just smarter search than what Drupal offers up by default.

Taking these all together: Can you say screaming performance? We've gone with Pantheon on just two smallish deployments so far, just to try it out.

And it.

Is.

Fast!

And you get SOLR search on top of all that. We love it so much, we're implementing it on two more sites, and Pressflow-sans-Pantheon for a third that is in a more rigid hosting environment.

Of course the code is open sourced. (Thank you, Four Kitchens (for Pressflow) and ChapterThree (for Pantheon)!)

Stay-in-the-community folks should know that Pressflow involves no database schema changes, so you're not really going down a forked path by opting for this Distribution. And the rest of Pantheon is just gravy. You can always take your site later and migrate it back to stock Drupal with no pain. —That is, if you wanted to.

But why would you?

Working in the Open Atrium

I've been a huge fan of Open Atrium since it came out last year. Now in a Beta-6 release, Open Atrium is rapidly becoming a richly featured project that can offer a serious alternative to Basecamp. We use it here for our client support desk, as well as basic ticket tracker and communications center.

Infrastructure is always a potential resource hog and time suck for a web company. The last thing we want to do is have to build and maintain our own homegrown Basecamp. And Basecamp is yet another paid service that, in the end, doesn't quite have the flexibility and power Open Atrium offers. Recently, Matt Butcher tweeted a link to his QueryPath script on GitHub that he used to migrate 3 years of Basecamp data into Open Atrium. It is possible to move on from old ways.

Of course the Open Atrium code is open sourced. (Thank you, Development Seed!)

Just last month the Development Seed team led an effort to provide some truly outstanding documentation on how to use, manage and customize Open Atrium.

Making Drupal Classes

So many of us love Drupal so much, is it time to objectify Drupal? By "objectifying Drupal" I don't mean placing Drupal on some sort of pedestal for self-gratification activities (though I suppose in a way we do that, too), but rather taking the organically evolved Drupal code from its almost biological interconnectedness of everything to everything towards an object-oriented model.

Drupal mud"Drupal mud." Each major part of Drupal core overlaps several other parts. Every change in one component cascades throughout.

Sometimes Drupal gets a little dirty to get things done. And we love it for getting things done. But Drupal can mature now, and perhaps benefit from a focused, high-level, directed approach towards refactoring to take advantage of the great benefits afforded by PHP 5.

Larry "Crell" Garfield described Drupal's current code as "mud" in his DrupalCon SF presentation presenting a vision of Object Oriented Drupal. He throws down a compelling challenge for the future. Drupal 8 is already in development, and starting to classify certain components of Drupal is on the agenda for many ... as part of what is likely a longer-term project.

Object Oriented DrupalObject Oriented Drupal. Clean. Each class contained. With this structure, without all the interlinked entanglements, Drupal core components can evolve more quickly.

Imagine a more modular, OO Drupal.

Letting Data Roam Free

RDF is the future. With Drupal 7, it's becoming the now.

One of the biggest barriers to a truly open, interconnected web is siloed data. Last year at SxSW, Robert Scoble told me that he was seeing increased open API development as the big thing then. And it's been great to see the various closed social networks developing APIs to allow interconnection and integration with other systems. This interconnectedness of separate systems is a start.

RDF is something different. But it can be much more profound, and have an impact greater by orders of magnitude, because it's about structuring data in meaningful ways so that site A can consume structured data from site B without having any access to site B's database. No API necessary for that.

RDF sets data free.

One immediate application of this is that RDFa is now being consumed by Google and other search engines, resulting in more meaningful search results for those sites feeding out RDFa-structured content. The web is ready for RDFa.

Somewhat difficult and time-consuming to implement in Drupal 6, RDFa is part of Drupal 7 core. That means all sites built in Drupal 7 will have RDFa support! (That is, unless the developer or themer disables it out of ignorance, negligence or choice.)

This topic is a particular passion of mine, which is why I'm thrilled to be "mentoring" [read: collaborating with] Lin Clark on her Google Summer of Code project on RDFa.

Of course, all of the RDFa efforts for Drupal are open sourced. (Thank you [for the Drupal 6 efforts], Arto Bendiken, Miglius Alaburda, Ben Lavender, Jeff Miccolis, Frank Febbraro and Stéphane Corlosquet, OpenBand and MakaluMedia. Thank you [for getting RDFa into Drupal 7 core], everyone who participated in these issue queues!)

777

In short, these examples demonstrate that, with Drupal, you can read, write and execute in all kinds of ways.

And as powerful as Drupal has been up through Drupal 6, even more power is unlocked with Drupal 7, which has a bunch of usability improvements, huge steps in the theming system, some important changes in the way the code loads and executes, a new database abstraction layer opening the doors to potentially running Drupal on all kinds of databases, and so much more.

I've seen some big changes in Drupal over the years, but Drupal 7 stands out as the biggest single leap in a major Drupal release. A big part of that is thanks to Angie Byron, who has proved to be cat herder extraordinaire in her role as Drupal 7 co-maintainer.

Everyone can help with Drupal 7 by reading up on what's still needed, writing patches and documentation, and executing patch tests.

And six years from now, we'll perhaps look back at these days as the quiet times in relative obscurity. Because....

Drupal adoption curveGraphic by Dries Buytaert showing Drupal.org activity through 2007. As I write this, the node count is in the 811,000s. My first user account in 2004 was in the 14,000s. Nearly 800,000 users have registered since then to participate in the community in some way. (Okay, I was one of them by registering again [for reasons off-topic from this post].) Throughout this time, from Drupal 4.5 through 4.6, 4.7, 5.x and 6.x, Drupal's adoption as pretty much doubled with each release. Imagine the community doubling in 2010 with Drupal 7!

...growth happens.

You see, the great thing about all these fabulous, powerful things coming out of the Drupal community is that they make developing easier, so that you — we — can spend more time on strategy, on design, on content, and less time on reinventing the wheel or riding the bicycle up that same hill over and over again.

It's a good time to be working and playing with Drupal. Let's make the future together.

We want to work with you!