planet

I'd like to share some of my thoughts and long-term visions for Drupal 8 and beyond:

1. Full CRUD for the Entity API

In the long-term I want to see the Entity API becoming our main CRUD-API, on which modules may build upon. For D8 I do think for every entity should be based upon a class implementing the "EntityInterface", which provides some simple methods to easily access identifiers, revision ids, labels, uris as well as save(), delete(), .. methods.

2. Improved DX for fields

Now, we have two kind of entity properties: Fields and non-fields. So should one use an entity property or a field? We have some nice APIs around fields, but they are not built for daily developer usage so programmatically re-using fields is no fun. Still, developers can go without a field for any custom data storage, but then we are loosing all the advantages fields come with - like flexible storage or the awesome module support (which I've tried to solve in D7 via hook_entity_property_info()). Once we have improved DX for fields in place, developers can easily embrace it and benefit from its advantages.

3. Everything is a field

So why not adopt fields for any entity property? So we could make entity properties easily translatable via the field API and benefit from the improved out-of-the-box module integration and stuff already written for fields, like widgets and formatters . Of course, some fields need to be hidden from the UI then.

4. Storage APIs

With everything being a field, entity data would be scattered around in lots of db tables. Also, it should be possible to use the API to register any remote data object as entity. So we need to have entity-storage and field-storage backends, such that also the remote-data-entity can have fields stored in the local database. Thus, with everything being a field we need to allow developers to delegate field-storage to the entity.

5. Describing data

Also, as of now field types have to describe the db schema to be used for saving. However, the schema API is built for the database system so it has no notion of describing stuff beyond it, like that a timestamp is a date. So maybe the contract between the storage API and the field system should not be the db schema, but an actual description of the data to be saved. I.e. instead of telling the system to save an integer which will be used for saving a node id, tell it that it has to save a reference to a node. Apart from that, the described data structure is what other APIs built around fields (widgets, formatters) have to use (or should use -> query), just as any developer working with fields. So simultaneously, modules making use of entities and their fields could rely on that information, e.g. to determine all entity references or just to get some data values of a certain type, e.g. textual values for token replacements.

6. Profile2 in core

With 1) in place, it should be rather easy to replace our old profile module with something new built upon entities and fields like profile2. We'll see how profile2 does for d7.

7. Rules in core

I'd really like to work on bringing a slightly simplified version of the foundational API of rules into core, thus re-placing the current action system. However with Rules 2.x the whole API is built around the way of describing data utilized for hook_entity_property_info() as well as on entities. Thus for Rules in core making sense, it would need something comparable in core - e.g. point 5).

After about a year full of work around Drupal 7, Rules & Web services I recently graduated my master studies "Information & Knowledge Management" at the Vienna University of Technology. As finally my thesis will be presented officially with others at this year's epilog, I thought it would be a good time to share my thesis to the public.

The thesis title is "Event-Condition-Action rules for distributed content management", thus I've worked on a re-architectured version of Rules that is able to work across system boundaries, e.g. multiple Drupal installations. That resulted in Rules 2.0 as well as stuff like the Entity API being created - but better, it finally made it possible for me to fully concentrate on working on Drupal - awesome!

Thesis abstract: For the popular open source Content Management System (CMS) Drupal the Rules extension module makes it feasible for users to configure reactions on a high level without requiring any programming expertise. To achieve that, the extension allows for the specification of reactive rules - or more precisely it leverages Event-Condition- Action rules, for which the actions are executed when a specified event occurs and the conditions are met. Finally the ability to create custom reactions constitutes an opportunity for users to rapidly adapt the behavior of Drupal based web applications. In this thesis the existing Rules extension module is analyzed and revised in order to obtain an extensible and reusable solution, for which all identified flaws have been eliminated. Moreover the module is advanced to work across system boundaries, such that it can be utilized for the rule-based invocation of web services as well as for reacting on remotely occurring events. Therefore the solution obtains the ability to work with arbitrary data structures with the help of metadata, so that the data of remote systems can be seamlessly integrated based on metadata. Building upon this capability we present the rule-based utilization of RESTful and WS* web services and introduce Rules web hooks - a novel approach for the interaction of Drupal based web applications that exploits reaction rules to enable custom near-instant reactions on remotely occurring events.

Get the full-text of the thesis at https://wolfgangziegler.net/thesis. At this point, let me thank my employer epiqo for sponsoring this work, as well as my academic advisors O.Univ.Prof. Dipl.-Ing. Dr.techn. A Min Tjoa and Mag. Dipl.-Ing. Dr. Amin Anjomshoaa of the Institute of Software Technology and Interactive Systems, who already did a great job supporting the Drupalcamp Vienna 2009.

Update 10.01.2011: In the meantime the Entity metadata module got merged into the main "entity" API module.

---

Drupal 7 modules working with entities often face the same problems:

• How to create/save/delete an entity?
• How to get referenced entities?
• Which properties are there and how can they be accessed or modified?

This is, what Entity Metadata tries to solve for Drupal 7. It collects metadata from modules, such that it knows how this things can be done and provides API functions for that purpose. There are API functions for full entity CRUD, for determining access, as well as data wrappers that simplify dealing with entity properties.

Metadata for data properties, why that?

You might think, we have fields. Yes we have, but not everything is a field. There are also entity properties, like the node title and author, the term hierarchy and lots of others. Entity metadata collects information about all that available properties - regardless whether they are fields or not - and makes them accessible the same way. For that you have to provide property info via a hook, e.g. this is the info the module provides for books:

DrupalCon San Francisco was an amazing DrupalCon - about 3000 attendees!!! I had really great luck to be there at all, we passed the volcano ash by one day! Sadly lots of European Drupalistas had not so much luck :( I really enjoyed the conference, in particular the core developer summit & the code sprints. Gladly the lightning talk I submitted for the core developer summit was selected, so I gave a short talk about Drupal's data APIs - Why inhomogeneous APIs suck - clearly entities are the way to go!

Wouldn't it be nice have your changes up and running at a development site just by pushing it there with git push dev? Well with Git that's pretty easy to achieve. First create a new Git repository on the remote if you haven't yet: Then push your code to this repository initially and then check it out: So now we need to make Git automatically checking out the latest code once you pushed it in. For that create the script ".git/hooks/post-receive": And don't forget to make the file executable. Now as the code is automatically updated, we need to disable the usual warning when one pushes to a remote repository with a checkout: That's it. Simple and useful, not? :) Credits go to http://toroid.org/ams/git-website-howto, which I used to come up with this.

Some months ago I gave git a first try for developing some stacked core patches. I quickly get used to it and to its nice features (german). Luckily its SVN integration is really nice and simple to use so GIT is even a fine replacement for the usual SVN client. Unfortunately for CVS things are much worse.. :( Poor cvs!

First tries..

I decided to don't go back to CVS for maintaining drupal modules. First I tried using the public GIT mirror of the whole drupal CVS from the French drupal community, however I quickly noted that it was broken for the rules module, so I tried to set up my on mirror. I started using Sam Boyer's scripts, however I had some misc small troubles with them, but more important those scripts copy the whole drupal CVS - but I didn't like to waste ~6 gigs of disk space and the time to sync all the unneeded stuff. So I ended writing my own script based on the instructions in the drupal handbook for maintaining a module with git and Sam's script. After some weeks going back and forth I finally ended up with something useful, so here are my experiences:

What is it about?

Currently I've set the script up to export the drupal CVS of some of my modules to GIT - its automatically pushing to my github account, which is quite convenient. It syncs only the really needed modules using rsync - which is an important step as its greatly speeding up cvsps when exporting from CVS. The nice thing is that it doesn't overwrite any changes not yet in CVS, so I can use the same git repository on github to develop new stuff and export to CVS later on.

With drupal 7 the concept of an entity gets introduced in drupal, thus nodes, comments, users, terms - all these are drupal's entities. Most important the also new field API is able to support any entity! So how will this affect the drupal landscape?

Preliminary Warning: "If you're afraid of classes and objects in PHP, run away now." - jpetso. Update: For simplicity the API has been changed so that all faces are incorporated, thus modules have to care about naming collisions theirself. Also in the meantime file inclusions support has been added. While figuring out a object oriented design for the rules engine I recognized the need for a possibility to allow modules extend objects in various places. Thus I developed a generic concept which does just that: Allowing modules to extend objects. I called the concept "Extendable Object Faces", which basically implements the Facade pattern in a modular way. So let's have a closer look at that. A module that wants to extend an object may only do so on top of a defined interface, preventing uncontrolled growing objects. Thus one can write some code, define an interface for it and attach it to potential any extendable object out there. Then other modules have an easy way to test whether some object has a functionality in place by checking for the availability of a certain interface. To use that functionality the caller has to use the right "Face" of the object - corresponding to a certain interface. This approach using different "Object Faces" make sure there can't be conflicting method names, so modules don't need to prefix their methods with the module's name which would result in ugly and not readable code. I've already implemented an initial version of the Extendable Object Faces API. As of now you can:

• Extend an object by providing an Extender Class
• Extend an object simply by some functions, each implementing a method
• Override any dynamically added method by providing functions or an Extender Class

So apart from "allowing modules to extend an object", this also allows one to:

• Easily lazy load huge parts of an objects implementation, invisible for the caller.
• Allow modules to dynamically alter an implementation by using overriding.

Hence such an Extended Object can also serve as a clean abstraction for module provided callbacks! Enough talk, let's show how it works: This code provides the class for the Extendable Object and already defines an interface, which modules may implement. So let's extend it with an Extender Class. object->name; } } ?> That's it. Now you can use the face! name = 'test'; $element->extendByClass(array('FacesTestInterface'), 'FacesTestExtender'); // Now use it. print $element->face('FacesTestInterface')->isWorking('Name:'); // This prints "Name:test". ?> As you see, one has to use the face to be able to use the added method - so potential name collisions are avoided. However the Extendable Class can define so called incorporated faces, which are built in the Extendable Object as soon as a module provides an implementation: So you can use it that way, once extended: isWorking('Name:'); ?> You can find more examples in the simpletests I've written. Also checkout the benchmarks I've run. So what's missing? Basically the implementation is complete and working fine. Though feedback and suggestions are very welcome! However currently the code doesn't deal with including code where the implementation can reside (lazy loading), though for drupal 7 that is already solved by the code registry (just add one call to drupal_function_exists()). For drupal 6 I think about adding the possibility to specify include files per added interface.

Dreaming...

If we would have an object oriented "Data API" in drupal core, this could serve as a way to let modules extend those objects. So instead of writing node-centric code code could would be written in a generic way - attachable to potentially each of those data objects (users, comments, terms). Having such generic code would allow us to finally end the "Everything should be a node" debate. Apart from that this would help us to easily lazy load big chunks of code, just exploiting the code registry!

Finally, after over 1 year of development the rules module has reached the 1.0 version! You can download it from the project page. Reactive rules? Reactive rules (or ECA-rules) are rules triggered upon a given event. This allows one to conditionally fire actions based on arbitrary events. As modules can provide further conditions, actions and events this enables site builders to automate a lot of things using reactive rules! As of now a lot of popular drupal modules provide already rules integration: CCK, Organic Groups, Token, Flag, Content Profile, Content Access, ViewsBulkOperations and many more. Features? The modules comes with a bunch of useful features, e.g. a flexible input evaluation system that allows to use token replacements or even PHP evaluation in your rules. To ease the management of rules the module supports tagging of rules as well as Import/Export. Often used behaviour can be put into Rule-Sets and is easily invoked by provided actions. Not enough, the execution of those Rule-Sets can be scheduled easily with help of the provided action, thus providing a powerful scheduling system, which allows you to schedule arbitrary tasks! It's finally grown-up? Rules is already stable for quite a time, however I didn't consider it to be complete. During the last months I added missing important features, fixed bugs and translation issues and completed the documentation! While there was quite a bunch of useful developer documentation for a while, now there is also a complete hook reference in the doxygen format. So it's grown up, but it's not (yet) perfect. So what comes next?

Finally the flag module got released - a really useful module and known as the successor of the previous "Views bookmarks" module. The project page says: Flag is a flexible flagging system that is completely customizable by the administrator. Using this module, the site administrator can provide any number of flags for nodes, comments, or users. Some possibilities include bookmarks, marking important, friends, or flag as offensive. With extensive views integration, you can create custom lists of popular content or keep tabs on important content. But even better the 1.0 (and 1.1) release is shipping with extensive rules integration. So once you have installed both modules you can create flags and use rules to add some behaviour to them, e.g. you can react once a flag is "flagged" or "unflagged" and send mail to the content node's author, unpublish it, change the content's access settings - or whatever you have actions and conditions for! Then the module also provides actions for flagging or unflagging flags, checking the flag count in case of global flags and even to "Trim flags", which allows you to restrict the number of flagged items. This way you could even create simple queues. Of course there also conditions to check whether a flag is flagged and again in case of a global flag you can check the flagging count. Amazing! Also the flag module comes with a bunch of useful settings, so you can configure the flag to have a confirmation form or to just work with an AJAX link. Together with the powerful rules integration one could implement a lot of useful stuff - want a custom link with custom permissions to publish a content node? Just use a flag and an appropriate rule!