entity metadata

A New Entity Property API for Drupal 8

Submitted by fago on Wed, 08/08/2012 - 11:33

Why?

While the conversion to full CRUD for entities and classed objects for Drupal 8 made good progress, we’ve not yet reached the goals of fully translatable entities and having a default entity serialization for import/export, content staging and web services. For those points we need to know what’s in an entity! Yes we can look up the fields, but there are also base entity properties and further stuff modules put on entities, which both are no fields so we do not know anything about them.

That is what the new Entity Property API is supposed to change: It’s about providing a unified interface to entity properties *and* fields, avoiding the split between fields and non-field properties. So fields will be entity properties working with the same API as well! Then, it’s about being able to instrospect what’s going to be in an entity - regardless whether it has been added by a module or by the user via field UI. But furthermore, with classed entity objects as the basis we can improve the developer facing API and make it easier to access property and field values (I’m looking at you, $entity->field_body[LANGUAGE_NONE][0][‘value’]).

What?

As we’ve outlined in the WSCCI Web Services Format Sprint report we want to have a modern OOP, interface based API for properties. The API should allow for introspection, so that it’s possible to look up the defined properties of an entity type in advance. If you know the Entity property information system of the Drupal 7 Entity module, it’s a bit similar to that, but instead of adding an extra layer above the existing entities it’s about supporting it natively. So there will be no need for entity wrappers - all the easy API and information will be directly available in $entity.

See the Entity Property API meta issue for a more complete list of planned features.

Drupal 8: Entities and data integration

Submitted by fago on Fri, 03/11/2011 - 19:41
As follow-up to my previous blog post Drupal 8: Approaching Content and Configuration Management, I'm going to shortly cover how the Entity API could help us with two more of Dries' points: Web services and information integration. First off, for getting RESTful web services into core, having a unified API to build upon makes lots of sense. That way we make sure we locally have the same uniform interface for CRUD functions available as we expose it to the web. But moreover, the possibility of having remote entities can help us a lot with integrating with remote systems. In a way, we'll get that anyway once we implement pluggable entity storage controllers (and you can even do so already in D7). But for that really being useful, we need to know the data we want to work with. This is why, I come up with the hook_entity_property_info() in the Entity API module for d7. While for d7 it is built on top of the stuff that is there anyway, I think it should play a much more central role in D8 for various reasons:
  • A description of all the data properties of an entity enables modules to deal with any entity regardless of the entity type just based on the available data properties (and their data types). That way, modules can seamlessly continue to work even with entities stemming from remote systems. This is how, the RestWS, SearchAPI and Rules modules already work in d7.
  • With pluggable storage backends, I see no point in SQL-centric schema information except we are going to use SQL based storage. By defining the property info, storage backends can work based on that though, i.e. generate the sql backend can generate the schema out of the property information.
  • When working with entities, what bothers is the data actually available in the object. To a module, the internal storage format doesn't matter. In a way, the property information defines the "contract" how the entity has to look like. Given that, all APIs should be built based on that, i.e. efq should accept the value to query for exactly the same way it appears in the entity and not in storage-backend dependend way (no one can predict once storage is pluggable). The very same way display formatters and form widgets could just rely on the described data too.
As we discussed in the entity API bof, it might make sense to build it around "field types" evolving to "data types" - thus being decoupled from being a "field". The very same way we can start building the display components around the entity properties, thus not necessarily being based on fields (bye bye field api "extra fields"). Any way the implementation will look like, let's let the entity API become our long missing data API!

Restful web services in Drupal 7

Submitted by fago on Mon, 01/31/2011 - 14:07

During the work on my thesis over the last year, I played around a lot with RESTful services based upon the Entity API. What I needed was a simple service that just exposes Drupal's entities in a RESTful manner, while obeying Drupal's permission and access systems. Now, me and klausi have created a small module that does exactly that: Restful web services.

So how does it work?

The module makes use of the Entity API and the information about entity properties (provided via hook_entity_property_info()) to provide resource representations for all entity types (nodes, comments, users, taxonomy terms, ..). It aims to be fully compliant to the REST principles. Drupal's entities are exposed at the unified $entity_type/$id paths, while respecting the Content Accept/Content Type headers of the HTTP requests. That means if a client requests node/1 with usual HTTP accept headers it will get Drupal's usual output, if it requests node/1 while accepting only JSON, it will get the JSON representation of the node. Similarly, all CRUD operations are supported as common for RESTful services. Then, the module supports GET requests on paths like node/1.json, node/1.xml or node/1.rdf too.

And authentication...?

As mentioned above, the solution just obeys Drupal's permission and access system. If there is an active session and the user has sufficient permission for the request, it will be served. So any add-on authentication strategies would have to plug into Drupal's usual user system. For example, the RestWS module comes with a small add-on module that authenticates users via HTTP basic authentication. So you can define a regular user for a client, configure their access permissions as usual, and just pass its credentials with a request.

So what about the property information?

The module makes use of the property information the entity API collects for all entity types, as well as the accompanying wrapper classes. While the API also allows providing non-entities as resources, it requires the existence of property information. Representations of entities are provided according to their property information. What does that mean?
So let's have a look at an example: The node author. In the property information about nodes, there is no uid property, instead there is an 'author' property, pointing to the according user entity. So the module makes use of that information to output a proper reference to the author, being the author's URI (URIs are the proper way to do references in RESTful designs). So instead of just outputting user id as uid property with an integer value, we output a proper reference to the node's author. Apart from that, the property information includes access permissions - so updating the node author will only be possible if you have sufficient permissions.
Then the property information could be used to provide a description of the web service for the caller, in a human as well as in a machine-readable way.

Which formats are supported?

The module currently comes with support for JSON, XML and RDF/XML whereas modules may add more formatters. As the property information is available to the formatters too, it's possible to do formatters that output some properties in a certain way, e.g. using a special XML namespace. Similarly the RDF formatter looks up the RDF mapping being defined for a property, in order to generate meaningful RDF output.

What's different to the Services module?

The main differences are:

* RestWS provides only RESTful services (no message-oriented or RPC-style web services like SOAP, XML-RPC etc.).
* RestWS strongly builds upon the Entity API and its property information, thus utilizes it for CRUD, access checks, getting property information, ..
* Property information is built into the API, so formatters may make use of it to format the data in a sensible way.
* There are no "service endpoints" to configure as resources are just available at uniform paths like node/1, user/1. We do not see a need to have multiple endpoints for the same resource in a RESTful desgin.

For more about the relation and partial overlap to the Services module, read and participate in the discussion over at http://drupal.org/node/1042512.

Metadata, what for? - Introducing Entity Metadata!

Submitted by fago on Wed, 08/04/2010 - 13:20
Update 10.01.2011: In the meantime the Entity metadata module got merged into the main "entity" API module.
Drupal 7 modules working with entities often face the same problems:
  • How to create/save/delete an entity?
  • How to get referenced entities?
  • Which properties are there and how can they be accessed or modified?
This is, what Entity Metadata tries to solve for Drupal 7. It collects metadata from modules, such that it knows how this things can be done and provides API functions for that purpose. There are API functions for full entity CRUD, for determining access, as well as data wrappers that simplify dealing with entity properties.

Metadata for data properties, why that?

You might think, we have fields. Yes we have, but not everything is a field. There are also entity properties, like the node title and author, the term hierarchy and lots of others. Entity metadata collects information about all that available properties - regardless whether they are fields or not - and makes them accessible the same way. For that you have to provide property info via a hook, e.g. this is the info the module provides for books: