Towards a generic integration of external data sources into Plone

Over the last days I blogged several times (link, link) about the ongoing integration of Plone with XML databases (eXist-db in particular as part of an ongoing XML publishing project with Plone). The integration is based on the WebDAV API of eXist-db and my module zopyx.existdb that integrates the hierarchical structure of eXist-db directly into Plone (similar to the Plone Reflecto add-on by Wichert Ackermann). Because of the integration layer is WebDAV it was almost trivial providing the same integration support for the XML database BaseX. But wait...there are other services providing access through WebDAV. During the Plone Conference in Bristo l gave OwnCloud a quick try and it was easily possible to mount OwnCloud (or a subfolder) into Plone for reading and writing. But then Asko had the crazy idea to try to mount Plone within Plone through WebDAV and it actually worked. Here is a screencast where plone.org/events was mounted into my local Plone sandbox. Just saying: the integration with other services works best for binary stuff like images, videos, office documents etc. It is not very suitable to access Plone test-ish content because Plone provides such content either rendered as  complete HTML pages through the HTTP port or as document source over the WebDAV port. There might be options to deal with this some way but this is actually not the first scope of my project. So my eXist-db integration got a more wider scope: providing a unique way for attaching external data sources into Plone.

After a glass of wine I had the idea to turn the existing code into a more generic module - codename "mr.mount".  On the implementation level I am using the pyfilesystem module that provides a generic and unified API for hooking arbitrary filesystem services into Python (as long as there is a driver of the FS). Fortunately pyfilesystem provides out-of-the-box support for WebDAV, local filesystem, SFTP or even AWS S3. So supporting all these filesystem services in Plone should not be that hard.  I evaluated Dropbox access some time ago with an older version of pyfilesystem and it worked nicely (apart from issues dealing with the OAuth token from Dropbox). After some research I found two drivers for SMB/Samba for pyfilesystem...so supporting Dropbox and Windows/Samba shares in Plone appears feasible.

To make it clear: the current approach is not a replacement for the ZODB and not another storage layer for Archetypes/Dexterity. It is all about providing a smooth integration of external data sources. The core implementation right now provides traversal support from Plone into those external services including basic UI support for managing folders/collections (add, rename, delete), for up- and downloading content, ZIP export/import and a basic integration with the ACE editor. I am sceptical about a more deeper integration into Plone as it was implemented in Reflecto. I do not think that external content should be treated in a way a primary Plone content. So I would hestitate providing some kind of indexing support because that did not work out well in Reflecto but I am open to suggestions.

These are my ideas about mr.mount and the integration of other services into Plone. The implementation of mr.mount is not of high priority since my basic goal of talking with Plone to eXist-db is perfectly working for my project. If you are interested bringing this idea forward (with suggestions and funding), please get in touch with me.