Blog of Andreas Jung, ZOPYX

FileSystemStorage for Plone

Andreas Jung -

Plone still has a poor support for storing large files. We all know that the ZODB badly supports the storage of large file (BLOB support within the ZODB is coming forward but it is not ready for prime time). For recent project we had the requirement to build a multimedia database as part of a Plone application. The database should be able to store several 100K  objects (images, media files) inside Plone. We choosed FileSystemStorage from Ingeniweb because it fits perfectly into the storage layer concept of Archetypes. Creating new types (derived from ATFile or ATImage) using the filesystem for storing large content could be implemented very easily.

Adding FSS support to your own content-type is very easy:

def updateSchema(schema):
    field = schema['file']
    field.storage = DirectoryFileSystemStorage()
    field.registerLayer('storage', field.storage)

class MyFile(ATFile):

    schema = ATFile.schema
    updateSchema(schema)
    .....

FSS by default supports only one Plone site per Zope instance since it stores the data as flat hierarchy directly in the var folder. We extended FSS in the following ways:

  • support for multiple Plone sites within one Zope instance
  • two-level deep directory structure based on the objects UIDs
  • configurable storage prefixes
  • working copy/cut/paste support
  • generated RDF contains more metadata (review state, effective, expires dates)

Most of the work was done by Simon Pamies (Applause).

The code is not fully polished. Since we were only interested in the FSS backend functionality we don't made any efforts in order to work on the Plone UI of FSS. There is a big chance that this functionality is broken. However the backend code is working fine even for some 100K objects.

A snapshot of the code is available from ftp://ftp.zopyx.com/open-source/FSS.tar.gz