December 20, 2006

ZopePageTemplates going Unicode

In Zope 2.10 we replaced the ZPT implementation with the one from Zope 3  (with some additional wrapper code). The Zope 3 ZPT implementation is pretty much clean and unicode-aware. So the next step is to make Zope more aware of unicode. For that purpose the Zope 2.11 implementation will use unicode as internal representation for the ZopePageTemplate file class (which is used by ZPTs stored within the ZODB). For output purposes (WebDAV/FTP) each ZopePageTemplate instance now has an output-encoding property that controls the conversion from unicode. While processing a ZPT everything happens on the basis of unicode. It is now up to the ZPublisher to convert the rendered output to some other encoding. This encoding is either given through the Zope default configuration or set by the application itself by using RESPONSE.setHeader('content-type', 'text/html; charset=xxxx').

However Products.PageTemplates also contains PageTemplate and PageTemplateFile classes. I think it does not make sense to enforce unicode directly inside the classes. The PageTemplate class for example is used by CMFCore.FSPageTemplate. It makes more sense to convert the data inside  these classes to unicode instead of messing up the base classes.

Near term goals:

  • backport  the latest ZPT changes to Zope 2.10.2 since Zope 2.10.0 and Zope 2.10.1 are partly broken concerning encoding issues and WebDAV/FTP support
  • making CMFCore.FSPageTemplate unicode-aware (you should be able to specifiy the encoding of a filesystem-based PT through its .metadata file)
  • checking how the ZPT unicode-awareness affects Five