Creating a local PyPI mirror

During the Blackforest Sprint a new package z3c.pypimirror has been created under the lead of Daniel Kraft. The major goal was to build up a distributed mirroring infrastructure. Why? PyPI is still a single point-of-failure and because there are a bunch of cases where you need an in-house mirror.

After  easy_install-ing  z3c.pypimrror you can use the pypimirror script for downlading all packages from PyPI (and as an option all external packages (experimental)). This requires roughly 4 GB of diskspace. After mirroring PyPI you can run a buildout against your local PyPI mirror. Right now I have full copy of PyPI available under http://pypi.zopyx.com. By default zc.buildout asks PyPI always when looking up a package. You can avoid that by the following configuration in your buildout.cfg:

[buildout]

find-links = http://pypi.zopyx.com

allow-hosts = *.zopyx.com

This configuration tells zc.buildout to also look for packages on my local server and to restrict any lookup to the hosts defined within the allow-hosts section - this works like a charm.

The local PyPI mirror also works nicely with easy_install:

easy_install -i pypi.zopyx.com