Crate is a PyPI Mirror/Python Package Index that was written to make it easy to discover packages, evaluate them for usefulness, and then install them. Additionally it also focuses on presenting an extremely stable interface to PyPI compatible applications (e.g. pip).
With Crate you should be able to quickly find, evaluate, and install your packages with no worries about if something is going to be down or not.
Discussion about crate can take place either in #crate on freenode or on GitHub Issues.
Crate.io is built on top of Python using the Django framework. It uses Celery to process its shared tasks, PostgreSQL to store its data, and Redis as its caching layer.
The software that powers Crate.io defaults to PyPI, but can technically be used with any index that presents the same XML-RPC API.
Crate.io the website currently only mirrors PyPI. All packages and their associated data come from PyPI. Crate.io provides a reliable place for those packages to be stored and accessed and tries to present a cleaner and more user-friendly interface to that data.
On OS X one possible way is to use Homebrew:
brew install postgresql9 initdb /usr/local/var/postgres9 mkdir -p ~/Library/LaunchAgents cp /usr/local/Cellar/postgresql9/9.0.7/org.postgresql.postgres.plist ~/Library/LaunchAgents/ launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
There are some alternatives to Homebrew for installing PostgreSQL at http://www.postgresql.org/download/
And create the PostgreSQL database:
createdb crate
For searching packages to work, you will need elasticsearch. I installed it via Homebrew:
brew install elasticsearch
Then create and activate a virtualenv -- everybody has their favorite way;
here's what I did (mkvirtualenv comes from virtualenvwrapper), inspired by dstufft's gist:
mkvirtualenv --no-site-packages --distribute crate-site echo "export DJANGO_SETTINGS_MODULE=crateweb.conf.dev.base" >> $VIRTUAL_ENV/bin/postactivate echo "unset DJANGO_SETTINGS_MODULE" >> $VIRTUAL_ENV/bin/postdeactivate workon crate-site # virtualenv is already activated by mkvirtualenv but have to refresh postactivate
Install dependencies:
pip install -r requirements.txt
Do the Django dance:
export LC_ALL="en.UTF-8" # Otherwise I get "TypeError: decode() argument 1 must be string, not None" in django.contrib.auth.management python manage.py syncdb python manage.py migrate python manage.py runserver
Admire the results of your work at http://localhost:8000/:
open http://localhost:8000/
You won't have any packages in your database which makes testing harder. A full sync from PyPI takes some time, like 2 days or so the first time but you can start a full sync and just stop it after 15 minutes or so and you'll have some packages.
To start downloading packages from PyPI into the local database:
python manage.py celeryd python manage.py trigger_bulk_sync
Once you have some packages, then you can build the search index so that you can search them:
elasticsearch python manage.py rebuild_index
Say yes to blowing away the index since you don't have one yet anyway. Indexing 1073 packages took a few seconds for me.
@@@ Todo Write More Development Docs?