LPTHW Exercise 46

In this exercise, we create a skeleton directory for future Python projects. This skeleton basically contains a directory structure, and an install script.

For all this to work properly, we need to have certain Python packages installed. The book recommends the following packages.

pip
distribute
nose
virtialenv

Now I am going to install all of the above packages.

Installing Pip:

pip is a Python package installer (which installs packages from the Python package index). It is meant to be a replacement for easy_install. So anything which can be installed with easy_install can also be installed with pip.

The pre-requisites for installing Pip are either setup_tools, or distribute. I read that setup_tools is not supported on Python 3.0. Even though this is not a problem, since I have Python 2.6, I figured it would be a good idea to go with 'distribute'

Distribute is also on the list of packages to install, so I install 'distribute' first.

curl http://python-distribute.org/distribute_setup.py | python

But I am also a bit curious about what distribute does. The documentation says it is a tool to 'Easily download, build, install, upgrade, and uninstall Python packages'. So basically this looks like a lower level tool which is needed by pip.

Now that I have installed 'distribute' I also installed Pip.

Installing VirtualEnv

Next I installed 'virtualenv' using pip:

'sudo pip install virtualenv'

I think there is a lot to virtualenv. I will write more about it in my next blog post. For now, I will link to the official virtualenv documentation page.

After installing virtualenv, I created an environment called ENV1.

I will now try to install nose in ENV1.

I activated the newly created environment running source bin/activate and then ran pip install nose.

Extra Credit:

So what do all of these software's do ?

1. pip - Is used for installing packages which are present in the PyPy package index
2. Distribute - Is a lower level tool for building, installing, and managing Python packages
3. VirtualEnv - Is a tool to create isolated environments for Python
4. Nose - Extends unittest to make testing easier

Understand setup.py

setup.py is used by the Python distutils as a standard way for installing third party Python modules. Before distutils, module creators would have to create install files for all the different platforms. Off course this is not easy, and I am sure there would be times when some module creators would not create such files. I believe since distutils uses Python, it is a way to use Python to install modules, rather than relying on the platforms way of installing Python packages. I also think this is good encapsulation :-)

If you are using distutils then you will know immediately if the package maintainer has packages their application in standard way or not. A Python package which is meant to be installed using distutils will always have a setup.py file bundled with it. This file will contain the calls it makes to distutils to install the package.

When we run

python setup.py install

There are really two things that happen. First is the build step, which puts all the files which need to be copied into the distribution root's build directory. I think by distribution root they mean the directory which is created for the distributed package. We can however change the directory in which the build process happens, if we so desire.

Once the build phase has completed, the install phase will copy the files to the chosen install directory. If no directory is specified then the default directory, which on *nix systems is usually /something/pythonX.x/site-packages. On my Ubuntu system it is /usr/local/lib/python2.6/site-packages However, it seems that modules are actually being installed in dist-packages.

It is possible to install the package using an alternate install scheme. There are various alternate install schemes. Even though I have not looked into them in any great detail, they seem to be able to not only customize the install location, but also determine who is able to access and use the package. Details for all of these install schemes is available here.

I will stop here as far as using disutils is concerned. This is a good introduction, and should be fine to start with.

Now I will understand the basics of using distutils for package creators.

Since distutils is also about packaging, let us try to understand some basic terminology. We will talk about general Python terminology followed by distutils terminology.

1. Module - This is the basic unit of code-reusability in Python. It is a block of code, which is usually imported by other code to be used. This is usually a single Python file.
2. Package - This is usually a module that contains other modules. It is usually contained in a directory. This directory must always contain a file called __init__.py
3. Root Package - This is the root of the heirarchies of packages. The root package usually contains the entire Python standard library, along with many stand alone modules which do not belong anywhere else. Now the documentation says something which totally freaked me out ... "Unlike regular packages, modules in the root package can be found in many directories: in fact, every directory listed in sys.pathcontributes modules to the root package"

distutils specific terminology:

1. Module Distribution is a collection of Python modules and packages, which are meant to be distributed en-masse.
2. Distribution Root is the directory which contains setup.py

Now let us understand the setup script from the perspective of a package creator. The main purpose of a setup script is to describe your module to distutils, so that the various commands that operate on your module do the right thing.

Without getting into too much detail, I will now create a simple module distribution and create a setup script for it. I will write about it in a separate blog post.