LPTHW - Exercise 52

This is the last exercise in the LPTHW book. In this exercise we will create a web based game engine for the Gothon game (the one we created in exercise 42), using the structure we created in exercise 47, along with tests and everything.

The first thing I did was to copy the entire contents of ex47 to ex52. Then I renamed game.py to map.py and made a corresponding change for the test case also. Then I ran 'nosetests' and everything seems to be fine at this stage.

In exercise 42, each room was a function in the Room class. But now in exercise 52, we will make each Room an instance of the Room class. I coded the class map.py (containing all the rooms) as shown in the book. Next, I am going to copy the map_tests.py file as is in the tests directory.

After copying the code in map.py, I created another Python module for messages, called messages.py. This module will contain the initial text for all the rooms, as well as the transition text, which is displayed to the user when they transition from one room to another.

Next, I refactored map_tests.py to test all the rooms, their initial text, and transitions.

The book then explains how lpthw.web maintains sessions. I understand how sessions are managed, but the session variable which seems global to the module confused me, because it makes me think that only one instance of that variable would exist.

I copied the apps.py file in bin, and also fixed a couple of bugs.

I also added two HTML templates - layout.html, show_room.html, and game_lost.html

Part of the code is embedded below:

LPTHW - Exercise 51

In this exercise, we learn how to deal with form input and a bit about testing web apps.

The first thing I did was to change app.py and refactor it to handle web input. We will handle simple input as GET input parameters to start with.

Here is the new app.py file:



Now if you notice, in this file we use the line

form = web.input(name="Nobody")

to handle the input. If the input contains a 'name' parameter, then we will use it, else it will default to "Nobody", since we have given a default value as an argument to the function call.

Getting the values as url parameters is not a good idea:

This is a good beginning, but we should really be getting the values in a form and not as command line parameters. For this we will need to add some HTML and a method in the Index class to handle POST data (since this is the default method for sending form data).

I made a few changes to app.py, index.html, and added a new file hello_form.html All the code is embedded below:



Don't repeat boilerplate code:

It's very good that we can now handle form data. However, the observant programmer must have noticed that both index.html, and hello_form.html have common boilerplate code. It would be good if we can remove the repetition. Well, we can, if we use templates.


All the code which uses templates is embedded below:

LPTHW - Exercise 50

In this exercise we will make a web application from the Gothons game, which we made in exercise 46. To simplify the process of making a web application, we need a we need a web framework. For this exercise we will use lpthw.web, which is the same as the webpy framework.

I installed the web framework thus
pip install lpthw.web

Then I created a project for the web application by creating a directory in my projects directory:
ex50
/bin
app.py
/docs
/gothonweb
__init__.py
/templates
__init__.py
/tests

I just copied the code provided in LPTHW in app.py

I ran the webserver with 'python bin/app.py' and successfully loaded the url http://localhost:8080

However, I realized that lpthw.web was not installed in my virtualenv as I would have expected it to be. It was actually installed in /usr/local/lib/python2.6/dist-packages/ This seems strange and incorrect to me. I am not quite sure what went wrong, so I asked a question on StackOverflow.

The code for this exercise is embedded below:

LPTHW - Exercise 49

In the previous exercise, we had created a scanner, which would take a line of words as input and return to us a list of tuples which gave us the type of each word.

In this exercise, we try and add some intelligence to our game, and identify sentence parts in the user input.

This is a bit of an over-simplification, but we will consider a sentence to consist of the following structure: subject verb object

If our sentence contains words from our lexicon and is of the above form, then we will be able to parse the sentence and take an action in the game based on it. We ignore stop words, and also allow for arbitrary upper/lower case. This gives the player a decent amount of freedom in creating their commands. So for example they could enter "eat bear", or they could also enter "Eat the bear" and both the inputs will be accepted as valid by the sentence parser.

For this exercise I copied the code from ex48 because we still need the lexicon and lexicon_tests files. In addition I added the files sentence.py and sentence_tests.py

The entire code is embedded below.

LPTHW - Exercise 48

In this exercise, we created an intelligent input scanner, which we can use in our games. This scanner takes a line of input, and returns a list of tuples, each containg the 'type of word' and the word itself.

We were given the unit tests (lexicon_tests.py), and were supposed to write the actual code (lexicon.py) to fulfill the tests. This exercise was a lot of fun.

The test suite and code is embedded below:

LPTHW - Exercise 47

In this exercise we create a very simple class for a game, and then create unit tests for it.

Since, I have already done much programming (albeit in Java), I am familiar with the concept of unit testing, and why it is important.

Python seems to have something called 'nose' which is a test harness similar to JUnit for Java. Here is the production and test code modules.



In our project, we have a directory called tests, which holds all our tests. The test module name is 'actualmodule_test.py', where 'actualmodule' is the name of the module which is being tested.

The test script has setup and teardown functions, but when I ran the tests with nosetests, these functions were not run. I need to figure out why.

Creating and installing a script in my distribution module

In the previous blog post, I explained how to create and distribute a package distribution. If the package were a library, then most likely this is where it would end. However, since I have created something which will be used as a command line desktop application, it would be wonderful if I could bundle a script which the user can invoke from the command line.

First I made a shell script, which would invoke a Python module, however I ran into an issue where the interpreter could not locate the Python module. I think it should have since the package has already been installed with distutils. I need to spend some time understanding how the interpreter locates modules.

In the spirit of moving ahead, I decided to create a Python script and make it executable instead. Once I created the script, I again ran setup.py, which caused this script to be copied in the 'bin' directory of my ENV. Here is the simple Python script I created.

Creating a simple distribution module

To understand better all the things I learned in Exercise 46, I am now going to create a simple distribution module which will work with distutils. Here is the documentation of how to create such a module with distutils.

I am creating a simple Python Pomodoro.

First, I copied the skeleton directory into another directory called 'pypomodoro'. Next I renamed the package directory 'NAME' to 'pypomodoro'. Next, in the tests directory, I renamed NAME_tests.py to pypomodoro_tests.py. Next, in setup.py I added 'pypomodoro' to the packages names parameter.

Packaging is usually a nightmare on all platforms... and it is not exception with Python, at least when learning it.

Anyways, here are some notes which I am hoping will help me understand all this better.

In Python project names usually use camel case. So I have renamed my python-pomodoro project to PyPomodoro.

The project directory will usually contain a few more directories and files.

PythonProject

\bin

\docs

\tests

__init__.py

module_name_tests.py

\main_project_module

__init__.py

module1.py

module2.py

setup.py

I realized that we refer to the terminology module for individual python files, as well as to directories. Directories which are modules however, must have a file called __init__.py

So, this is a bit confusing. A bit of Googling takes me to this tutorial on Python packages. This tutorial states

"Packages are a way of structuring Python's module namespace by using dotted module names"

This makes sense. Looks similar to packages in Java, but the term 'packages' in Python seems to be overloaded. I believe I also encountered it in the context of the thing which contains setup.py. Perhaps this is not a package, but a 'package distribution'.

We can import sub-modules like this:

import A.B

where we expect B to be a submodule of A. Note that B cannot be a function or a member inside A. It has to be a submodule. What this means is that the directory A, must have a file called __init__.py and also if B is a directory, it must have a file called __init__.py However, if B can also be a file. The __init__.py file is invoked when the module containing it is loaded. Here is a tutorial explaining the usage of __init__.py

Question: What if the module is loaded multiple times? Perhaps I should also ask the question, if a module CAN be loaded multiple times.

Ways to install a Python package

After doing exercise 46 of LPTHW, I realized that there are multiple ways of installing a Python package.

Here are three ways that I know of.

pip install package_name

easy_install package_name

Download and unzip the tar file and then do
python setup.py install

LPTHW Exercise 46

In this exercise, we create a skeleton directory for future Python projects. This skeleton basically contains a directory structure, and an install script.

For all this to work properly, we need to have certain Python packages installed. The book recommends the following packages.

pip
distribute
nose
virtialenv

Now I am going to install all of the above packages.

Installing Pip:

pip is a Python package installer (which installs packages from the Python package index). It is meant to be a replacement for easy_install. So anything which can be installed with easy_install can also be installed with pip.

The pre-requisites for installing Pip are either setup_tools, or distribute. I read that setup_tools is not supported on Python 3.0. Even though this is not a problem, since I have Python 2.6, I figured it would be a good idea to go with 'distribute'

Distribute is also on the list of packages to install, so I install 'distribute' first.

curl http://python-distribute.org/distribute_setup.py | python

But I am also a bit curious about what distribute does. The documentation says it is a tool to 'Easily download, build, install, upgrade, and uninstall Python packages'. So basically this looks like a lower level tool which is needed by pip.

Now that I have installed 'distribute' I also installed Pip.

Installing VirtualEnv

Next I installed 'virtualenv' using pip:

'sudo pip install virtualenv'

I think there is a lot to virtualenv. I will write more about it in my next blog post. For now, I will link to the official virtualenv documentation page.

After installing virtualenv, I created an environment called ENV1.

I will now try to install nose in ENV1.

I activated the newly created environment running source bin/activate and then ran pip install nose.

Extra Credit:

So what do all of these software's do ?

1. pip - Is used for installing packages which are present in the PyPy package index
2. Distribute - Is a lower level tool for building, installing, and managing Python packages
3. VirtualEnv - Is a tool to create isolated environments for Python
4. Nose - Extends unittest to make testing easier

Understand setup.py

setup.py is used by the Python distutils as a standard way for installing third party Python modules. Before distutils, module creators would have to create install files for all the different platforms. Off course this is not easy, and I am sure there would be times when some module creators would not create such files. I believe since distutils uses Python, it is a way to use Python to install modules, rather than relying on the platforms way of installing Python packages. I also think this is good encapsulation :-)

If you are using distutils then you will know immediately if the package maintainer has packages their application in standard way or not. A Python package which is meant to be installed using distutils will always have a setup.py file bundled with it. This file will contain the calls it makes to distutils to install the package.

When we run

python setup.py install

There are really two things that happen. First is the build step, which puts all the files which need to be copied into the distribution root's build directory. I think by distribution root they mean the directory which is created for the distributed package. We can however change the directory in which the build process happens, if we so desire.

Once the build phase has completed, the install phase will copy the files to the chosen install directory. If no directory is specified then the default directory, which on *nix systems is usually /something/pythonX.x/site-packages. On my Ubuntu system it is /usr/local/lib/python2.6/site-packages However, it seems that modules are actually being installed in dist-packages.

It is possible to install the package using an alternate install scheme. There are various alternate install schemes. Even though I have not looked into them in any great detail, they seem to be able to not only customize the install location, but also determine who is able to access and use the package. Details for all of these install schemes is available here.

I will stop here as far as using disutils is concerned. This is a good introduction, and should be fine to start with.

Now I will understand the basics of using distutils for package creators.

Since distutils is also about packaging, let us try to understand some basic terminology. We will talk about general Python terminology followed by distutils terminology.

1. Module - This is the basic unit of code-reusability in Python. It is a block of code, which is usually imported by other code to be used. This is usually a single Python file.
2. Package - This is usually a module that contains other modules. It is usually contained in a directory. This directory must always contain a file called __init__.py
3. Root Package - This is the root of the heirarchies of packages. The root package usually contains the entire Python standard library, along with many stand alone modules which do not belong anywhere else. Now the documentation says something which totally freaked me out ... "Unlike regular packages, modules in the root package can be found in many directories: in fact, every directory listed in sys.pathcontributes modules to the root package"

distutils specific terminology:

1. Module Distribution is a collection of Python modules and packages, which are meant to be distributed en-masse.
2. Distribution Root is the directory which contains setup.py

Now let us understand the setup script from the perspective of a package creator. The main purpose of a setup script is to describe your module to distutils, so that the various commands that operate on your module do the right thing.

Without getting into too much detail, I will now create a simple module distribution and create a setup script for it. I will write about it in a separate blog post.

LPTHW Exercise 45

In this exercise we learn about IS-A and HAS-A relationships. We also learn how to invoke A superclass constructor from a subclass.