Start Lending

May 13, 2013

Start up

Welcome to Getting Started with Django episode 4, where we'll start the lending app. As usual, we start in our normal GSWD directory on our host computer and we'll do a vagrant up because we need to get our VM running. We'll let it boot up.

Even though our vagrant up gave us back some scary red error message, we should still be able to do vagrant ssh to get into the machine. These errors seem to come from Chef trying to rebuild our VM even though it doesn't need to.

Once inside, we, of course, have our blog-venv virtual environment that we made way back in episode one. We're starting a new project so we won't be using this virtual environment any longer. We need to make a new one, so we'll run virtualenv lending-venv. This time, too, we're going to pass in the --distribute argument so the virtual environment uses distribute instead of setuptools. I actually wasn't aware that this is a better method until recently. Once it's created, I'll go ahead and source lending-venv/bin/activate to get into the virtualenv and then we'll hop back to our /vagrant directory and into the projects directory in there.

Our microblog project is still here, of course, but before we can start our new project, we need to install Django. pip install django==1.5.1 will get that started for us. I'm explicitly stating the version this time so that it'll be easier to make sure everyone ends up on the same version of Django. This is actually a decent step up from the last project we did, which was on Django 1.4.2 and above. Django 1.5 provides a couple of nice new features and just general improvements in stability and security. We'll go over a couple of the new features later.

Settings

Now that Django is installed, we can start our project with django-admin.py startproject lending and in the lending directory, we'll see the usual lending proto-app and manage.py. In this internal lending directory, we have the Djange defaults of settings.py, urls.py and the like. We, again, don't want this single settings file; we want our settings module like in the first project. Let's make a settings/ directory and touch settings/__init__.py and then move settings.py into settings as django.py. Why? So we have a Django-created version to compare our old settings to.

Then copy /vagrant/projects/microblog/microblog/settings/base.py to here and do the same with local.py. In our text editor, on our host machine, let's open up our new project, specifically the settings/base.py and settings/django.py. Quickly looking between these two files, the first new setting that stands out is ALLOWED_HOSTS, which we want to copy from django.py to base.py. The rest, however, is actually still the same so we won't add or remove anything. We want to remove everything from our THIRD_PARTY_APPS and LOCAL_APPS tuples. We do, also, need to correct instances of the word microblg with the word lending. Then we can delete the django.py file from settings because we don't need it anymore. And, lastly, in local.py, we should change microblog to lending in the DATABASES dictionary because we want to use a new database. We also have a PROJECT_ROOT import that we don't need, so delete that, too.

Database

Back in the terminal, we need to run createdb lending to create a new database named "lending". Coming back to the /vagrant/projects/lending directory, we can run python manage.py syncdb we get…an error. The reason for the error is twofold: first, our settings files aren't actually in the settings directory. We can fix that by moving them into there. Secondly, our lending/settings/__init__.py file is blank and it needs to be updated to contain the correct imports:

from .base import *
try:
    from .local import *
except ImportError:
    pass

Now, back in the same directory as our manage.py, running python manage.py syncdb reminds us that we haven't installed dj_database_url. We'll install that and psycopg2 using pip. Sometimes it seems like, starting a project from not-quite-scratch, you end up with lots of little things that you've forgotten to install. Luckily, most things throw loud errors when you don't give them the things you've promised them.

We'll take this opportunity to install south as well. And while it's installing, we'll add 'south' back into our THIRD_PARTY_APPS tuple. When we run python manage.py syncdb this time, it works. We'll skip the creation of a superuser for now. Running python manage.py runserver 0.0.0.0:8000 and loading http://127.0.0.1:8888 in our browser gets us the usual "It worked!" page.

If you compare the amount of time it has taken to get this far now, versus the first lesson, you'll notice how much we've sped up. Aside from being a testament to how much you've all progressed, this also means we can go further in our first lesson for this project than we could in the last one.

Changes from Django 1.4.x to 1.5.x

Unfortunately, due to bad timing or planning on my part or just the open source gods conspiring against me, Django 1.5.0 came out right as I was wrapping up and releasing the first episode, so it was all built on Django 1.4.2, which was the most current version at the time of filming. Django 1.4.3, 1.4.4, and 1.4.5 all followed quickly after with security fixes, as did Django 1.5.1. So what's changed with all of these versions?

The first thing, as you should remember from us adding it earlier, is the ALLOWED_HOSTS setting. ALLOWED_HOSTS controls which hosts are allowed to make requests to your Django application. This prevents servers you don't run from hitting your applications, which gives just that bit more security. This setting must be set if your Django installation is greater than 1.4.3 and is running in production, or DEBUG = False, mode. When DEBUG is set to True, this setting is ignored and all requests are allowed through. If, for some reason, you want to disable this setting, you can set it's value to '*', but I don't recommend doing that.

The other major change in Django 1.5 is the ability to have custom user models integrated with the entire system. You could create your own user model in all Django's up to 1.5, but the default Django functionality, the admin and createsuperuser management command and other bits, didn't know about them or pay them any attention. Now, though, you can substitute in a new model in the old django.contrib.auth.models.User model's place and everything, more or less, continues to work the same. This requires a decent amount of wiring still, though, so we won't be creating a custom user model in this series. If you do create your own user model, though, you'll set it in your settings with AUTH_USER_MODEL being equal to a dotted path to your model.

Packages for this project

So, back to the project. There are a few more packages we can go ahead and get installed, even if we won't be using them right away.

First off, we'll install django-discover-runner again. We'll also bring in django-braces because we'll be writing class-based views. pip install django-discover-runner django-braces gets those both installed for us.

The other apps we'll be using are new, so let's go over them one at a time.

django-model-utils

Django-model-utils is like django-braces in that it's a collection of common utilities, this time for models and managers. It was created by Carl Meyer, a great Django and Python developer. Django-model-utils provides us with a few great gems like a Choices object that works like Django's built in ChoiceField but much more cleanly; StatusModel which builds on top of the Choices object to make switches like our last project's published BooleanField more powerful; and TimeStampedModel which is a convenient abstract model that automatically holds on to created and modified timestamps for us.

django-floppyforms

Django-floppyforms is a fairly small library from Bruno Renie that gives us HTML5 form elements instead of Django's default HTML4 ones. No, this isn't a giant boon to our project, but it makes our output more modern and, thanks to how form fields are rendered, gives us some easier widget-level control over rendering, if we should need it. It also plays very nicely with our next app….

django-crispy-forms

Django-crispy-forms, which you will see me misspell time and time again. Django-crispy-forms, from Miguel Araujo, is an amazing project that gives us the ability to control form structure and rendering from within our Python code, instead of having to build complex forms field-by-field in the HTML. Like django-floppyforms, it comes with custom field-level templates and also templates for layout-level items as well, like <div>s and <fieldset>s. This library and django-floppyforms are two that I install on every single project I create.

We'll install each of these with Pip by running pip install django-model-utils django-floppyforms django-crispy-forms and waiting a few minutes.

The first app

Now run pip freeze > requirements.txt to get our requirements file started and then cp /vagrant/projects/microblog/Procfile . to copy it from the old project to this one. As far as these two projects are concerned, at this time, their Procfiles are identical. When this project nears completion, we will change the Procfile to run our server as gunicorn instead of Django's runserver, and we'll likely have a Celery consumer as well. For now, though, the default is fine.

So let's start the first app by running python manage.py startapp lender. Then add 'lender' to our LOCAL_APPS tuple in our settings file. In the lender directory, we have the usual files, models.py, views.py and the like. We know we're going to be writing custom forms eventually, though, so go ahead and touch forms.py. Now open up lender/models.py in your text editor.

Models

Before we start creating models, though, we need to decide what the model holds and represents. The most logical first model to build is the model that represents an item that is being lent. We'll name it the very clever name of Item and, normally, we'd extend it from models.Model. But we have the handy django-model-utils library installed, so there's likely something more useful in there to base this off of. For instance, having this be a model that logs when it is created and when it's updated would be handy, so we'll base it off of TimeStampedModel, which we'll import from model_utils.models.

Our model will also have a CharField that holds on to the item's name, with a max_length of 255. We'll also add a SlugField, since we'll want to be able to refer to the item easily. Why use a SlugField and not just a CharField since they both hold the same data? SlugFields are automatically set as database indexes, and since we'll be querying on the slug, this should speed up our queries. You can specify a database index on any field by passing in the db_index argument, set to True. You don't want to build indexes randomly, though, so this is usually something you'll add after you have real data and metrics to decide which fields to index.

Next, we'll add a "description" field, which will be of the models.TextField type.

As a first pass at permissions, we'll add a final field named "available" which will hold whether or not the item is available to be borrowed. We want to use some of the shortcuts in django-model-utils for this, though, so we need to import a few things first. We want to import StatusField and Choices from model_utils. In our model, we'll create a new Choices object named STATUS with the options of 'yes', 'no', and 'friends_only'. We'll also add a field named status that is of type StatusField().

To make this field a bit more powerful, we'll also import QueryManager from model_utils.managers. We'll set a new attribute on the model named public which is a QueryManager with the status attribute set to 'yes'. We'll also set the objects attribute to models.Manager() which ensures the usual Model.objects queries will work, too. Then, to see the other statuses in action, we'll create private and friends attributes, both set to QueryManager instances with their status arguments set to the right values. We'll give each of these three attributes an order_by method as well, set to '-modified' so we get the newest items first. Modified, of course, comes from the TimeStampedModel base class.

We need to provide a __unicode__ method, which only takes the self argument, and it will return a Unicode string, denoted by the leading u. We'll be using the String format method on this string. We'll only provide the string with the first argument, and in .format() we'll pass in self.name.

def __unicode__(self):
    return u'{0}'.format(self.name)

We'll also go back to the top and convert all of the STATUS strings to be Unicode instead of standard byte strings. Same with the string values to order_by.

We also want to provide a custom save method on this model that will auto-slug the name of the item and save it to our instance's slug field. In this case, though, we want to make sure that our slugs don't change, even if the name does, so that people can bookmark items and always be able to get back to them. We'll also need to import slugify from django.template.defaultfilters.

def save(self, *args, **kwargs):
    if not self.slug:
        self.slug = slugify(self.name)
    super(Item, self).save(*args, **kwargs)

Migrate

Hopping back over to our terminal, we can run python manage.py schemamigration --initial lender. This gives us an error, though, because I got an import wrong. We should import StatusField from model_utils.fields, not model_utils itself. Only Choices comes from the base model_utils. This is a good example of why you should always read the docs.

Now if we run our command again to create the migration, it goes through without a hitch and we can run python manage.py migrate to create it in the database. Now, to play with this model a little, we'll pop into the Django shell by running python manage.py shell.

Shell

To start, we need to get our model in, using from lender.models import Item for that. Then we'll create an item:

>>> item = Item(name="Arduino Uno", description="An Arduino Uno microcontroller", status="yes")
>>> item.save()
>>> dir(item)

This creates an item in memory, saves it, and then gives us a readout of all of the attributes and methods on our object. If we print out item.slug, we get u'arduino-uno'. item.status gives us 'yes'. So now let's try our QueryManagers that we set up earlier.

Item.objects.public() gives us an AttributeError, because we called it the wrong way. Item.public.all(), however, gives us back a queryset of Item instances. In this case, just the one we created. Item.private.all() gives us back an empty list, as does Item.friends.all(), because the only Item we've created so far is completely public. So that means our QueryManagers are working appropriately.

Model, take two

We have a problem, though: our item doesn't belong to anyone. Now, this might not seem like a big deal in some egalitarian society, but our "friends only" idea of sharing doesn't work if we don't know who the owner is friends with. So, back in our lender/models.py file, we need to import User from django.contrib.auth.models. Then we'll set a new field on the model named owner that is a foreign key to User. We'll give it a related_name argument of u'items', because an owner would call their items "items". I guess we could have called it u'stuff', too, if we wanted.

Now, though, our model doesn't match our database. We can create a new migration to address this, but that seems a little messy, especially since our code hasn't been ran anywhere other than our system and there's only one Item instance in the database. Let's just update the original migration.

python manage.py schemamigration --initial lender --update will notice that 0001_initial has already been applied, so it rolls that migration back, setting that app's migrations to zero. Then it creates the migration again, in place, and tells us that it's ready to be run. python manage.py migrate will run this new-and-improved migration for us and get the new version of our model in the database. The rolling back to zero does remove all of our data, though, so this shortcut does have its price.

Shell, take two

Now we need a user. python manage.py createsuperuser will give the same set of prompts to create a superuser as you normally get when you first syncdb. If you created a user at that step, you don't need to do this again. Answer the questions and we're ready to jump back into the Django shell with python manage.py shell.

This time, though, we need to import two models, User and Item.

>>> from django.contrib.auth.models import User
>>> from lender.models import Item

First things first, we need to get our user. Since we know we've only created the one user, we can use the latest method on the default model manager to grab the newest one. user = User.objects.latest('id'). If your model didn't have an auto-incrementing field, like id, you'd want to provide a different lookup to latest. Doing this on a live site won't always get you such a reliable result, obviously.

And since we don't have any Item instances in the database, we'll obviously need to create a new one.

>>> item = Item.objects.create(owner=user, name="Arduino Uno", description="An Arduino Uno microcontroller", status="friends_only")
>>> item.public.all()
[]

When we ask for all the public items, we get back an empty list, because the item we just created isn't public, it's friends-only. If we do user.items.all() we get back all of our user's items, including the one we just created. And, of course, item.owner gives us back the user we created and set. As one last sanity check, doing Item.friends.all() gives us back our single item. Ultimately, though, we'll want to filter all of these querysets based on the requesting user, and that's best done in the view, so we won't worry about that on the model level, at least not at this stage.

Admin

As a final step for this lesson, we'll set up a basic admin view for our new model. First, touch lender/admin.py and then open that file up for editing.

At the top, import admin from django.contrib and import Item from .models, since we want to use relative paths. Then just register the model in the admin with admin.site.register(Item).

We need to turn the admin on in our lending/urls.py, so open that file up and uncomment the two lines at the top of the file and the one url route like before.

Then run the server again with python manage.py runserver 0.0.0.0:8000 and open up /admin in your browser. Since we created a superuser, we can go ahead and log in.

We should see our Items app available, with the Items model below it, and, clicking into that model, we should see our Arduino item that we created in the shell. Going and looking at it, everything shows up just as we expected it to.

Homework

We won't be using a custom admin for most or all of this project, so I'm not going to cover it. If, however, you want more practice, go ahead and configure a custom ModelAdmin for the Items model. Change how the list is displayed and any other attributes that you want to configure to meet your needs. Setting up the auto-slugging in the admin would likely be a very handy extra step, too.

Last steps

The last thing we'll do this time around, mainly as preparation for the next lesson, is create our templates and assets folders.

mkdir -p templates/{layouts,partials} will create our desired structure, and then we'll do touch templates/layouts/base.html because we know we'll use this file later. Same with templates/home.html.

Then for the assets, mkdir -p assets/{css,js,img} in order to get those directories ready and waiting for us.

Finally, we'll start using Git. git init will cerate the repo and then we'll copy over the .gitignore from /vagrant/projects/microblog/ to just have a bit of a starter. Open .gitignore up in your text editor and change microblog to lending and save. Then git add . followed by git status and make sure our settings/local.py doesn't show up in the list. Make a first commit and we're good to go.

Summary

We're not going to push this to Heroku just yet. If you want to, you can, but, officially, this project isn't ready for Heroku just yet. Let's talk about what we've accomplished in such a short time:

Installed several new libraries, including Django 1.5, django-model-utils, and django-crispy-forms
Updated our settings for use with Django 1.5, including the new ALLOWED_HOSTS setting.
Started a new app, created its first model, and even used some features from django-model-utils
And tested everything functionally, both in the browser, through the admin, and in the shell.

Our next lesson will cover some improvements to our model and the beginning of functionality in the forms and views. We'll also start writing tests as we go to make sure our code is covered. Thanks and be sure to check out our sponsors the DSF, Divio, and Umbrella.io!

Questions? Answers