Creating and Maintaining Django Webapps for CollectionSpace

Owned by John Brandon Lowe MLIS PhD
Last updated: Jun 02, 2014
4 min read

 

Overview

This document describes the soup-to-nuts engineering process for creating and maintaining a "cspace_django_project" Project for your institution. This process is the culmination of some months of experimentation and development at UCB, and what you see here is really the tip of the iceberg. The many details of how this process came about, its motivations, and its justification, are here.

Below, however, please find the essential workflow for this process outlined; consult the outlinks for details, tips, troubleshooting, etc.  

It assumes you are familiar with Django, GitHub, and QA and Release Engineering practices.

 

Prerequisites to development

At UCB, development of Django webapps is done using Pycharm. Therefore, your first step in any case is to install PyCharm on your development system. (Unless of course all you are doing is deploying existing code from GitHub, in which case skip ahead to the deployment section.)

Once you have PyCharm installed and working, you can create new projects and webapps in this framework or maintain existing ones. Follow the steps below that are appropriate to where you are in the process.

Creating a new CSpace Django Project for an institution

Skip this step unless you are the one creating a GitHub repo for your institution's Project(s).  

The UCB practice for Django Webapps is to have one (or perhaps more) Django Projects per institution.  Each of these is forked from the parent project, called cspace_django_project and is maintained as a separate entity.

The "normal" project creation workflow assumes you are the developer who will be creating the institution's initial project and webapps, and further that you are a CollectionSpace committer who can add repos under http://github.com/cspace-deployment.  However, such permissions are not required.  Read up here for the distinction in roles

If no Django project exists for your institution, or you need to make another one, the following steps will create such an institution level project in GitHub:   

What Should I Be Doing?

Checking out an existing institution's project for development

More likely, however, an institutional project already exists in GitHub (typically, it will be called something like myinstitution_project) and you can check the code for it out from GitHub and do what you need to do on your local system. In general, we don't modify code directly on our VM deployments. The process of getting your code back into a GitHub repo and making deployments and releases is described further below.

  • From PyCharm checkout the project, or better yet, your own fork of the project: from the command line: git clone http://github.com/cspace-deployment/myinstitution_project.git
  • Open this directory in PyCharm on your local machine
  • Configure the project to run locally (i.e. created needed local infrastructure (directories, etc.), local configuration files, etc.)
  • Run the development server or debugger in PyCharm

For the gory details, you can look at:

Creating of a new webapp for an existing project

Once you have a local clone of the institution's repo, you can add or modify the code.  If the webapp you want to work on exists, go to the next step, otherwise:

  • Run startapp (link to Django docs) or copy an existing app directory to a new directory and customize from there.
  • Edit cspace_django_site/settings.py (INSTALLED_APPS)
  • Edit cspace_django_site/urls.py (urlpatterns)
  • Resolve any other site-level dependencies: logging, cacheing, configuration, other python or code dependencies
  • Create an example config file (if necessary) See: Use of the config directory and .cfg files

There is nothing special about the steps here: they are the standard steps for creating an application within a Django project.  Except that you should plan to adhere to some of the coding and configuration conventions used for these CSpace-enable webapps.

Maintaining an existing webapp

"Maintaining" here means making modifications to existing code and pushing them back to GitHub. The actual release process is documented 

  • Check out latest code (see Checking out an existing institution's project for development, above)
  • Modify and test, modify and test, ...
  • git add ...
  • git commit -m ...
  • git push (to your own Git repo, or directly up to the cspace-deployment repo. See Making a Release.)

Deploying an institution project to a "Linux Server"

The initial deployment of Django projects on any server entails a number of "devops" activities. It is assumed that deployers reading this have an understanding of the process for deploying software on the target servers; here we assume that the deployment target is a "type IS&T VM" like those in use for the existing UCB CollectionSpace deployments. Your experience may differ.

Basically the procedure is the same as those for development deployment with PyCharm, but some important details differ. Here are the steps.

  • Verify operating environment, resolve dependencies (Apache and mod_wsgi, Python, Django, virtenv, SELinux, python packages)
  • Clone project to /usr/local/share/django  (this is the UCB "standard" for Django deployments)
  • Modify Apache configuration to accomodate new project (i.e. edit conf/wsgi.conf, see )
  • Copy example configuration file(s) to config/ and customize per deployment (if needed)
  • Restart Apache (sudo service httpd graceful)
  • Verify webapp function (e.g. by visiting their start pages in a browser)

The myriad practical details that need to be considered at UCB may be found at:

Pushing your spiffy revised project to a Dev or Production Server

Let's suppose your project is already deployed and running (see the step above). You have made some wonderful fixes and improvements and you need to update the code on the server to the latest version.  Assuming your changes have been pushed up to the master branch of your institutions repo, you need only pull the code changes for that release into the deployed project's repo on the target server (keeping any local mods via git stash) and restarting Apache:

  • ssh in to the server
  • cd /usr/local/share/django/my_project
  • (sudo git stash)  # needed if you have local modifications to the code that you want to preserve.
  • sudo git pull -v
  • make any local mods required: e.g. update/add configuration file(s): "sudo vi config/webapp.cfg"
  • (sudo git stash apply)      # needed only if you stashed local changes above
  • sudo manage.py collectstatic
  • sudo service httpd graceful

Maintaining the parent CSpace Django Project

- defer

Souping up your webapps

Solr

PIL

...