CSpace-Django webapps - setting up a code repository

Instructions for creating a new CollectionSpace UCB Django project in GitHub. Follow these instructions if you want to extend the cspace_django_project for a particular museum or collection at UC Berkeley. If your museum or collection already has an extension of the cspace_django_project called something like pahma_project, look at the instructions at CSpace-Django webapps - setting up Python and Django in a development environment to get started.

These instructions are written for CSpace-Django webapp developers at UC Berkeley. CollectionSpace deployers at other institutions wanting to develop CSpace-Django webapps for their collections should begin by creating an account on GitHub and forking the cspace-deployment/cspace_django_project code to their own repository. (See GitHub help for documentation on how to do those steps.) After doing that, deployers at other institutions can follow the steps below, substituting their institution's GitHub repository name for cspace-deployment.

The cspace-deployment/cspace_django_project repository on GitHub is a UCB specific fork of the collectionspace/cspace_django_project repository from the collectionspace GitHub account/organization. It includes improvements to the code and enhancements such as cascading style sheets and a CollectionSpace WebApps logo. These modifications have not yet been contributed back to the CollectionSpace repository.

Overview

Our goal is to create a fork of the cspace_django_project in GitHub that will contain the code for our particular local deployment (we want our own code but, but also to maintain the connection between our project and the parent, in case there are bug fixes or ehancements to it later on that we will want to pull down).

Then we can clone that fork locally to make modifications – indeed to create our own custom webapps.

Since we will want to modify some site-specific details of the parent cspace_django_project in our project (e.g. modifying the CSS so that the webapps we build all have our look-and-feel), we'll want to fork a local deployment-specific repository that has the GitHub deployment-specific repository as its origin. Unfortunately, we can't simply fork the GitHub cspace_django_project within the  same GitHub account. (That's not allowed.) Instead, we will:

  • Clone the cspace_django_project to a local git repository
  • Clone that clone to create what will become the deployment-specific local repository
  • Create a new GitHub repository to hold the deployment-specific code
  • Set the new GitHub repository as the origin for our local deployment-specific repository

1. Get a Local Copy of the cspace_django_project Repository

Create your local copy of the cspace_django_project in one of two ways, depending upon whether you have committer status to the organizational GitHub repository (in our case here at UC Berkeley, github.com/cspace-deployment).

If you are a committer (that is, you have the necessary permissions to commit changes back to the repository codebase), make a local clone of the cspace-deployment/cspace_django_project on your workstation by running:

git clone https://github.com/cspace-deployment/cspace_django_project

If you don't have committer privileges on the organizational GitHub account, you must first fork the cspace_django_project to your own GitHub account. To do this, browse to the organization's GitHub page (https://github.com/cspace-deploymenthttps://github.com/cspace-deployment), then:

    • Find the cspace_django_project repository link and click the name to open the repository.
    • Click the Fork button in the upper right-hand corner of the repository page.
    • Choose your GitHub account from the options presented (in response to the question, Where should we fork this repository?)

Open a new tab or otherwise navigate to your own GitHub repository. Under the Repositories tab, you will see your new fork of the cspace_django_project. Make a local clone of this project on your workstation by running:

$ git clone https://github.com/cspace-deployment/cspace_django_project
Cloning into 'cspace_django_project'...
remote: Counting objects: 122, done.
remote: Compressing objects: 100% (81/81), done.
remote: Total 122 (delta 28), reused 122 (delta 28)
Receiving objects: 100% (122/122), 41.04 KiB | 0 bytes/s, done.
Resolving deltas: 100% (28/28), done.

Signing up for a GitHub account

For instructions on how to create a GitHub account, see https://help.github.com/articles/signing-up-for-a-new-github-account. A free user account will be sufficient for our work.

 

By either method, the local clone you just created is your working copy of the remote cspace-deployment/cspace_django_project repository. Changes to this local repository and commits to the corresponding remote repository should be limited to things that can be shared with ALL of the downstream repositories - i.e., the repos that are a forks/clones of this repository for institution specific deployments like PAHMA, Botanical Gardens, etc.

You will edit this project only if you intend to make and contribute improvements to the cspace_django_project itself. For the most part, you will use this file only as the source of your project's Django code. See step 3, below.

2. Create the New Repository in GitHub

Next, we'll create a new deployment-specific (e.g., PAHMA, Botanical Gardens, etc) repository in GitHub. From the cspace-deployment GitHub account/organization home page, create a new empty repository. For clarity, use a name that indicates this new repository is a clone of the more general cspace-deployment/cspace_django_project — e.g., name it something like pahma_project. This repository will be set in step 3 as the origin of your local deployment-specific project repository.

3. Clone the Local cspace_django_project to Create a Local Deployment-specific Project

In your local repository, you will now create a clone of the "vanilla" cspace_django_project in which you will do your deployment-specific development. Give this local project the same name as you've given to the deployment-specific repository in GitHub. Assuming you've chosen the name pahma_project, run the following git command on your local machine:

$ git clone cspace_django_project bampfa_project
Cloning into 'bampfa_project'...
done.

To create the institution-specific project as a peer of the cspace_django_project, run this command from the same directory into which you cloned the cspace_django_project, that is, the directory that contains the cspace_django_project. (You should still be at that location).

At this point, your local pahma_project points to your local cspace_django_project as its git origin. Instead, it should point to the pahma_project in the cspace-deployment GitHub repository. To make this change, begin by changing directories into the pahma_project directory you just created, then remove the existing remote and add a new one:

cd bampfa_project
git remote rm origin
git remote add origin https://github.com/cspace-deployment/bampfa_project 

4. Populate the New Repository

Lastly, we need to push the contents of the local pahma_project to its origin in GitHub (see step 3). To do so, run the following git command from the current directory:

$ git push --mirror https://github.com/cspace-deployment/bampfa_project
Counting objects: 122, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (81/81), done.
Writing objects: 100% (122/122), 41.04 KiB | 0 bytes/s, done.
Total 122 (delta 28), reused 122 (delta 28)
To https://github.com/cspace-deployment/bampfa_project
 + 6a8547e...ecf645d master -> master (forced update)

This command essentially clones the contents of the local pahma_project repo (which is as yet an untouched copy of cspace_django_project) into the newly created and empty cspace-deployment/pahma_project.

NOTE: older versions of git may not be able to do this: if --mirror returns an error, consider checking to see you have a recent version of git.

You now have a local copy of the cspace-deployment/pahma_project repo which is derived from the cspace-deployment/cspace_django_project repo.

5. Setup a git "remote" to the cspace_django_project Repository

From the current directory, run the following git command:

git remote add upstream https://github.com/cspace-deployment/cspace_django_project

Adding the cspace_django_project as a remote to your deployment specific repositories allows you to push and pull changes from it as well as compare against it. Go ahead and create a local branch that tracks the cspace_django_project's master branch. I call mine upstream_master, but you can call it whatever you want:

$ git fetch upstream
From https://github.com/cspace-deployment/cspace_django_project
 * [new branch]      master     -> upstream/master
$ git checkout -b upstream_master upstream/master
Branch upstream_master set up to track remote branch master from upstream.
Switched to a new branch 'upstream_master'
$ git remote -v
origin	https://github.com/cspace-deployment/bampfa_project (fetch)
origin	https://github.com/cspace-deployment/bampfa_project (push)
upstream	 https://github.com/cspace-deployment/cspace_django_project (fetch)
upstream	 https://github.com/cspace-deployment/cspace_django_project (push)

The fetch command gets the current state of the newly-added upstream remote cspace-deployment/cspace_django_project and stores it in your local project. The checkout command creates a new branch tracking upstream/master, called upstream_master, and switches to that branch.

If in the course of getting your project to work properly you have needed to make changes to the shared ("vanilla") code, you can use this upstream_master branch to push those changes to the cspace-deployment/cspace_django_project repository. From there, those changes can be propagated to other downstream projects and, perhaps, contributed to the collectionspace/cspace_django_project for others to use.

Finally, let's make a "release branch" for this project. The release process is described elswhere. Here all we need to do is kick it off by making a branch with the correct label .

$ git checkout -b bampfa_4.0
Switched to a new branch 'bampfa_4.0'
$ git branch -v
* bampfa_4.0      ecf645d NOJIRA: minor enhancements to this project before forking again...
  master          ecf645d NOJIRA: minor enhancements to this project before forking again...
  upstream_master ecf645d NOJIRA: minor enhancements to this project before forking again...
$ git status
On branch bampfa_4.0
nothing to commit, working directory clean

Ok, now we can start doing the customization needed for our institution and start adding apps.  You'll want to use PyCharm (refer to Details on installing and using Pycharm for webapp development) and follow the standard Development and Release Process for Webapps.

Editor's note: ADD SECTION on setting project-wide Django configuration, that is, editing wsgi.py (to set mount point/URL path segment for the project's apps) and settings.py (to set the static_root directory for project)

6. Pushing changes upstream/Cherry-picking from downstream

Editor's note: This activity has not yet been performed in earnest, so the instructions below should be viewed as speculative and provisional.

 

There are a few ways to make changes to the upstream cspace_django_project.

If you have not yet started development of a feature you'd like to contribute to the cspace_django_project, you can:

    • Do your development in your local cspace_django_project and, when done, push your changes to its origin. If you have commit privileges in the organizational GitHub account, the origin will be cspace-deployment/cspace_django_project. If you don't have commit privileges, the origin will be your-github-account/cspace_django_project. You can make a pull request from there to get the code into cspace-deployment/cspace_django_project.

      cd your/local/repo/cspace_django_project
      git checkout master
      git checkout -b new-development-branch #create a new branch based on the master branch; do your work here
      git commit
      git checkout master
      git merge new-development-branch
      git push origin master:master

      To get these changes to your deployment-specific project in your local repository:

      cd your/local/repo/pahma_project
      git checkout upstream_master
      git pull upstream master
      git checkout master
      git merge upstream_master

      When you're ready to begin (or resume) work on your deployment-specific project, create a new working branch based on the updated master branch:

      git checkout master
      git checkout -b new-working-branch
    • Alternatively, you can checkout the upstream_master branch created in step 5, above, and begin working there. When you are finished, commit and push the change back up to upstream master branch. (If you have commit access, this will be in the cspace-deployment/cspace_django_project repository; if not, it will be in the your-github-account/cspace_django_project repository – from which you can make a pull request to github/cspace-deployment when you are ready.) Once the new code is in your upstream master, you'll need to update the master branch of your local deployment-specific project with those changes:

      git checkout upstream_master
      git commit
      git push upstream master
      git checkout master
      git pull upstream master
      
      # When you are ready to begin or resume work:
      git checkout -b new-working-branch

Editor's note

Need to vet the following.

If you've already started development of a feature on your own project's branch - in the master branch of your PAHMA repo, for example, attempt to keep your commits relatively 'atomic'; each commit should leave the checked-in code in a working state and doesn't mix institution agnostic code with institution specific code. Once you have finished the feature, commit, push to origin master, then switch to your clone of the cspace_django_project. Create a 'downstream' remote and create a local tracking branch of the downstream's branch which contains your changes (master in this example). Then checkout the master branch of the cspace_django_project, cherry-pick the commit and perform any manual merges necessary, and push up your changes:

From the directory containing the clone of cspace-deployment/cspace_django_project:

git remote add downstream git@github.com:cspace-deployment/pahma_project.git
git checkout -b pahma-downstream-master downstream/master
git checkout master
git cherry-pick <commits> -x
git push origin master

It should be possible to perform this from the pahma_project clone - after making your changes and pushing them up to origin master, checkout the upstream_master branch created above, and cherry-pick the commits you'd like to apply to the cspace_django_project repo, then push these changes back to upstream master. However, I was unable to get this to work.

pahma_project amy$ git push cspace-deployment-upstream master
Username for 'https://github.com': amywieliczka
Password for 'https://amywieliczka@github.com':
To https://github.com/cspace-deployment/cspace_django_project.git
 ! [rejected]        master -> master (non-fast-forward)
error: failed to push some refs to 'https://github.com/cspace-deployment/cspace_django_project.git'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and merge the remote changes
hint: (e.g. 'git pull') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
pahma_project amy$ git pull cspace-deployment-upstream master
From https://github.com/cspace-deployment/cspace_django_project
 * branch            master     -> FETCH_HEAD
Already up-to-date.

 

 

Editor's note: Add option for users without committer privileges... i.e, mirror to user's github account.