Create and/or update QA test plans for the release
For each significant new unit of functionality that has been added to this release:
- Create a new QA test plan covering this functionality.
- Add that test plan to the QA Test plans - Listing page.
If any existing functionality has been changed in this release, in a way that materially affects customer-facing behavior, make sure to update the relevant QA test plans accordingly.
Set up a testing tasks page for the release
Create a testing tasks page for the release. This page links to all of the QA test plans relevant to that release.
The entire list of QA test plans can be found on the QA Test plans - Listing page. On that page, you can also find a more or less complete list of what to test for major and minor releases. (In general, more QA tests need to be performed for major releases than for minor releases.)
As representative examples, you might look at:
- The testing tasks page for the 4.0 release (a major release).
- The testing tasks pace for the 3.2.2 release (a relatively minor release).
Confirm the end of development on the release
When development is scheduled to end on a release, the Project or Release Manager (who may be CollectionSpace's QA Lead or some other designated person) will confirm with the developers working on each of CollectionSpace's layers that they have:
- Completed what was planned for the release; or
- Otherwise arranged for any unimplemented functionality/remaining bugs to be pushed to the next release.
Create release branches
Once development on the release has been confirmed to be completed, the Project or Release Manager - or the developers for each layer, if specifically requested by one of these managers - creates a release branch in each of CollectionSpace's layers. See Creating Release Branches for step-by-step instructions for branching for a release, including the naming conventions used for release branches.
These release branches are then frozen to any commits, except for:
- Fixes to Blocker or Critical bugs identified during QA testing, and designated by the Project Manager as needing to be fixed during that release; or
- As otherwise specifically permitted by the Project or Release Manager.
Note that each bug fixed will likely require some additional QA testing: first to verify that the bug has been fixed, and secondly, to ensure that no regression has occurred as a result.
The master (aka trunk) branch is still open to commits during this time. However, it is expected that many developers will be participating in QA testing and/or fixing bugs encountered during the QA testing period, rather than working on the master branch.
Update the UI landing page to use the lastest version number
As of release v4.4, the landing page included just one file (
src/main/webapp/defaults/js/Footer.js) that included the version number. Edit that file and replace the old version number with the new/next version number.
Set up the CollectionSpace server for QA testing
Continuous Integration setup using Bamboo
Introduction to Bamboo
CollectionSpace uses Atlassian Bamboo, a server-based continuous integration application, to automate the build process on the project's QA server. This simplifies the tasks of setting up, maintaining and updating the QA server during the entire QA process. For more details, see Continuous Integration Service.
The initial build for QA requires each layer to have branched to the current Git branch. This can be verified by looking on each layer's github account for that branch version. For instance, the v4.4 branch for the UI layer can be found on github here: https://github.com/collectionspace/ui/tree/v4.4
Bamboo uses build plans to organize and configure how a complex project should be built, and the Bamboo instance for the QA server is broken into several build plans that span across all the layers. Each build plan represents a different aspect of the build process and all of them must be "pointed" to the same version branch in the CollectionSpace source code repository (depending on which layer is involved in that build plan, of course).
Here is a list of the current build plans (https://qa.collectionspace.org/bamboo/browse/NIGHTLY)
01 - Service Layer - Build(build the entire service layer)
02 - Tomcat - Stop Servers(Stops the Tomcat server. Necessary for deploying and reinitializing the database. No git access needed)
03 - Service Layer - Deploy(Deploys the built war files)
04 - Service Layer - Database(Reinitializes the database. This wipes out all saved data, recreates all tables (based on schema and Nuxeo), and restores all permissions to default values)
- 05. - (deprecated)
06 - Tomcat - Start Servers(Starts up the Tomcat servers. No git access needed)
07 - Service Layer - Test(Runs through the Service layer's unit tests)
07b - Tomcat - Stop Servers(Stops the Tomcat server. Necessary to build and deploy the App layer)
08a - App Layer - Build and Deploy(Builds and deploys the Application layer)
08aa- Tomcat - Start Servers(Start Tomcat server so we can run the App layer tests)
08b - App Layer - Test(Runs through the Application layer's unit tests.)
08c - UI Layer (default)(Builds and deploys the UI layer with minification)
08d - Initialize Authorities - As of v4.4, not necessary.(Populates the CollectionSpace instance with default vocabularies)
08e - Create Reports(Populates the CollectionSpace instance with the default reports.)
- 09 - Populate Data - Off by default (Populates the CollectionSpace instance with sample data. This is useful for stepping through the scripted test plans.)
10. - Tarball and Distribute(Creates a tarball for post-QA testing. This may not be current anymore.
Bamboo lists the build plans in alphabetical order. Therefore, we've prefixed the plan names with numbers to show the order in which they are executed. Build plan "
08b - App Layer - Test" should be configured to have either "
08c - UI Layer (default)". Each build plan can have a dependency and a dependent -which determines the order in which the plans are executed.
Edit the release branch name
To change the name of the release branch from which the QA server is built (for instance, to change it from a
v4.0 to a
v4.1 branch, when entering QA testing for v4.1):
- Log into Bamboo at http://qa.collectionspace.org/bamboo
- Click the Administration tab at top
- Click Global Variables in the sidebar at left
- Change the value of
currentReleaseBranchto the name of the release branch
Note that, by default, all of the build plans use the value of Bamboo's global variable,
currentReleaseBranch, to identify the branch used for the build
If, during a QA test period or otherwise, you'd like to change the branch used to build CollectionSpace's Application, Services, or UI layers - individually or across two or three layers - you don't need to visit the configuration for every build plan and make changes in each. Rather, you can change the branch to be built from globally for any layer, via https://qa.collectionspace.org/bamboo/admin/configureGlobalVariables!default.action
Rebuild the QA server
The QA server should be rebuilt:
- Just before QA testing starts.
- Whenever a fix is made to a Blocker or Critical bug, that requires additional QA testing.
- Under rare circumstances, if the server must be rebuilt in order to fix critical problems that interfere with QA testing.
The QA server is typically rebuilt outside testing hours. (Since testers may be working in a variety of different time zones, and may even be international, these hours should be arranged and announced in advance.) Under exigent circumstances, and ideally with at least some advance notification, it can be rebuilt during testing hours.
To rebuild the QA server:
- Log into Bamboo at http://qa.collectionspace.org/bamboo
- From the Dashboard, scroll down to the "
Full Build" (https://qa.collectionspace.org/bamboo/browse/NIGHTLY)
- Click the triangular (play) button to the far right of "
01. Service Layer - Build".
(Once that build plan completes successfully, it will automatically start plan "
02. Tomcat - Stop Servers" and so on, as the build plans are chained together.)
Some plans should remain off/deactivated until needed. For instance, plan #10 doesn't need to be run until the QA process for the entire release is complete. Some plans don't need to be run more than once. This varies depending on circumstances, but generally steps 08(d-e) only need to be run once at the very beginning of QA. In this case build plans 8(d-e) could be temporarily disabled after the QA server is built for a QA period.
Build plan disabling/enabling
Throughout the QA process it may be necessary to update one or more of the layers depending on what has been recently committed. It is possible to run the build plans out of order without starting from plan #1. Remember, the build plans are linked and will start the next successive build plan once one completes successfully. It is possible to temporarily deactivate a build plan in order to prevent it from running because its previous build plan completed. For example, say you want to rebuild ONLY the Services layer. You could do this by disabling plan 07b -which prevents plans 07b - 08e from also running once you kick off a build of 01 to rebuild the Services layer.
A build plan can be disabled/re-enabled in the following steps
- Click the pencil icon on the far right of the build plan you wish to disable
- Click the
Actionspop-up menu near the top on the far right
- Click the
Enablemenu option listed
Once the server is ready, an announcement is sent to the project's "Talk" and "Announce" email lists saying that QA testing can begin. The email should include a link to the testing tasks page and the server to test on (usually http://qa.collectionspace.org/). It is also a good idea to mention what functionality is new/affect, and ask testers to prioritize these tasks (eg. for in version v4.x, testing of Roles and Permissions and Vocabulary related things is priority)
Testers can then begin by assigning themselves one or more of the testing tasks set up for the release.
If any blockers are found during QA that should be fixed. The fix for the blocker should be committed to release branch and potentially to the master branch as well. Once this is done, the testers should be notified of downtime of the QA server, and the steps described above should be carried out to update the server (only the sources of the layers that were involved in the fix need to be updated). Once the QA server is back up and running, the testers should be notified and testing can start again. Any area potentially affected by the fix should be at least superficially, and preferably thoroughly, tested again.
In the QA testing phase, the automated tests should be run too. It's a good idea to do this early in the testing phase, since some tests might be outdated/broken and requires fixes. Optionally, during the development phase, the tests can be run against nightly to check whether they are broken so they can be fixed before QA starts. In any case, the automated tests should also be run during QA - instructions on how to do so can be found here: Running Automated QA Tests
Tag the release
Once testing is done and all blockers are fixed, the layers are ready to be tagged. The Project or Release Manager - or the developers for each layer, if specifically requested by one of these managers - will create a tag in each layer.
See Tagging Releases in Git for step-by-step instructions for tagging a release, including naming conventions used.
Create the tarball and publish on FTP server
As soon as all the layers have tagged the release, we are ready to create a "tarball" of a CollectionSpace server folder for that release. Use the CollectionSpace Atlassian Bamboo build plan on the QA (https://qa.collectionspace.org:8443) server -see https://qa.collectionspace.org:8443/browse/FB-TR1CDT. This creates a tar file on the
qa.collectionspace.org server here:
/var/www/html/builds. You need to copy this tar file to the CollectionSpace FTP server on
nightly.collectionspace.org. For example, after you've logged into the
qa.collectionspace.org server, use these commands (used for creating the v4.4 release) as an example/guide:
Test the tarball and update the install instructions for Linux distributions
After the tarballs have been created and made available on FTP (placed at
nightly.collectionspace.org), they need to be tested in our supported environments.
Update the wiki and more with the new release
The implementer documentation, and a release page for the new release should be set up on the wiki. The procedure for this can be found here: End of Release Check List, towards the bottom of the page. "Documentation rotation" is the process of cycling out the the unreleased documentation into the current release documentation.
If this is a major release:
Update the demo server
- If possible, do a fresh install of OS on the slice
- Set up a "Temporarily Down for Upgrade" page on the server
- Follow the install instructions to install the release
- Check that the install is successful
- Replace the temporarily down page with a redirect to collectionspace
Release is announced to list