This process is driven by the need to foster collaboration across all the layers, and to increase our productivity - to deliver features faster. We need to improve our development focus so that it is clear what everyone is working on and where they are in that particular effort. We also need to clarify the scope of each feature, including what is in scope, and what is out of scope, for a given release. This will help to reduce inefficiencies and ugly surprises that come with mis-aligned expectations across the team.
A major change in our process is to have shorter release cycles that focus on fewer features. We want to focus our efforts together, to align everyone around fewer features, and to have more regular milestones that mark our progress. These shorter cycles will be lighter weight than our full, public release cycles have been, so we can move more quickly. We will periodically have a cycle devoted just to review, refactoring, and bug fixing, as we recognize that this is part of the development process. At certain points (to be determined), we will go through a full release process to provide a public release.
As part of this new process, the entire team will work together on these features, rather than staggering work across different layers. We talked at length in the past about dependencies among the project teams, and have been trying process models that staggered development work. After much discussion and review, and input from team members, we have concluded that the staggered process was causing more problems than it was solving. These include problems of getting a shared understanding of scope for features and releases, of correctly identifying dependencies, and working through requirements among the various parts of the project. More fundamentally, the staggered process was seen as a major impediment to getting everyone to work as a single team, on a single product.
The new process will roll out like this: 1. The functional and UX teams will gather requirements and prepare prototype UX wireframes for a set of candidate features; 2. Team leads will review these and narrow this set to those features we think can fit in a release cycle; 3. Then entire team will participate in a review, specification, and estimation sprint to explore and clarify issues. We'll review the scope again, and then 4. the team will begin the development sprint proper; 5. A brief test and release process will complete the release cycle.
Although the length of the sprints will vary with the scope of the feature cycle, we are expecting to limit the design sprints to one week or less, and the development sprints to about 2 weeks.
1: Functional Documentation and User Experience Design
The functional team (led by Megan), will continue to prepare user stories, work flow diagrams, schema requirements, and related information, that illustrate at a detailed level, what needs to be accomplished for a particular set of functionality (no more than about 6 features). These priorities and features will continue to be based on feedback from implementers, grounded in community-based standards, and reflect a collective view of how our users will work with CollectionSpace. This process is documented more fully on a separate wiki page .
The UX team (led by Erin) will create wireframes to illustrate what users will see and how they will interact with each feature/functional component/group. The wireframes will continue to be based on schema requirements and reflect feedback from the functional team and implementers.
These documents and wireframes are the essential inputs to the next steps in the process. The start of this work will overlap with the close of the development sprint of the previous feature set; this will ensure that the information is available well in advance of step 2 - scope review with team leads.
2: Scope Review with Team Leads
Team leads (PM:Penelope, Functionality:Megan, UX:Erin, UI:Michelle, App:ChrisM, Services:Patrick, Test/QA:Kasper) will meet and review the list of candidates proposed as a result of step 1. and will roughly scope out a list of functionality (1-3 features) that can reasonably fit together into a review-and-specification sprint.
The scope for each sprint will include both new features, as well as some mix of outstanding bugs. Once the scope has been determined, each person will go through their jiras, target the relevant ones to the new release the team is working on, and leave the rest targeted for 1.x.
3. Review and Specification Sprint
This sprint begins with a full-team call wherein Megan and Erin respond to questions about the functional descriptions and UX prototypes they have prepared, and that the team leads have agreed will be reviewed in this sprint. The activities in this sprint include:
- Broad design discussion across the layers to explore what is in scope and what is not in scope.
- Open discussion of the interaction of these features with other existing and planned features.
- Review of the UX prototypes, especially if we need to consider design alternatives that allow us to produce a feature more quickly (e.g., alternative widgets, etc.).
- Detailed design and investigation phase across the layers.
- Specification of APIs, payloads, configuration requirements, etc. among and within the layers.
- Discussion of testing requirements and approaches.
- Discussion of work involved, and ways (i.e., design alternatives) to reduce this work.
- Discussion of how to share work, what dependencies will exist.
- Discussion of the high priority bugs that could be considered for the development sprint.
- This part of the process should also focus on REST calls and Payloads
- We need to iterate in the design sprint
- Erin can sketch up ideas before creating wireframe to make iteration through design faster and require less upfront work
- Identify need for re-factoring early - communicate it to the team - justify why it is necessary. Raise this in Stand Up, discuss impact on others, schedule any resulting meetings. Define a concrete period of time for re-factoring.
At the end of this sprint, we will conduct a Scope and Time Review where the development team leads present work estimates to implement this functionality. If there are alternatives that would offer a smaller scope and time scenario, these will be resolved and an approach will be chosen.
The outputs of this sprint will be a clear set of requirements, specifications, and work estimates for everyone to pursue in the development sprints.
Identify need for re-factoring early - communicate it to the team - justify why it is necessary. Raise this in Stand Up, discuss impact on others, schedule any resulting meetings. Define a concrete period of time for re-factoring.
4. Development Sprint - part 1
We will need more communication and coordination among the teams to ensure that dependencies are met. Part of the idea behind mixing bugs and features is to ensure that no team need sit idle while they are waiting for a dependency to be completed.
- Create Milestones and dependencies within development sprint including deadlines for re-factoring.
- Re-use existing code where possible to make a proof of concept early - Frankenstein version - so can iterate in the development sprint. This should be clearly labeled as Frankenstein and not actually intended for release.
- Need sample payloads early.
- Develop in your layer and the other side to accept your payload
- Few people will work across all layers and also build unit tests
We are currently discussing some tweaks to our stand-ups to make them more open and effective for coordination, for raising scheduling and dependency issues, etc.
- Meetings need to invite the attendees on Google Calendar.
- Come prepared to meetings - review all relevant documents
- Anyone can call a small focused sub meeting of relevant people
We are also working out our practices and policies for using the wiki and JIRA to document and track our work. We know this needs work, and we will all need to continue to contribute ideas.
5. Development Sprint - part 2
This test and release phase will be reduced in time and scope. We will try to limit it to a day or so. If we find blocker bugs for the current functionality, we may extend the sprint, but we will generally try to stick to the schedule. If we cannot fix bugs, they will be documented and assigned for resolution in a later sprints.
6. Sprint Review Meeting
The Sprint Review Meeting will consist of developers demonstrating the sprint task completed for the release. This meeting will also discuss documentation needed to support the task, code altered to prioritize testing, and also potential risk for QA.
7. Closing a Sprint
At the end of the sprint, on the release page for that iteration, a section called Summary of Work Completed is added so that it is clear what made it in and what did not as things can change during this process. It will also indicate if this is an interim release or a major, stable release.
QA and maybe Release
At the end of the sprint there will be a day or two of QA performed by a couple of team members who did not code the feature they are testing. At the end of this period, the release is tagged and made available to implementers to download if it has things in it they can use. If we only work on infrastructure, then it will not be available to implementers to download.
Penelope will send an email to the Talk and the Work lists announcing the new release and linking to where they can download it, and where they can find information and documentation about it.
Putting this into practice
There is a good deal to be refined and ironed out here, and we expect everyone to continue to contribute ideas to make this new process work better for us. We have planned to start this new process right away, with the 1.1 release cycle. This means that the All Hands Meeting will fall right in the middle of the development sprint. This will give us a great chance to talk over the new process, discuss refinements, etc.
Bug Fixing Iterations of this process
Prior to the start of a Bug Fixing sprint, the team will spend time identifying what are high priority issues they need to address for their layer, what they will need from other layers to accomplish this, and we will identify areas of functionality to focus on (right sidebar, roles and permissions, etc).
We will then assign JIRAs to the sprint, add time estimates, and scope what can be completed in the time assigned.
At the end of the development phase there will be full regression QA, and a release of software.
As a team, we need to be able to work across layers during times when one team has more work than another.
In preparation for the all hands meeting, Michelle put together a list of what she think are good places for people to get involved in the UI layer:
1. get all the unit tests passing in the swarm
2. automate the swarm test running
3. automate the swarm test checking
4. improve unit test coverage
1. determine an acceptance test strategy
2. create acceptance tests which would cover as much of the QA plans as possible
3. simplify the QA plans to remove things that are adequately covered by automated tests
4. automate acceptance test running and reporting
1. determine if the currently open JIRAs are still valid
2. triage JIRAs as deeply as possible
3. pick up and fix simple JIRA issues - can't really determine which these are until after triage
4. pick up JIRA issues related to smaller components such as DatePicker, Sidebar, TitleBar, SearchToRelateDialog and Repeatable.
1. make the current HTML pass the W3C validator
2. simplify the HTML to remove extraneous markup resulting in markup that carries as much semantic information as possible
4. introduce the use of ARIA in order to add even more semantics to the markup
5. change DatePicker to be more inline with how repeatables work - the markup should be generated by the component instead of requiring implementors to alter the markup when they want to make a date field
1. consolidate the CSS under a cspace namespace (cs-)
2. ensure we aren't using the csc namespaced
3. ensure we correctly scope component css to a component
4. move to using FSS where it makes sense - requires looking into what we are doing with CSS and where this overlaps with FSS
5. create a single skin file for cspace that can be easily replaced or overridden by an implementor