Forecast Sales System for Airline - Part III

Intro

This is the third publication in a series of building a forecast sales system for an airline. It covers the organizational and management challenges that have appeared in this project and the solutions to solve them.

Executive Summary

The current team experiences challenges during the growth of new tasks which brings overhead on management and the development team. This leads to a set of unhealthy symptoms in the development process that should be fixed to avoid future issues in software delivery.

  • Formalization of requirements by tickets in CRM with user stories in the Kanban style
  • Daily stand-ups in chat by sharing what each team manager did, plans to do, and blockers on their way
  • Meeting minutes after calls to track agreements, action points, and responsible persons

Measures introduced to stabilize the codebase:

  • Setup and keep an up-to-date README file with all steps to deploy and run the project
  • Close the master branch for writing and push all work via Pull Requests after code review working by GitFlow in separate branches
  • Setup Continuous Integration to compile the project and run automated tests to confirm stability (still in backlog)

Organizational measures to solve flaws:

  • Fix changes in requirements and scope of the project within a sprint to let the team focus on tasks and deliver them
  • Setup relationships with the key stakeholders and gather feedback regularly
  • Formalize all requests from the client in tickets to confirm we are all on the same page
  • Regular rotation of team members over project tasks to mitigate silos of knowledge and cope with the bus factor

Similar to our bodies constantly exposed to viruses and threats, projects constantly face changes and challenges. Proper organization helps projects pass through challenges without much issue, just like a healthy person does. It's important to set up the right measures in management and development work from the beginning and evolve them over time as new challenges arise.

Transmitting Knowledge When Scaling Up the Team

Issue

  • Knowledge silos within team members' heads
  • Specifications don't cover recent key decisions
  • No roadmap
  • No regular updates on project status
  • Lack of formalized relevant requirements

Solution

  • Formalize knowledge into the specification or tickets
  • Map specification into epic tickets for road mapping and structured work planning
  • Schedule and perform daily meetings to track progress and blockers
  • Track new client requests through tickets prioritized in the backlog

Miscommunication with a Client

Issue

The client insists on one thing based on past discussions, while the vendor refers to another.

Solution

  • Follow up all calls and meetings with a protocol outlining decisions, action points, and responsible parties
  • Regularly show delivered features to the client to gather feedback

No Track of Recently Made Decisions

Issue

  • Meetings leave vague outcomes in participants' minds
  • Difficulties in referring to past agreements

Solution

Similar to the previous point:

  • Follow up all calls and meetings with a protocol outlining decisions, action points, and responsible parties
  • Regularly show delivered features to the client to gather feedback

No Clear Plan on Deliverables and Timeline

Issue

  • The project looks like a single piece of work without clear deliverables
  • Splits exist but only as oral agreements between team members
  • Change requests and delivery date shifts are not tracked

Solution

  • Formalize knowledge on the level of tickets
  • Map specification to tickets and create epic tasks that are later split into smaller tasks
  • Carefully listen to client change requests, document them as user stories, and prioritize them in the backlog
  • Communicate changes in delivery schedule, budget increases, or functionality drops when client insists on immediate changes

No Verification of Working Software in Repository

Issue

  • Each code change or commit is a potential bug
  • Verifying compilation and stability on each commit is crucial
  • Lack of such verification is a serious issue

Solution

  • Set up a Continuous Integration/Continuous Deployment (CI/CD) process
  • Close the master branch from writing and work in branches merged via Pull Requests and code review

Unclear Instructions for Building and Deploying Project

Issue

  • Repository access is not enough to compile and deploy the project
  • Despite Docker usage, many optional features, configuration files, and shell scripts complicate deployment

Solution

  • Provide a clear and comprehensive README file describing how to compile and deploy the project
  • Resolve issues to fully automate the process

Dependency on Heroism of Individual Team Members

Issue

  • Engineers face new tasks that require invention
  • Organizational and business challenges complicate work
  • Requirements change due to evolving business environment
  • engineers always face with new tasks for them; engineering process often is not a set of steps to complete to get things done, but an invention of a new thing
  • besides engineering challenges organizational and business challenges makes this work even more complicated
  • even if all parties do an excellent work to clarify and write down all requirements and expectations, a few months later business environment will change which will lead to changes in requirements
  • in this way engineering process doesn't look like a factory where you put resources at the start and get result at the end, but an innovation process with multiple iterations where each round brings new requirements to a product and evolves it with time

Solution

Our ability to deliver a product to a client should not depend solely on the personal heroism of each team member. It should be based on a systematic set of actions, by performing which we will steadily move forward, minimizing known risks.

One of the issues with this is applying the whole set of the best practices becomes very costly. One american engineering company who do this and with whom I had a chance to work with sell their engineer hour by 250 USD per a team member.

A possible solution for this is to start with minimum set of such practices and gradually extend it to meet a growing complexity of the project by spotting rising issues and choose an appropriate solution for them.

In this way each project would require their own set of measures which makes its unique in terms of organization and management. That's why I warn you to use this set of measures described here as a reference, but not as a ready-to-implement action plan.

Stakeholders Not Clearly Identified and Involved

Issue

  • Negative feedback and change requests close to delivery date
  • Work closely with one stakeholder but not with the actual users
  • Requirements of actual users not considered

Solution

  • Set up communication channels with all key stakeholders
  • Regularly show the current product version and gather feedback
  • Gather requirements from all potential users to mitigate implementation issues

A lot of changes in requirements on the fly when deadline and budget are fixed

Issue

  • New requests come from the client when a budget and a deadline are fixed.
  • Work on the math module is a true R&D with constant adjustments.
  • Database schema evolving with years and new requests impacting the whole project.
  • The client is a government company, work with whom is specially regulated by federal law to mitigate risks of corruption, bringing restrictions on types of contracts we could sign.

Solution

The issue with changing requirements is known and even natural for software development. Even if all parties do an excellent job to clarify and document all requirements and expectations, a few months later, the business environment will change, leading to changes in requirements.

The change in requirements is normal. What is not normal is attempting to fit such work within the boundaries of an inappropriate contract.

The industry has developed a set of practices (Agile, Scrum, Lean principles) to meet such needs. Dedicated team or time & materials types of contracts should be used for this, but Russian federal law doesn't allow this.

It might sound odd, but the lack of experience working on projects with a corrupt client (one who used this project to steal money) doesn't allow me to give a proper recommendation on how to cope with this. However, the following roadmap initially looks right:

  • Initiate lobbying for a new law that would better fit the nature of software development.
  • Meanwhile, do your best to fit the reality of custom software development into the existing forms of law regulations.
  • Do it by iterations, where each iteration brings the product closer to a final product.
  • Freeze changes in requirements until the end of each iteration, with the following change of the scope, budget, and deadline.
  • Build mutually trusted relationships between all involved parties in the software delivery.

Unexpected illness of a key team member with loss of valuable knowledge besides team's velocity

Issue

  • One of the team members had been injured with hospitalization for a month or more.
  • It caused a loss of valuable knowledge about the project.
  • It also caused a decrease in the team velocity to deliver new functionality.
  • Generally, it is known as a bus factor.

Solution

How would you deliver the project when one or more of your team members get 'crashed under the bus'? That's how this risk is formalized in the working environments I have experienced. Answering this question helps you proactively develop a set of steps to mitigate this risk.

When you lose a team member for any reason, you don't just lose a worker; you lose unique project and business knowledge this person has together with their technical expertise, which takes years to gain and is sometimes quite unique, making it difficult to replace. That's why retaining your talents is even more important than finding and hiring them. However, there are also sets of practices that minimize loss from a team member's temporary or permanent leave.

  • First, formalize your knowledge about the product and requirements in the form of specifications, tickets with user stories extended by technical requirements, and automated tests, which serve as actual requirements for the software.
  • Second, organize your development process to ensure all your code appearing in your master branch passes through a code review by at least one team member, preferably more. This fosters the sharing of project knowledge with other team members.
  • Finally, consider regular rotations of developers every two days among assigned tasks. This means two days of working on one feature, and then switching tasks with a colleague, so everyone would know the whole codebase. No one should be assigned to a single feature without much knowledge about the rest of the codebase. Important points:
    • Tickets should be well-prepared and within epic tickets which define a whole feature of the product.
    • Engineers, at the end of the workday or just before the switch, should perform a WIP (work-in-progress) commit of all their current work in the repo.
    • Switch between tasks should be made on different features; there is no sense in exchanging tasks within the same feature.

This technique is actively used by the Pivotal Labs team.

Government regulations of work with state companies which don't match well with how software projects should be developed and delivered

Issue

  • The client is a government company, work with whom is specially regulated by federal law to mitigate risks of corruption, bringing restrictions on types of contracts we could sign.
  • This does not match well with the nature of custom software development and has a negative impact on our ability to deliver a product.

Solution

The issue with changing requirements is known and even natural for software development. Even if all parties do an excellent job to clarify and document all requirements and expectations, a few months later, the business environment will change, leading to changes in requirements.

The change in requirements is normal. What is not normal is attempting to fit such work within the boundaries of an inappropriate contract.

I already shared my view on a possible solution earlier, but I would repeat it here in a separate paragraph to emphasize its importance -- it seems to be the root cause of many troubles this project faces.

The industry has developed a set of practices (Agile, Scrum, Lean principles) to meet such needs. Dedicated team or time & materials types of contracts should be used for this, but Russian federal law doesn't allow this.

It might sound odd, but the lack of experience working on projects with a corrupt client (one who used this project to steal money) doesn't allow me to give a proper recommendation on how to cope with this. However, the following roadmap initially looks right:

  • Initiate lobbying for a new law that would better fit the nature of software development.
  • Meanwhile, do your best to fit the reality of custom software development into the existing forms of law regulations.
  • Do it by iterations, where each iteration brings the product closer to a final product.
  • Freeze changes in requirements until the end of each iteration, with the following change of the scope, budget, and deadline.
  • Build mutually trusted relationships between all involved parties in the software delivery.


Related publications: