After the success of agile transformations that gave organisations the ability to respond to a change in the market more quickly and frequently, the last decade saw DevOps emerge as another magic wand. While valuable by itself, DevOps should actually be treated as one of many aspects of the Shift Left paradigm applied to the full software delivery life cycle. Patrick Debois, Andrew Shafer, John Allspaw and Paul Hammond coined the word DevOps around 2008. It comes down to the insight that combining the disciplines of development and IT operations by removing the thresholds between them allows for the elimination of waste and improvement of quality and speed of software delivery. The concept of Shift Left comes from the field of testing and boils down to the idea that the earlier in the software life cycle a fault is found, the cheaper it is to fix it. The accompanying model for this concept is that the process for delivering software goes from left to right as it goes from concept to cash.

Traditional left to right view of a software delivery pipeline

Software delivery pipeline

Going from left to right along the delivery pipeline, the steps represent various fields of expertise in software delivery, and each field has its own best practices that support a Shift Left mindset.

Design

It is a pitfall to think that a good practice in the light of Shift Left is to have a waterfall approach where a design should be complete before building. Avoiding this pitfall does not mean foregoing design entirely. "Big Design Up Front is dumb. Doing no design up front is even dumber" as many people quote Dave Thomas. In this context I will take design to mean the 'why, 'what' and 'how' of what a development team will build, before they’re building it. It is not actually design that is to be shifted left (as it already is - by design, heh), but the means to get feedback on that design.

Shift left feedback on design

shift left pipeline design 1

  • Scrum: The Product Owner is part of the team and offers a way to regularly describe, evaluate and readjust business requirements. The product owner is involved as the team implements these requirements, and the product owner can cancel a sprint when the sprint goal no longer serves a valuable business purpose.

  • Data or value driven design: The best idea emerges from empirical data. A good product owner anticipates this and doesn’t only prioritize his ideas but also the means to measure the actual value of his ideas - think of online analytics dashboards and visitor behavior recording. By using A/B or multivariate testing and bandit algorithms: design becomes more concerned about how to measure its value iteratively and building on that incrementally, rather than spending more time on that up front.

  • BizDevOps: As organisations realised that Dev & Ops working together became even more efficient when the business silo was also torn down, bringing business requirements and the measurement of resulting value closer to the teams, some people coined the term BizDevOps. It is in fact a logical consequence of the previous two points.

  • Agile, evolutionary, sacrificial and emergent architecture. There are 'real world' limitations like available technical capability, training facilities, time restrictions warranting standard solutions and financial means. Architecture should take these into consideration and give direction to the collaboration in and between teams, but not impede the ability to respond to progressive insight and emerging needs. Architecture should come with an exit and migration plan for expected future bottlenecks, using patterns like the strangler pattern and concepts like tight cohesion & loose coupling.

  • Design for failure: An architect who anticipates not only change but also failure will take into consideration auto healing, the horizontal scaling rule "1=0; 2=1; 3=2" and ways to continuously monitor functional availability. This allows shifting left the inevitable incident and problem management that comes with the DevOps notion "you build it you run it": "you troubleshoot it".

Development

In Scrum terminology, developer means anyone in the team who contributes to the realization of a product increment that could go live. This might mean someone who writes code, but also a business analyst or tester. For this section, I’ll have it mean someone who writes code.

Shift left quality validation and techical debt reimbursement

shift left pipeline dev

  • Static code analysis: You can rely on the skills of senior developers and their knowledge of common and best practices in your team, but by using static code analysis tools like SonarQube that are supplied with industry standard rules, you can have your code quality validated using a larger body of knowledge and refactor code before its shortcomings become manifest as bugs in production.

  • Boy scout rule: Leave any code you touch in a better state than you found it. Foregoing this practice may result in fear driven development (fear of touching code that you know needs rework or is simply hard to understand) and leaving it untouched increases the impact of an incident causes by such code: effectively, the boy scout rule is shift left thinking applied to writing code.

  • Continuous integration: There are various strategies with regard to merging the code of developers into a single code base. Git flow is a strategy that allows developers to delay integration by staying on a feature branch until the end of the sprint, which may result in integration problems not showing up until the last moment of the sprint. Shifting left integration can be done by mob/pair programming and trunk based development.

  • Automated build: By letting a code commit automatically trigger the central build pipeline, a developer gets feedback asap. Ideally, the build pipeline adopts a "build once, deploy many" strategy, where the artifact built for the development environment is the same as for every other environment.

Testing

Shift left test definition

shift left pipeline test

  • Test driven development (TDD/ BDD / ATDD): In Scrum, the product owner through his involvement knows what a development iteration will approximately bring to his product. In my organisations however, the true acceptance tests happen after delivery of the increment, either directly by a stakeholder or by designated testers. Such procedures often introduce wait time between delivery and deployment. During this time, a development team continues with the next batch of work, and the later the feedback from the acceptance test, the harder and more expensive it is to process it. By making acceptance criteria concrete up front, developers can strive to fulfill the acceptance criteria during development, effectively shifting left acceptance.

  • Load testing: Although an application can be tested to be functional before deployment, what is often overlooked is that in a live environment, the application and infrastructure are subject to peak loads, continuous loads, data volumes and non-compliant data input that impact operational capabilities (think denial of service). To not be surprised by this, a development team can design load tests and data sets that take these matters into consideration.

  • Test automation: Using test automation tools such as JUnit, Geb and Cucumber, tests on various levels of functionality can be scripted to decrease the chance of manual error. Further more, execution of test scripts can be automated to be faster and, more importantly in the context of this article, provide earlier feedback on functional quality.

Operations

Shift left operational concerns

shift left pipeline ops

  • DevOps: In a very simplified way, operations is tasked with keeping a live environment stable, which traditionally results in risk avoidance and consequently change aversion. On the other hand, development is tasked with implementing changes. This conflict of interest has become more apparent with the short iterations introduced with agile development, as ops procedures became more and more of a bottleneck. The solution lies in appreciating the responsibilities that ops are tasked with and anticipating them: Think of log formatting, identifying SIEM, VM, https://en.wikipedia.org/wiki/Technical_surveillance_counter-measures[TSCM] events and integration with appropriate monitoring tools. This may very well mean including ops engineers in the development team to ensure said concerns are taken into consideration during development.

  • Canary release: Risk avoidance can paralyse your product development. Paradoxically and similarly to what was mentioned above concerning design and multivariate testing, it isn’t risk mitigation that you shift left, but the ability to get feedback on risks. With a canary release, you incorporate the ability to get operational feedback during your deployment.

  • Immutable infrastructure as code: Previously ops would log onto a server, deploy an application and make other necessary changes, risking the creation of snowflake servers with unique, hard to trace unique behavior. The solution that DevOps offers is not about giving developers access to servers to make the exact same mistakes. Instead, you shift left both the installation of an application on a server and any changes to that server and its surrounding infrastructure itself (including the use of SaaS or serverless functions). Script them in a way that can and should be triggered by code commits, taken through automated quality checks and ultimately deployed automatically. When using DTAP environments, you test not only functional changes but also the deployment process itself. Like test driven development does for tests, this practice shifts the concerns of deployment to the left.

Security and compliance

DevSecOps has been an attempt to explain that the idea of DevOps is good but security concerns should be explicitly considered in this endeavor. The Shift Left paradigm, when applied to security and compliance, means that a development team shouldn’t fear an audit, but anticipate and plan for it:

Shift left security concerns

shift left pipeline sec

  • Compliance as code: There are various security benchmarks for security compliance, such as STIGs and CIS, that can be applied to applications and infrastructure. These benchmarks can be integrated in the software delivery pipeline and automated. For example, when building a server image using infrastructure as code, an Ansible script can be triggered to verify that a server built from the resulting image is compliant. It can even make corrections resulting in a new compliant image.

  • Static code analysis: Although generic static code analysis tools like SonarQube cover code smells that might be related to security concerns, there are tools that are dedicated to security. Often, the OWASP top 10 is used as a baseline. What needs to be taken into account is that reports of such tools still need to be interpreted and acted on, which requires appropriate subject matter expertise in the development team.

Further Reading

Although many good books have been written about the various practices I’ve described from a perspective that takes Shift Left as its guiding paradigm, the single most important book I can recommend is The Phoenix Project by Gene Kim, Kevin Behr and George Spafford. Even though its prosaic story focuses on DevOps, it touches on crucial aspects that are exemplary of a mindset of kicking the can down the road which is intuitive to far too many people in our industry. And more importantly, it shows how a change in focus and behavior, explained as the Three Ways, help provide a culture where shifting concerns left is the natural way to deliver value.

shadow-left