Agile rewrite of an existing system
The times I’ve worked on a project where the scope is "rebuild the existing implementation, but with new tool / techonology X", I’ve encountered various pitfalls that make these projects much harder than they need to be.
Let me offer some tips on how to deal with them.
As what is, exactly?
Problem
"As-is" migrations tend to start from the assumption that there is knowledge of what "as-is" entails.
That assumption often turns out to be wrong.
Developers are onboarded and pointed to specs that turn out to be non-existent, incomplete or wrong.
People who were involved with building the original system have left, remember only fragments of what the original intent was, or refuse to be involved with anything reminding them of whatever the horror project was that spawned it.
Tips
-
Before you assemble your development team, ensure you have domain experts to explain to your team all that is. Ideally, such an expert is part of your team.
-
Consider doing something like Event Storming to find out the big picture and hotspots so you can validate your idea of what the system does.
-
Be very clear about the difference between "as-is" and "well if we’re going to be touching this, we might as well add these features that never made it into the original product" and "don’t invest in migrating feature X, because nobody uses it". When you’re doing domain driven design and/or model driven engineering, you’ll notice a story about your system quickly becomes convoluted when you’re mixing up these views of your problem & solution domain.
-
Beware that there are two systems where scope creep lurks: the existing system and the one you’re building. That means there might be two moving targets.
-
Be very clear about the difference between "as-is" and "as-intended". People using your system will invent features. You made a SPA talking to a REST API? Customers may have made their own applications talking to it, expecting it to be stable. When rebuilding "as-is", you need to cater for actual use; not (only) for intended use.
Should you deliver increments that constitute less value than what you already have?
Problem
The agile way of working is great for being able to deliver increments quickly and getting feedback from the stakeholders to learn what to improve.
It’s not so easy when the system you’re building is a new version of something that’s already been in use.
We all know the pictures of delivering a bike, tricycle, car, etc to explain that you want to deliver value incrementally, but when your customers are already driving a full featured car, it’s going to be hard to deliver anything less than that.
Tips
-
Technical debt should be subtracted from a system’s value. Knowing why the existing system needs to be (partially) decommissioned helps determine what the relative value of new increments is. Did it lack ability to be instrumented for monitoring? Did it lack a delivery pipeline making changes hard? Did it have tons of undocumented opaque stored procedures? Did it have inexplicable memory leaks causing unpredictable crashes and data loss? Was it wholly dependent on outdated or insecure technology? These reasons matter as they imply the assumed value of the existing system is in fact significantly lower than the mere functionality it offers.
-
Separate deployment from release. You want to be practiced in delivery; a deployment should not pose any surprises. The agile way of working boils down to being able to makes changes quickly and get feedback on those changes. Scrum talks about delivering "an increment that could go live". So as long as you get new ideas in the hands of your customers and get them to give feedback, you can leave the existing system in place for whatever they can’t live without.
-
You can use the strangler pattern to see if you can isolate and slice off functional parts of the existing system, and replace them with new increments. This practice is greatly facilitated by having a modular architecture. The biggest challenge here is data communication between the old and the new: for every slice you cut off, you’ll need to adjust the part you leave behind to be able to interact with the new. The question is then always: is it worth the investment to do this, or should we leave the old system up and running in its entirety until we’ve rebuilt all its parts?
-
Manage stakeholders to ensure they’re on board for the long haul, as abandoning it halfway will leave them with a more complicated landscape than when you started.
Do you know why the initial system was written the way it was?
Problem
There can be a variety of reasons why the existing system has the flaws that you intend to address.
Tips
When you’re a developer that’s brought in to build a system that will have none of its predecessor’s shortcomings, you may be eager to show how much better you are than whoever built it.
Not so fast!
I once picked up a saying that I later traced back to Marshall Goldsmith: "What got you here won’t get you there", and it applies to whenever people try to change their situation without properly considering how they got into that situation. Most commonly, people do not look at the organisation context from within which a team produced a system. If management has a command and control attitude toward teams, then replacing a development team without changing management is likely to produce the same systems with the same flaws as the one they’re replacing.
In fact, you may want to consider if taking away the reasons the system ended up in the undesirable state in the first place will allow you to simply fix the existing system rather than rebuild it.