The Measurement Problem That Undermines Programme Governance
The standard approach to architecture modernisation ROI measurement waits for the programme to complete before assessing whether it delivered on its business case. This is the measurement approach that most fails the programmes it is designed to govern.
A programme that is measured only at completion cannot be course-corrected while it is in flight. The scope expansion that is diluting the ROI is visible only in the final accounting. The technical decisions that are creating the operating cost overruns compared to the business case projections are visible only when the operating cost bill arrives after go-live. The benefits that were projected to materialise from the modernised architecture are visible only if they were ever measured against a baseline established before the programme started.
By the time the post-programme ROI assessment is complete, the sponsoring executive has moved on, the programme team has disbanded, and the lessons are available for the next programme rather than the current one. The measurement that would have changed the programme’s outcome was never performed.
The third article in this architecture modernisation series provides the continuous ROI measurement framework that makes programme performance visible throughout execution. This is the framework that allows programme sponsors to manage modernisation programmes as investments rather than as technical projects.
The Baseline That Makes Everything Possible
Continuous ROI measurement requires a baseline established before the programme begins. This sounds obvious. It is consistently not done. The reasons are structural: the programme team is focused on getting the programme started, the baseline data may not be readily available, and the investment in establishing the baseline does not produce visible output that the programme can point to in its early governance reviews.
The baseline for an architecture modernisation programme should cover four dimensions.
Delivery velocity baseline: the current lead time from code commit to production deployment, the deployment frequency, and the engineering time allocated to architectural overhead (release coordination, technical debt remediation, architectural workaround maintenance). This baseline provides the comparison against which post-modernisation delivery velocity improvements will be measured.
Operational cost baseline: the fully loaded operating cost of the current architecture, including infrastructure, licensing, engineering support, and the incident overhead attributable to the architecture’s reliability characteristics. This baseline provides the comparison for operating cost changes after modernisation.
Incident profile baseline: the frequency, duration, and business impact of production incidents attributable to the current architecture over the twelve months prior to programme start. This baseline provides the comparison for post-modernisation reliability improvements.
Business capability baseline: the list of business capabilities that the current architecture cannot support or supports with significant constraint. These are the potential benefits from the modernised architecture that the business case is built on. Establishing them as a baseline allows the programme to track how many have been unlocked as the modernisation progresses.
The Leading Indicators That Predict Outcome
The business case benefits from an architecture modernisation programme are trailing indicators: they appear after the new architecture is operational and the organisation has had time to realise them. Managing a programme against trailing indicators alone means that problems are only visible after they have manifested.
Leading indicators are the programme health signals that predict whether the programme will deliver its business case benefits. Three categories of leading indicator are reliably predictive.
Migration completeness and pace measures whether the programme is completing migrations on schedule, and specifically whether the most complex and dependency-laden migrations are being addressed in the sequence that the programme plan requires. The programme that is ahead of plan on the simple migrations and behind on the complex ones is in a different health state than its overall migration percentage suggests. The pattern of migration completeness reveals whether the programme’s risk is front-loaded or back-loaded.
Parallel operation overhead tracks the actual cost of operating the old and new architectures simultaneously, compared to the business case estimate. If the parallel operation overhead is higher than estimated — because the migration of the long-tail workloads is taking longer than planned — this is the leading indicator of the cost overrun that will appear in the final programme accounting. Catching this indicator early allows the programme to adjust: accelerating the migration of the most costly parallel workloads, or revising the business case to reflect the actual transition costs.
Early architecture benefit realisation measures whether the components of the modernised architecture that are operational are producing the benefits they were designed to produce. If the first product team to migrate to the new architecture has not experienced the delivery velocity improvement projected in the business case, this is an important signal. Either the architecture is not performing as expected, or the team’s processes have not adapted to take advantage of the new architecture’s capabilities, or the velocity improvement will appear once more of the architecture is complete. Understanding which of these is occurring allows the programme to address the root cause before the pattern repeats across the full migration.
The Reporting Structure for Sponsor and Board
The ROI measurement framework is only valuable if it produces information that reaches the sponsor and board in a form they can act on. The reporting structure that achieves this has two levels.
The monthly programme performance report for the sponsor includes the three leading indicator categories, with trend data that shows whether the programme is on track to deliver its business case, ahead of track in specific areas, or behind track in specific areas that require attention. The report should explicitly compare current performance to the business case projections rather than reporting absolute performance data without a reference point.
The quarterly board report provides the financial summary: cumulative investment to date compared to the planned investment at this programme stage, expected final cost based on current trajectory, the benefits realised to date compared to the timeline projection in the business case, and the expected final benefit based on current trajectory. The board that receives this report has the information required to make governance decisions about the programme’s continuation: whether the programme is performing well enough to justify continued investment, whether a scope adjustment is required, or whether the business case assumptions need to be revised in light of actual experience.
The Governance Change That Continuous Measurement Requires
Implementing continuous ROI measurement for architecture modernisation programmes requires a governance change that most technology organisations have not made: treating architecture modernisation as a capital investment with continuous financial performance tracking, rather than as a technical project with milestone-based progress reporting.
This governance change requires the CFO to be a genuine partner in the programme governance, reviewing the financial performance reporting alongside the technical milestones. It requires the programme sponsor to have financial accountability for the ROI, not just delivery accountability for the timeline and scope. And it requires the measurement infrastructure to be established before the programme begins rather than during or after it.
The architecture modernisation series has argued across three articles that the financial rigour applied to the modernisation decision, the programme management discipline applied to execution, and the continuous measurement that makes financial performance visible are together the determinants of whether the investment produces its promised return.
Programmes that apply this rigour produce better financial outcomes than those that do not. The evidence is in the post-programme assessments of the programmes that have both kinds of governance. The governance choice is available to every programme sponsor before they sign the investment commitment.