Building Block Architecture – Part 1

Background

I propose developing a Building Block Architecture (BBA) activity to break down complex programmes involving IT delivery, before procurement, with delivery by smaller, independently managed procurements.  This approach, along with rigorous Programme Management and the use of a consistent development methodology, will reduce the risk of poor outcomes and overrun on IT delivery projects.

Introduction

Developing IT and software solutions is always challenging. In many cases the initial budget and timescales are widely exceeded, hopelessly underachieving the desired outcomes / benefits, or even result in programmes being abandoned completely.

In my view, there is a missing element within the typical procurement / development process. This element (Building Block Architecture) is needed in order to reduce development risk and ensure successful delivery of programmes with significant IT components.

Packages verses Bespoke applications

In an ideal world, it would be possible for organisations to buy a “Commercial off-the-shelf” (COTS) package to meet complex business requirements – but this is rarely possible in government IT and, indeed, many other sectors.

This leads inevitably to a “bespoke system” being developed – usually based on a number of core components (sometimes including a COTS product) and significant amount of development work to integrate and produce an acceptable solution.  The bespoke solutions often then need to communicate with other systems, leading to a bewildering array of interfaces that have only been partially identified at procurement, and technical and business dependencies that make changes particularly difficult.  These development projects are inevitably complex and difficult to deliver, with many unknowns (known and unknown!) when the project is initiated (and usually) procured.

Programme Management and Development Methodology

It should be a given that rigorous Programme Management and an appropriate Development Methodology (e.g. Agile or Waterfall or a combination) are fundamental to delivering complex programmes involving bespoke software development.  Good Programme management practices emerged from recognising that project management was not best suited to dealing with uncertainty.  Programme management allows the overall endeavour to be broken down into smaller delivered components or projects (and work packages), themselves of shorter duration and smaller budget.

Delivery Using Small Teams

Once a programme of software development reaches a certain size, significant problems emerge and it can be very difficult to scale successfully.

Anyone who has been involved with IT development will know that there is a huge variation in capability and vision of developers and architects.  The reality is that most successful software developments are coded and  orchestrated by a small number of top performing developers.  (See reference 1).

Of course, for companies employing full time staff who move from one project to another, it will be relatively clear who the top performers  are.  However, many government projects consist of teams hastily thrown together from individual contractors and staff from recognised systems integrators.

When a development team is large, several risks increase:

1. too many top performers, with different opinions and approaches;
2. top performers do not emerge, because the team is too large or dysfunctional;
3. Incompetent or unmotivated ‘lower performing’ staff can hide more easily, while being unproductive;
4. Inertia to ‘get on with things’ and reluctance to stand back from the detail and spend time truly understanding the business requirements (and pushing back on some things), planning the overall architecture and development approach.

One of the mistakes in Government IT with the use of Agile approaches has been the setting up of teams from multiple suppliers where there are no top performers and the assumption that Agile alone is sufficient to sort things out!

My proposition is that an individual development team should never be larger than 20 people in order to maintain a manageable team – this should include business analysts and designers.

However, many projects may require more than 20 developers.  So, to deliver successfully with small teams in this ‘programme of projects’ world, I believe that there is an additional and fundamental component required – for these larger programmes, we need to break down the development into smaller, discrete pieces of work – each piece of work called a ‘Building Block’.  In addition, an integrating IT architectural ‘guiding mind’ is also required to provide the vison and oversee and integrate the whole.

Delivering a Building Block Architecture using APIs

Rather than a single technology delivery workstream within a Programme, delivery should be broken down into a number of discrete deliveries of Building Blocks, each delivered by a separate team and with well-defined starts and ends (ideally coincident with the individual projects or work packages).

For some Building Blocks that are critical to the delivery of the programme, it could be considered asking two teams to develop the same Building Block independently, in much the same way that safety critical systems include built in redundancy.

The Building Blocks, when complete, need to be able to operate alongside each other.  There has been an enormous growth in the use of Application Programming Interfaces (APIs) between systems – this can allows effective and efficient communicate between Building Blocks.  The reason we can do this now is that APIs are fundamental to the way that systems are currently developed – rather than a single homogenous system, we normally develop separate front end and back end components (and “microservices”) and use APIs to communicate between the front and back ends.  The APIs are not just for communication between different systems, but between the different components of a single system.

Government often buys IT solutions without recognising the importance and potential of APIs to simplify the system. Typically, there may a definition of the integration required with third party systems (usually incomplete in my experience) but few attempts are made to use APIs to modularise delivery of the solution as a design strategy.

My proposition is that designers can exploit APIs and services to reduce risk in the development of large complex IT programmes by following this Building Block approach.

Practical Application of Building Block Architecture using APIs

How is this achieved in practice?  

Typically, the current Procurement Process looks as follows:

1. a business area expresses a requirement for a new or replacement system and develops requirements;
2. Procurement articulate the scope of the system with minimal description of external APIs and a supplier is selected;
3. A single supplier attempts to deliver the complete system.

In this process, there is only peripheral “Architecture Assurance” during procurement – hopefully more during delivery.

My proposal is that this should be revised to a “new Procurement Process” so there is active involvement from Business, Solution and Technical Architects during the pre-procurement phase (see below) :

1. A Business area outlines requirements;
2. “Architects” breaks down the requirements into discrete Building Blocks, with outline APIs between blocks – clearly differentiating the potential for COTS versus (riskier) development;
3. Procurement describe scope as a number of discrete Building Blocks which are procured, ideally using simple and quick frameworks e.g. GCloud, DOS and potentially from multiple suppliers;
4. One or more suppliers deliver the Building Blocks for the solution (one could also act as the overall integrator and manage the development of the APIs).

As I mentioned earlier, programme and project managers already break down complex activities into smaller components, but it is often not possible to do this effectively in the procurement process.

Building Block Architects

It may not be clear who has the capabilities to carry out the task of breaking down functions into Building Blocks prior to the procurement processes.  Most government departments employ ‘technical architects’ to provide an advisory / assurance role on procurements but they are usually assigned once the procurement is underway (I have performed this role for a number of government departments).

There is a requirement for a group of people – let’s call them “Building Block Architects” – whose job it is to break down the delivery aspect of a programme into Building Blocks before any procurement takes place.

Where are Programmes going to find Building Block Architects

There is a practice known as Enterprise Architecture  (EA) which can help here.  EA was devised to help organise and map an organisation and its IT systems and business process, in order to improve decision making when business and IT change is required.   EA looks at business as well as technical architectures.

Enterprise Architecture is a sensible and mature answer to some of the issues faced by large and complex organisations and programmes.  The problem with EA is that it is often a real challenge to develop a business case for investing in EA – the justification for a team of people working at the centre of an organisation, mapping the organisation but not aligned to a business unit or delivery programme.  As a  result, EA has failed to gain traction with business sponsors, although many aspects of their methodologies and outputs will probably have been incorporated into programme deliveries (often without acknowledgement).

‘Enterprise architects’ have the tools, methodologies (and many should have the capability) to deliver what is needed to act as ‘Building Block Architects’.   Their role should start pre-procurement, to:

• break down a delivery into a manageable number of logical components- these can be both Business Components and IT components, ensuring that any size constraints are met;
• define the Application Programming Interfaces and business interfaces between components;
• break down the raw business requirements to match the building block architecture;
• determine the sequence required for delivery, prioritisation, and effort estimates.

This approach will lead to more successful technical delivery – it should also have benefits for Programme and Project Management.  A building block approach would allow better visibility of progress – and would allow identification of critical components that may need to be doubled up to minimise risk.

Practical Application of Building Block Architecture using APIs

The Building Block Architect role would not stop at procurement completion, but should extend to delivery phases in order to:

• Ensure that the building block principles are adhered to, and that the architecture description is modified in line with changes, and is coherent at all times;
• Ensure that delivered Building Blocks meet the requirements;
• Provide configuration management of a repository of descriptions of building blocks and their interfaces.

While the aim of this approach is to improve IT delivery, it would have the side effect of providing a more visible and useful role for Enterprise Architects within organisations, and improve traction for Enterprise Architecture, in general.

In part 2, I provide more detail on the approach including benefits and principles.