Does your Software Architecture tell a Story?

Have you wondered why we often call the Software Architecture an Art when it is well and truly an technology discipline?   

Well, there might be many reasons, starting from this being a figure of speech to being a tip of cap to the venerated tradition of construction architecture, which over millennia have been among the highest expression of artistic instincts.  

However I would very much like to think that Software Architecture itself affords its practitioners ample scope to exercise their right-brain.  Starting from the Conceptualization, to Evolution, to Completion and Reviews, there are multiple opportunities for an Architect to exercise creative thinking.  

In the past we have explored how Design Thinking can be a harbinger of innovation and creativity in software architect. Today we explore another such opportunity. 

A common question that an architect needs to often answer is whether the software architecture that is being worked on, is ready for the next phase.  Let us explore a non-traditional way that relies on intuition and judgement to answer this question that I have found quite useful and use regularly.

Why are we even discussing the question of Architecture Readiness?

Today the world follows Agile Architecture Practices.  Consequently, the Architecture is never really done-done. Instead, it evolves over time. Architecture is no-longer a front-loaded body of work like in the heyday of waterfall models, where months were spent upfront.  

Given this, is the question, Is the Architecture ready, a valid one, even?

Well the answer certainly is an Yes! 

But, why?

Because, whether it is a two-pizza team working in a mode of extreme agility, or a multi-geography team-of-teams working in a more deliberate manner; the availability of an “ready” architectural blueprint is often the single most important element that ensures success of the venture by providing the unifying theme. It is the element that separates a well-organized execution from utter chaos.  I would liken the architecture to a musical Raga, that in a similar fashion provides the foundation, the common theme, to a team of talented musicians working in-concert.

Furthermore, in many, if not most, of business focused organizations, architectural readiness is also a trigger for assignment of teams, resources, and date commitments.  

The question therefore is very valid and will definitely be asked.

And as architects, we are expected have a clear notion about the readiness of our architecture.  

But, how do you even know if your Architecture is Ready?

As I mentioned before this is where an architect’s judgement and maturity come sharply into focus, because many times when you are asked this question you have been working on this solution for a while. You perhaps already have domain models, have identified services and interaction sequences, have drawn up deployment strategies and what not.

So how do you know if you are ready enough to hand this over to be taken forward?  How do you save yourself and your project from having gaping holes that would eventually lead to chaos? Or at the least rework and missed commitments?

Well, there is a Two Parts answer to this question:

First and foremost, there is the Objective Evaluation Model for Architecture aka the traditional way, which consists of steps such as:

• Ascertaining you have all the right views of your architecture model such as Business, Application, Data, Deployment etc. This is a fail-safe mechanism that that ensures each aspect of the solution design is given a proper treatment. Even in a evolutionary model, it is prudent to have the different view available as living documents.
• Validation against standards Architecture validation checklists.  Many universities and standards bodies have published such checklists which ensure key concerns haven’t been missed out.  Extremely useful in ensuring that the non-functional aspects are handled adequately too. 

I have been a strong proponent of following these tools mandatorily for every significant architecture exercise.  In fact, I plan to do a subsequent blogpost on the best practices around these two objective evaluation steps. 

However, what I want to cover today is the second part. It is, what I call, the Subjective Evaluation of Software Architectures. While the objective criteria ensure that the structure, standards, and practices followed in your architecture are proper, the Subjective Criteria ensures that the Architecture, whatever slice of it is being evaluated, makes logical sense.

Let us dig deeper!

The Basis of Subjective Architecture Evaluation Model

The basis of this model is what we started this discussion with, that the process of Architecture definition is as much an art as it is science.  That the patterns, practices, standards are essentially like raw materials or at best pre-cast construction material, the solution architecture is the narrative that glues them together to form a coherent structure.

The Subjective Architecture Evaluation Model posits that until you have a coherent narrative that you can use to express both the functional and non-functional aspect about the architecture, the architecture is not ready.  In other words, your architecture should be able to tell a complete story or as some like to say paint a complete picture.

A coherent narrative has three parts to it:

• Structure
• Tangibility
• Optimality

Let us explore each of these points individually

Structure

The first and foremost aspect of a Coherent Architecture is that it is structurally complete. For whichever slice it is supposed to handle one must be able to produce evidence that all the stated and implied requirements for that slice has been addressed in the architecture.  Where I find most architectures trip a bit is handling the implied requirements. 

The approach that I have found most useful is the following

1. Performing a in-depth analysis of the "implications" of the stated requirements and listing those out before starting working on the architecture. Doing a review (or many) with the requirements or business owners is something I highly recommend. 
2. Addressing all the stated and implied requirements in the Application Architecture, which must identify the components at a sufficiently detailed level of granularity
3. Traceability matrix between the requirements for a slice and the components of the architecture

The subjective part of course is the analysis of implications which I have seen getting missed out once too often. 

Let's take a simplified example, just to illustrate the point:

Let's assume we are trying to design a make-believe system to manage Tier 1 partners (distributors) and their relationship with Tier 2 partner (resellers).  Let's assume that the requirement is that each region has a distributor from which the reseller is expected to collect inventory based on agreements that are made time to time between the distributor, reseller and the Original Equipment Manufacturer.  The distributor is responsible for managing their own inventory and the relationship with the resellers. 

How do we proceed? Well, one obvious way would clearly be understand the stated requirement and create a structure that meets these requirements. 

Here, we can clearly see that the data attributes that need to be managed are at different levels of abstraction. The Distributor level artifacts such as inventory etc. needs to be contained within Distributor Resource whereas a different resource such a Reseller Account would need to contain the the Reseller Level attributes. The Distributor resource would also contain a list of Reseller Objects which represents its Tier 2.    So in this design, we create a Distributor Resource, which contains the Reseller Resource (one or more) which in turn contains it own contained resource, so on and so forth.  This structure would meet the stated requirements as well ensure we have a proper collection of single responsibility resources.  

We deploy our service and everyone is super happy UNTIL someone comes as tells we now have a second distributor in the same region who would need to have relation with some of the same resellers that the 1st Distributor has relation with.  

With our Reseller now a contained Object of the 1st Distributor, how do we handle this?  Secondly, how do we handle the fact that 1st Distributor and 2nd Distributor would have their own independent set of inventory items that Reseller needs to collect, or that there are separate agreements for each relationship?   Create a second hierarchy the second Distributor with a new entity to represent an existing reseller? Clearly non-optimal and would be a big operational headache.  So what went wrong?

We clearly missed out the unstated but obvious requirement that it is normal for a reseller to handle multiple product lines and that a reseller would most likely have relationship with multiple distributors. In fact we must allow multiple distributors for the same product lines and not paint ourselves into a corner. 

This example is a made-up one, just to show that depending only on stated requirement is never a great idea. We are expected to dig deeper.

A creative extrapolation of the implied requirements, in my opinion a mandatory step for generating a structurally complete architecture. An activity that demands creative thinking.  

Additionally I can't emphasize enough the “sufficiently detailed level of granularity” aspect.  There is no pre-defined approach here.  For instance, mapping all the requirements into a bock which is named the “glorious monolith” is of never of any use. On the other hand mapping them into exact class that implements the functionality is too much at this state and would only delay design. That level of traceability is useable during the design/dev phases.

The most important trick here is to spend sufficient time analyzing and extrapolating the requirements to read between the lines and just regarding the architecture diagram to work out how using the structure in front of you, you would address each use-case. I normally try to ensure all the stated and implied requirements can be effectively traced to a combination of service, service cluster or in some cases an infra component (which, of course, in most cases is a service) that make up this structure. 

The Moral: Uncover the plot complexities and ensure that all plot complexities are covered well in the structure of the story.

Tangibility

Tangibility is closely connected with the completeness but is more nuanced.  You could theoretically have a “complete” architecture, but the components could still lack tangibility.   It essentially boils down to how much detail you have created around the architectural constructs where it counts.  

A simple example could be that for a quote datastore you architecture could have any of the following level of details:

• The Quote Data is stored in a Data Store
• The Quote Data is a multi-layered data model that is stored in the Data Store. Quote Data typically consists of the Opportunity Id, Customer Id, Sales Org, Quote Owner, Locations that are covered in the Quote, the Products, Services associated with each Location, the Cost and Price information at different layers of the Quote
• The actual Data Model of the Quote which would be stored in the Data Store
• And many other variations.

So, what is the right of detail?  

The answer is, as expected, it is subjective and situational.  If the slice you are working on is Quoting system, the decision would be to veer towards richer detail.  However, if the slice under consideration is something that only uses a quoting system peripherally, all that may be of importance would be the API through which the quote service is accessed, how the quote data gets stored can and perhaps should be opaque.

The trick in this case is knowing what to focus on and have a richer amount of detail.  Again, I would liken this to storytelling, you create a complete canvas of situations and characters, however the strength of your narrative is on how real, and tangible are your central characters and the situations around them.

The Moral: In your story know where to use a real person versus a cardboard cutout. 

Optimality

This is perhaps the most misunderstood aspect of the Subjective Architecture Evaluation Model, since it depends a lot on your instinct and intuition, or as they say in the trade “your nose”.  This intuition develops only through practice of your profession.  This intuition how senior doctors seem to magically know a health issue even before the test results have come in, and when they do come in, in most case they serve to confirm. A similar intuition is seen in kindergarten teachers who seem to have a magical way to predict a mischief before it has happened.   Magical maybe, but not unscientific, what this simply points to is that years of practice has trained you to notice patterns, pathways and nuances that someone less experience might overlook.

Here, I don’t have a trick to share, but examples of a few things that I have typically look at in the past to ensure optimality. As you play out the use cases as discussed earlier, a few things to watch out for are:

• For a user centric outcome, how many hops the needed, how many services participate and what is the level of parallelism in achieving outcomes?
• What is the amount of data finally needed for outcome, where is it stored, how far must it traverse to get to where it is processed, how much data actually moves, what can be cached?
• What are the optimization trade-offs between computation and storage, where to draw line?
• What part of the out can be just in time, what should be done offline?
• Where could concurrency issue arise, what resource, data might be the object of contention?
• What could be a bottleneck for scalability, what could come in the way of future extensions?

These are just examples. The fact is as you play out your use cases, keep an eye out on whether things are happening in an optimal fashion. This needs one to keep an eye on the evolutionary path as well, not just in terms of the subsequent slices to come but also the overall evolution of the story that the architecture is trying to narrate.

The Moral: In your narrative, avoid the urge to use a cannon to shoot a fly.  

***

And there you have it. The theory of Subjective Architecture Evaluation Model.  The discerning would have no doubt noticed by now that what I am proposing here is primarily a thorough and focused review by the way of playing out the use-cases and ensuring the business objectives are met by looking beyond the obvious stated requirements.  

At the end, that's what our teams and stakeholders expect of us.   

Other Uses

This technique can also be used when you are reviewing architectures developed by others.  I have followed it regularly with my team of architects and with good results.  In fact, not just in reviews of work done in-house, this technique lends itself to review of third-party product architectures equally well.

The Subjective Architecture Evaluation Model with its focus on Structure, Tangibility, Optimality is a powerful tool in the hands of the architect to self-certify the architecture as being ready for use as well as a very useful means to ensure completeness of Architectural reviews.  

***

What Creative Methods do you use as a Software Architect?  Please do share. I would look forward to hearing from you.