Digital Journey

The typical architecture representations are diagrams, abstracts, views, and perspectives that are documented formally or informally.

However, a modern architect does not stop there. These representations start at version 0.1 and gets iterated through testing, measuring and validation.

Example: Interface Definitions (a closely observed architectural artifact) goes through several versions.

How to test architecture?

Need to inspect the architecture, to determine, what needs to be tested.

1. Test the concept/assumptions: Search-ability is a quality attribute for an image archive. Do a spike to test the assumptions about what/how. E.g. Spike: Storing meta-data of an image as tags on the file image is sufficient and efficient for the query use-cases; Alternative is to store the tags as a JSON document in NoSQL DB with indexes for efficient search; Alternative is to store tags as a JSON document in ElasticSearch or Apache Solr. This will force sampling the search use-cases and validate that the approach selected will indeed meet the use-cases with desired performance & cost attributes.
2. Test the interface definition: Interface Granularity & Availability is a quality attribute. Create a mock API and push for consumers to consume the API, to test implicit or unsaid requirements. Iterate the API’s quickly, until, details are flushed out; this will help meet the DOR (Definition of Ready) requirements for the interface definition before significant coding is done to implement the interface, and integration issues surface.

How to measure architecture?

Three things to look out for coupling, cohesion, and complexity. These days, people also put naming into this bucket.

If developers measure coupling and cohesion of classes, architects measure coupling and cohesion of services (*) and teams. The goal is to minimise coupling and maximise cohesion. High coupling between services (e.g. use of common database, use of common cache data, direct dependency) is indicative of low modularity. High coupling between teams developing services (and continuously in meetings to discuss interfaces) is indicative of potential integration issues. In some cases, micro services may be a good answer, and in others it may be a very bad idea! High cohesion for a single responsibility (including deployment, startup and shutdown lifecycle) is a great thing! Coupling cannot be completely avoided, dependencies is a reality. For sanity of the system, its necessary to track the coupling and cohesion, if not numerically, it can also be done visually. If there are couplings from platform services to application services, it’s a RED Flag. If the build system, can generate an inventory of services and the service graph, this graph can be used to measure the coupling and cohesion.

* A service could be a RESTful service, or a UI Component in the UI container, or products in a solution.

Complexity is a different beast. This one is hard to measure. It can be felt alright. If there is complexity, a good indicator is teams trying to push it out from their scope to someone else. Example:

Platform Architect: Failure re-try is an application concern.

Application Architect: Best effort and guaranteed delivery is a QOS that platform service should provide. I want to fire & forget.

Platform Architect: OK. This will introduce complexities in my service that I had not planned for. In that case, I can only provide you an asynchronous interface. Will notify you.

Application Architect: Huh. Can you design the interface to have both synchronous and asynchronous? The asynchronous model will create complexities in my workflow.

In such areas, I personally think, consistency is important. We should not have one platform service make re-try an application concern, and the other takes on the burden.

This is a place where the system architect must step in to define guidelines. Each case is different. And. this must be dealt with case-by-case.

If you can feel complexity, it can be visualised & addressed (if not numerically measured). This can be done by enumerating RISKS.

---

Testing and counting is a time-tested best practice to validate. This is architectural governance.

The things (rumblings) that I have been hearing, and is getting amplified every passing day:

1. Google does not have people titled architects. Sr. Engineers play the role under the covers. Prevents an unhealthy conflict between people designated “engineers” and “architects”
2. The role / title “architect” is very limiting and restrictive. Generally, it’s perceived as a consultant, or someone doing very high level design. It’s not seen as somebody with planning or problem solving skills.
3. Agile practitioners promote a team culture, where someone in the team is the “architecture owner”, but there is nobody titled “architect”.
4. Most of the architecturally relevant concerns are getting automated. E.g. API Gateway addresses the concern of API management, Architectural patterns are well known and documented, Cloud simplifies infrastructure concerns and delivers it as a service. A technically competent leader or engineer can make the choices, and hence we don’t need the architect.
5. There is sufficient training (online MOOC) to train engineers to pick up new skills (as required, on-the-fly); the need for architects (seniors) coming in to help engineers (juniors) is going away. There are many role models, who want to be senior engineers.
6. It’s prudent to have the team design and architect for a customers problem, than be fed the design and architecture to implement. The latter will de-motivate the team, and the former could improve engagement & innovation from all team members.
7. The new era is going to be very cost restrictive. Small teams have to build large stuff. We cannot afford an architect, and since generally, architects are not hands-on, having an architect on the team is overhead. If we really need architecture guidance beyond, we may reach out to an architecture consultancy to seek inputs.
8. Look @ UX. It’s all about UX Design. There is nobody for UX Architecture. Maybe UX Research. This will eventually happen to software as well.
9. It’s very difficult to define the role of an architect. It’s more difficult to separate it from an engineer. Can the roles be combined? Can the manager also play the role of the architect?
10. In the industry, most enterprise architecture initiatives have failed. Architecture initiatives were started to address the increasing complexity of the portfolio (IT systems, products), and alignment with business goals. It seems that standardization and rationalization are not good things to pursue, in this rapidly changing technology world. It’s going to get more messy. Architects will fail.

OK. 10 statements are enough for this blog post. There is truth in all the statements above.

This argument is similar to the one raging about DevOps. Thought leaders are saying “DevOps is a culture thing. Don’t create DevOps teams. Don’t create DevOps engineer. Don’t create DevOps career path. Create a team of developers and operations experts to build a DevOps culture in both species”

Lots can happen in next 5-10 years. 2025!! Whether there would be people titled architects or not, “architecture” and it’s “patterns and practices” are here to stay. System decomposition, making choices, planning and sequencing development, and solving problems would continue to remain. Tools and technologies will improve, making software development simpler and more professional.

In 2025, we may see a mix. Titled or not. All titles may go away in the spirit of teaming!

Architects. Engineers. Experts. Consultants.

Humans will continue to classify!

Unable to make a decision quickly? If you are analyzing options, assessing current state, making trade-offs, communicating to stakeholders, incorporating feedback, achieving consensus, and/or validating ideas; its clear that your work type is of a software architect. Titled or not.

Traditionally, in waterfall projects, architecture was an upfront exercise, and a time consuming one. The architect had to think through requirements, system views, x-cutting perspectives, and generate architectural descriptions for review and/or communication purposes. In today’s world, architecture is still documented, not formally (unless regulatory or corporate mandate); but remains minimal. As the system evolves, and new concerns emerge, architecture evolves & adapts. Not continuously, but continually.

So, it’s necessary to make decisions quickly.

1. Which cloud? Eh. Google. AWS. Azure. Eh. Eh. AWS!
2. Which NoSQL Database? Eh. MongoDB, Couchbase, DynamoDB. Eh. Eh. MongoDB
3. Which Messaging Service? Eh. Kafka, AWS SQS, RabbitMQ. Eh. Eh. SQS!
4. Where is interface definition for this component? Eh. Will get you version 0.01 by EOD.

Yay! Go build.

Oops we don’t have MongoDB Skills! Eh. Eh. DynamoDB. Go build!

Oops we are paying too much for DynamoDB in the cloud for development. Eh. Eh. Use DynamoDB on developer machine. No testing in cloud, and cloud resources should be consumed only when absolutely necessary. Be a miser. Spend more than $10, and it will come from your salary!

This is the norm in the industry today. Speed in everything. As the saying goes: “A good product, is a great product, if fast enough”

Some architects “feel” that such an approach will have an impact to software quality. Search for quality on google, and you get this … Quality software is reasonably bug or defect free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. ISO 8402-1986 standard defines quality as “the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.”

Given this definition of quality, it just seems that its more of pursuit to quality (just like pursuit to happiness).

1. Eh. Defect free?
2. Eh. Delivered on time?
3. Eh. Delivered within budget?
4. Eh. Meets explicit requirements?
5. Eh. Meets implicit requirements?

Pretty damn tough! How to correlate speed of architecture decision to this definition?

If architecture decisions can be made with speed, and pivoted (if found to be problematic) with speed, then architectural impact to software quality could be minimized?

• Yes. Definitely.
• No. If there are too many pivots (statistically > 3). Spinning will loose time.
• Maybe. If architecture changes are not prioritized alongside features.

For a modern architect,

1. All analysis has to be done yesterday, and choices have to be made for projects ASAP AND/OR
2. The team culture has to adopt architectural changes to meet new needs or architecture changes due to bad but quick decision.

Improved delivery platforms/ecosystems, and improved libraries in the industry, will help to build above and around, with some quality attributes taken care by design. The remaining ones can be time-tested through quick failures (pivots).

By Bayes Theorem,

• B = Software Quality is BAD.
• E = Architecture decisions are made quickly.
• p (B/E) = p(B) * p (E/B) / p (E)

Go find evidence and compute!

Architecture reviews help identify issues and get insights, only prioritized executed actions improve architecture. Issues to actions is easier said than done. The impediments to actions are several (mostly cultural).

I like architecture reviews. Call them audits, evaluations, or discussions. If done well (*) it does give good insights. No architecture is perfect. There is always something to find – a security issue, an extensibility issue, a technology choice issue, and/or an interoperability issue.

* Most of the times reviews are smelly. Typical smells – too many reviewers, done after most decisions are made and the goal is to defend, too broad and not targeted for any specific quality attribute, calendar (process) driven, and non-accountable reviewers.

The output of an architecture review is a formal report with problems and actions owners. Sometimes, this report is too big to read and is very negative (full of criticisms). The output of such reviews leads to:

1. De-motivated team OR
2. Architect blamed for not being able to defend, loss of credibility both ways OR
3. First impressions amplified to senior leadership (call-2-action), leading to micro-management of the team/architect.

This is the reason, that consultants are generally disliked in the industry. Even when the advice they provide is genuine, useful and helpful.

The two (less than three) things that have worked for me, to provide inputs are:

1. Don’t provide too many inputs at a time. LESS THAN THREE IMPROVEMENT AREAS is statistically good. Provide this input with MORE THAN THREE AREAS OF STRENGTH. Evolve the architecture incrementally.
2. Use offline collaboration platforms to provide inputs. Early peek & influence into the thought process, before it gets cemented.

Being non-judgmental, avoiding early escalation and engaging with a clear goal to be accountable is critical. A litmus test of success, is when the team/architect volunteer for a review, and are not driven to review by process.

A downside of this approach is the chance of loosing credibility with the management team. An audit is supposed to find all issues, so that we are perfect. However, my experience with software projects tells me that it’s better to lose credibility with the management team, than to fail in the project. Failure in software projects is not just architectural, but also due to:

1. Lack of a good problem statement.
2. Sponsorship for the project in the organization
3. Project Management
4. Team Skills
5. …and many more.

Therefore, as an architect, its necessary to:

1. Manage negative attitude towards architecture & architects.
2. Improve communication with development teams, involving and engaging them.
3. Improve communication with management teams, listening and engaging them.
4. Selling architecture – value selling or solution selling. Adopting the right approach.

Small. Focused. Engaged. Incremental. Action Oriented….is the way to improve architecture.

Nobody reads multi-page critical review reports. Those who do, pray that their intent is genuine & not political.

People like things visually appealing. People use things that fit the purpose. If both come together, and the timing is right, and with some luck added, the innovation takes off.

When something is successful, the architect is usually praised for the appeal, and less for “form follows function or simplicity of internal structure (a.k.a. engineering)”.

In software, the visual appeal is the user interface design and interactions. This is a specialized field of User Experience (UX) designers. Practitioners of this art form, are the front-end architects. The skill can be learnt by any user interface (UI) architect or engineer, however, the combination skill (UI + UX) is much rare. There are specialized UX companies, that offer prototype UI  (in a given technology – HTML5 etc.)  development with the help of developers. Such prototypes are great for formative testing with customers, however, architects and engineers of UI software have to structure and implement (construct) the software to meet other quality attributes (MVC Pattern, Connecting Data Sources, Re-usability, Modular Designs, Performance, ….).

The role of an architect in software (unlike civil or construction fields), is less about the visual appeal, and more about the internal structures of software. Even in the civil or construction fields, such roles are specialized (interior designers, industrial designers, exterior designers, etc).

A master (software) architect partners with UX designers, to make UX quality attributes a top priority for the project.

Selecting (making the choices) of the patterns and technologies to be used, in agreement with the projects scope (cost, requirements, resources), is where the software architect spends most of his/her time. Specialized art forms (like UX) are executed in partnership with UX specialists.

That said, I don’t intend to say that there is no “art” to architecture. Indeed there is. However, the other side of the coin (selecting, organizing, executing, partnering, modeling, validating, etc) is more significant.

Software Architecture is a Software Engineering Discipline. It’s a creative field in an engineering sense. Given that engineering is itself an artistic (sometimes deliberate) field, with greater emphasis on application of knowledge and professionalism for trade.

It’s taught & preached. Make wise choices. Left-right look, Prediction, analytics, …

Architect’s have an interesting job of making choices. They could use intuitive thinking, logical thinking, critical thinking or design thinking.

While making choices, it’s common to choose a trending technology or a trending pattern. Developers want it, managers can hire people who want to build it, and you could become an expert at the use of this shiny new thing, and would be a good thing to put on your resume. Everyone wants new. Until the promise of value fails. It’s then a ‘Pivot’.

Pugh matrix’s can be gamed by changing the weights. There is influence of the environment, corporate, community & industry on the choices; and this process is influenced by feelings/sentiments and not only data. Like Wall street or Nifty! A successful architect is deemed to be credible, and the choices made by him/her, or a recommendation coming from him/her, could become significant. A successful business relationship between your company and a partner, could become significant. Biased by design.

Also, given the explosion of technology choices and patterns, its difficult to keep track of the changes, direction, and value proposition of all such stuff. This is just the side effect of innovation and industrialization. Multiple choices results in commoditization, and drives down the costs.

Example: Smalltalk vs. Java. You have to know both to compare, and as a result you could land up in the Smalltalk camp or Java. Smalltalk is very powerful, and early adopter of ideas, and adopted in the academic circles. However, Java has been more prominent in commercial circles. So, how do you make a choice?

One way is to ignore, and become lazy, and procrastinate. Since, anything in this world can be justified, you could say that this choice is not architecturally relevant, and adopt the polyglot trend. Alternatively, you can criticize (or critically appreciate) the adoption of polyglot, collect data to show the perils of doing so, and drive a corporate initiative to standardize (naming, blaming, shaming to drive compliance to standardization).

Sometimes, if your choice is not an industry consensus or trend, the environment (managers, consultants, developers) will be pushing back on you to adopt (Mr. Architect, you should take some risks! *)

*at your own risk

Since, architecture is about making choices and defining a structure, two architects could come up with two different (but similar) architectures. Each one can claim that their choices are data driven, time tested, and the best for the business.

So, how to make this work?

The only thing that I have seen “work” is to have “one” decision maker (not necessarily by hierarchy) for a choice. The decision maker is free to change decision downstream without being killed (named, shamed, ousted, …). The decision maker should not be continually influenced (or threatened), but informed. The decision maker could be the person doing the work to make a choice, or collecting inputs from various experts to make a choice. People who disagree with the decision, should have an open environment to disagree, provide inputs, challenge the status-quo. This holds true for those up in the hierarchy, or down in the hierarchy, or sideways in the hierarchy. In all cases, once a decision is made, the team/organization should support & pursue. This is culture.

In general, people are good. They like to make the business successful. Hence, they like to make decisions using intuitive, logical, analytical, or critical thinking. Culture, will get all the pieces to fit gradually.

Therefore, there should be only one architect for a choice. Make a decision, move on. If it’s wrong, talk about it and fix it. If it’s right, talk about it, adopt as best practice.

“Architect must code. Manager must code. Scrum Master must code. Tester must code. Product owner must code. Developer can become architecture owner. Developer should become tester. Developer must understand customer challenges”

The way the debate goes, it almost seems like everyone is trying to do someone else’s job. And, this behavior gets justified as self-organizing teams.

In a real world, people have mustered skills and aspire to learn new skills. This is true for an architect as well. It’s very rare, that we are able to assemble the perfect team. Mature @ t=0.

It’s important to pull your weight as part of any effort/job. If coding is your strength, do it. If customer engagement is your strength, do it. If inspiring people is your strength, do it. If making choices (trade-offs) is your strength, do it. If you have multiple skills & you are ambidextrous, do it (lucky!). Just don’t burn yourself out, doing it all.

If the challenges for the team are not architectural (i.e. most architectural choices are made and time tested, or, team is mature and governance is not required), then the architect needs to pull his/her weight to address the challenges. This may mean coding, testing, and/or selling. If this type of work is not interesting (or boring or offending), it’s time to change than be a burden.

If the challenges for the team are realizing the architecture, and they look up to the architect for help, the architect MUST code to show how to get things done. It’s OK to ask for HELP, if you believe the architecture is right, but you don’t have the coding skills to get it done. It’s part of learning journey.

Winning is important. Need to give yourself permission to do the right thing. Sometimes, the right thing is to go up the Ivory Tower and preach. Sometimes, the right thing to do is go down the trenches and teach.

It’s a trade-off.

In this new brave world, anything can be said, done, and justified.

This sentence is complex. English! My god. Such a complex language.

• Continual = repeatedly (say the tv show is running continually)
• Continuously = non-stop (say the tv is running continuously)
• Emerge = come-out (say a baby bird emerges from egg)
• Evolve = grow-into-a-new-state (say a caterpillar evolves to a butterfly)

So,

• Continual emergence = A new architecture showing up repeatedly (every sprint or PI) for life.
• Continual evolution = A new architecture after emergence, evolves repeatedly to adapt to repeated change in requirements. The architecture continual evolution stops, when requirements continual evolution stops; i.e. Product is mature & moving to EOL (end of life)
• Continuous emergence = A new architecture showing up every day (or second) for life.
• Continuous evolution = A new architecture after emergence, evolves non-stop to adapt to non-stop change in requirements.

Practically speaking,

•  In the early stages of the project, the change in requirements that impacts architecture can be  continuous. Architecture also emerges and then evolves continously.
• In the later stages of the project, once requirement volatility has been addressed, the  architecture remains more-or-less constant (gets cemented) and changes are infrequent. Architecture is evolved continually to address new requirements that would have emerged requiring architectural change. This architecture change could be a costly (impact regression, $) project.
• The software does reach a state, where change in architecture, is hard to address certain types of requirements. At this time, a phoenix (re-birth) or hydra (supplement) type strategy is required.
• The software finally does reach a state, where disruptions and innovations in the industry, would have disrupted the value proposition, or architecture. This is the time to EOL the product.

In Summary, though anything can be done i.e. continual or continuous emergence or evolution of architecture throughout the product life cycle, it’s not the norm. Emergence happens early, evolves continuously till proven, has a few (less than 3) major continual evolutions during the life of the product.

Having the power to do something, does not mean it’s a responsible thing to do!