Myth – Digital Journey

Myth: Architects only create Diagrams

Like it or not. Diagrams (visual) are a great communication tool. If one of the responsibility of architects is communication, there is no better way than visual communication. Contextual communication requires that the same information to be represented differently for effective communication.

Architects (titled or not) have a responsibility to analyze data from various sources:

a) Requirements coming from customer or product manager.
b) Complaints coming from customer or product manager.
c) Constraints coming from customer or product manager.
d) Constraints and opportunities from operational leaders.
e) Technology advances in industry.
f) Patterns and practices in architecture.
g) Feedback from development teams.
h) Feedback from independent consultants (peers, stakeholders).
i) Inputs from security teams.
j) Inputs from operational teams.

All this data needs to be analyzed to produce conceptual and detailed sketches (as required) for construction. Today, most of these sketches are conceptual. Teams are very skilled to develop the detailed sketches right in code.

The architect is like a data scientist working on all this data to determine the function that ‘fits’. This function is represented as a diagram – a view, or a perspective. The diagram could even be a simple table.

Stating that ‘architects only create diagrams’, however, is a poor critique of the effort. Creating conceptual clarity is important for the architect. But the architect’s job does not end at diagrams. They have to communicate, plan, and code. Unless, the diagram is realized, it’s useless.

Just like code and configuration should be treated like ‘code’; documentation and diagrams also need to be treated like ‘code’. This means – reviewed, maintained, tested, critiqued, destroyed, re-factored, …

Only treating ‘code’ like ‘code’ is bad ‘coding’. Code, configuration and concepts need to be treated like ‘code’.

Dismissing concepts (usually the work of an architect) is immature.

The best representation of ‘architecture’ is ‘code’. The first draft’s of ‘code’ are ‘concepts’. ‘Concepts’ are represented as ‘diagrams’ for communication. AND. communication is a good thing.

Enough said.

Myth: Form Follows Function

This argument gets philosophical.

In classic building architecture, the form of a stadium is meant to perform the function to host large audiences. The form of a house is meant to perform the function to host families. A stadium form is not used to home a family! So, it follows, form follows function. This argument then leads to architects and designers to start from function to create forms.

a) Can an architect create form of a stadium to home a family of four? Why not?
b) What about Bill Gates house? Is that form follows function, or form follows money spent?

Let’s take another example. How about a car? The function of a car is to take a few passengers from point A to point B safely.

However, cars are also used for

a1. Kidnapping people
a2. Moving goods (not passengers)
a3. Racing

Seems like new functions were discovered once the form (‘car’) appeared. The form was adopted for other ‘functions’; not originally intended for…

Let’s take another example. How about email? The function of email is to communicate a message from a source person to several destinations persons.

However, emails are also used for

a) Advertisement
b) Spreading malware
c) Sharing photos
d) Sharing legal documents

Seems like new functions were discovered once the form (’email’) appeared. The form was adopted for other ‘functions’; not originally intended for…there may be specific platforms for ‘sharing photos’ e.g. Instagram; these specific forms could also be used for ‘selling drugs’ and ‘pornography’; not the original function.

Let’s take another example. How about Blockchain? The function of blockchain is to maintain a distributed ledger of transactions.

However, blockchain can also be used for

a. Creating and maintaining a patient’s historical record
b. Internet of Things
c. Voting

Seems like new functions were discovered once the form (‘blockchain’) appeared. The form was adopted for other ‘functions’; not originally intended for…there may be specific platforms for maintaining patient’s record or maintaining registry of connected things.

These examples above, indicate that “Unintended Functions Follows Form that Follows Original Function”. Kind of a recursive loop!

Having a functionalist (modernist) mindset in design & architecture is good; however, it creates blind-spots in designers that makes them choose from past (known) stereotype solutions and patterns. This results in re-use of known patterns; good for manufacturing & speed, bad for creativity.

OK! Just trying to say that there it’s more complicated than form following function. These days, it’s really form following profit and function following form. Flexibility and open mind is required to keep up in this changing world.

Myth: Architecture field does not have specializations.

A Quick search for software architect titles would dispel this myth.

Application Architect
Java Architect
Cloud Architect
Database Architect
Data Architect
Analytics Architect
Security Architect
SOA Architect
Solution Architect
Product Architect

Ten (#10) is a good number to make my point. Now let’s look at super-specialization.

Cloud Architect
        AWS Architect
        Azure Architect
        Cloud Foundry Architect
        Google Cloud Architect
Database Architect
        MS SQL Architect
        Postgres Architect
        Azure SQL Architect
        Cassandra Architect

Generalizing this, it could be reasoned that, the ask in the marketplace is for:

b1. Big “A” Architect, known for his/her knowledge of patterns and practices of general architecture.
b2. Specialized Architect, known for his/her knowledge of patterns and practices in a specialized field (e.g. cloud, security, database)
b3. Super Specialized Architect, known for his/her knowledge of patterns and practices in a super specialized field (e.g. Azure, AWS, Postgres)

Hmmm…now now, if you have different super specialties that are required to build a product, would you

c1. Build a team of super specialized architects?
c2. Hire an architect, and have collaboration with super specialized engineers, consult with super specialized architects on a need basis?

My personal preference is #2.

Let’s take a healthcare analogy. In healthcare, you can find generalists, specialists and super specialists – general physicians, cardiologists, pediatric cardiologist, neurologists, dermatologists, endocrinologists, etc.

If you have a headache, where do you go? General Physician or neurologist?

d1. The chance that your condition is classified as a common condition is high, if you visit the general physician.
d2. The chance that your condition will be over-tested is high, if you visit a specialist.

It’s generalist and specialist bias.

Same is true with software architecture. Go to a software architect for an authentication problem, the solution proposal will look simple e.g. Use LDAP. If you go to a security architect for an authentication problem, the solution proposal will be comprehensive e.g. Use LDAP, Enable SAML & OAuth2 for single sign on, Test for OWASP Top 10 Web Application Security Risk, Develop threat model, …

The specialist is also costlier than the generalist; and the super specialist is costlier than the specialist.

You can always take second opinions with a specialist. However, its best that the system encourages generalists to front-end specialists. A tiered approach is better than walking directly to a specialist.

Just like a family physician (PCP – Primary Care Physician) is the first line of consult, a general architect (Big “A” Architect) is a good start to lead the architecture of a software product/solution.

But – hey – specializations exist in software architecture.

Myth: An architecture can be sold by it’s quality attributes & trade-offs.

An architect has the responsibility to sell her architecture. This could happen in reviews with peers, reviews with stakeholders, reviews with developers, or reviews with operational managers.

Certain things can be learnt by the architect from the patterns and practices of the sales discipline. There are two key practices in sales.

Value based selling. In this art form, the sales person describes and re-inforces the value of the product that she is selling; some key quality attribute e.g. brand, reliability, best-seller, discounted only for the month, etc.
Solution based selling. In this art form, the sales person understands the customers pain points; and creates a basket of products/services that address the pain points. The outcome is a personal solution to the pain points.

I have seen that good architects, can be perceived as great; if they can adapt to the various stakeholders – peers, developers, users, managers, etc. i.e. adopt solution selling.

The general practice in architecture is to collect requirements from different stakeholders, do some industry research, apply some brain power and experience, and come out with an architecture that could meet the needs of various stakeholders. The architect is proud of the architecture that she has developed, and would like the stakeholders to accept the proposal; and seeks to sell the value proposition of the architecture produced…i.e. the guarantees of best-in-class and latest technologies, the quality of data stored, etc.

However, for most stakeholders, it’s really about

“What’s in it for me?” and
“How is my need met?”

For a peer architect, usually coming in from a very specific perspective, it’s about making sure a certain quality attribute is done well. E.g. dependent API’s.

For a developer, it’s usually about the technology used to build and ease+comfort of developing the product.

For a user, it’s usually about meeting the primary functional and non-functional goals.

For an operational manager, it’s usually about having an architecture defined that is generally accepted; and knowing the potential structure she needs to setup to execute the plan; and knowing what risks need to be tracked.

So, when an architecture is described using – logical view, deployment view, security perspective, etc; the stakeholders of the architecture (users, developers, managers, peers) are not specifically addressed.

In my experience, an architecture view is very personal to a stakeholder, and must be created to meet the needs of the stakeholder (i.e. solution selling). It’s important to understand the pain point of the stakeholder, and sell the architecture in context of the pain points.

Bottomline, an architecture description should be minimal enough to sell the architecture to various stakeholders. A comprehensive data view is great for the architect to get the BIG picture, but not a great selling proposition for users, developers, managers, or peers.

Specific and customised selling helps to improve and iterate on architecture. A good architect, is a great architect, when she can sell the architecture successfully. Capitalism!

Myth: Architect belongs to the architecture team

Yes, and no. Necessary, but not sufficient.

Architecture Office (AO) in particular is very strict & diligent about their patterns and practices. They are empowered & responsible for governance (in a good way), and rationalize (harmonize) efforts in the organization. The architect feels a belonging to this team to derive rules & priorities; and contribute/influence the big picture.

Agile (scrum) teams in particular have high cohesion. They disown anybody that don’t work with them in their pods. In spirit, of an empowered team, they want the team to make decisions, and hence, the architect must belong to them. This is the best way to influence work that happens on the ground.

X-Functional Leadership teams are typically composed of program, product, operational, business, architectural, manufacturing and engineering points of contact. Teaming with such a x-functional group is very critical to the success of a x-functional initiative.

Each team is a mission focused team. Each team is empowered. Each team is connected to other teams (team network grid) for information flow & communication.

The architect (and other titled roles) find themselves to be part of several teams. I have had architects tell me that they see the “scrum role” as solid line, and “AO” or “X-Functional LT” as dotted line. Some others tell me that “AO” is solid line, and “scrum role” or “X-Functional LT” is dotted line. i.e. in essence they are prioritizing a team to either spend time or to trade-off a quality attribute. It’s not just navigating a matrix, but a network of matrices. Trading off – Authority & Influence. This can become complex.

If Architecture is the art of managing complexity, and complexity is due to structure, then the ‘architect’ has to play in space between spaces. (S)he has to be in different teams. Given the nature of the complexity, sometimes, (s)he has to play devil’s advocate with the scrum team, and sometimes with the “AO”. (S)he has to own the architectural decision, that (s)he is responsible for, through transparent communication. (S)he has to make all such teams believe/perceive that (s)he is part of that team.

Being part of multiple teams is a reality. Different leaders pulling in different & sometimes conflicting directions is also a reality. Being part of multiple teams is an opportunity to influence & carry information. The last thing that should happen is to lose your mind trying to do everything. It’s best to plugged into each team, make them believe/perceive that you are working for them, however, focus on one thing at a time. Multiplexing is like an adult choice.

Myth: Architect-led design and building has higher chances of success.

From Wikipedia, “Etymologically, architect derives from the Latin architectus, which derives from the Greek (arkhi-, chief + tekton, builder), i.e., chief builder.^“

Architect-led designing and building is a deduction that can be made from the above statement. There is still a reasoning error in this statement.

Architecture is about managing complexity. It’s patterns and practices that help manage complexity.

Applied to software deployment, deployment patterns and practices represent the body of knowledge, and selected patterns and practices represent the software deployment architecture. E.g. CI/CD practice, Blue-Green Deployment pattern, Container pattern, …
Applied to software development, development patterns and practices represent the body of knowledge, and selected patterns and practices represent the software development architecture. E.g. Agile practice, XP Practice, Microservices pattern, …
Applied to business, business architecture patterns and practices represent the body of knowledge, and selected patterns and practices represent the business architecture. E.g. TOGAF Process, BPM Process patterns, …

Architect is a practitioner (bachelor, master, Phd) in architecture applied to a specific subject area. E.g. Enterprise Architect, Security Architect, …

It’s necessary to have leadership qualities to be a good architect.

However, Architect-led design and building success is still an error in reasoning. Architecture-led design and building success makes more sense. In some way, the prior is like Conway’s law, and the latter is Inverse Conway Maneuver.

One way to achieve architecture-led design and building is to separate the architect from the design and building organization to address the biases & conflicts introduced by the organization structure (Conway’s law). This design has it’s own perils; however, can be successful, if architectural practices are valued & prioritized at the business & portfolio level. Easier said than done.

Development-led architecture, design and build is an alternative approach. In this approach, architecture emerges early, and evolves to a minimal and good enough state to meet the user’s needs. This assumes that developers have been trained on architectural practices. This approach can achieve speed.

These two approaches can be combined i.e. “Development-led architecture, design, and build” and “Architecture-led design and build” to tune the system for both speed and consistency. It takes a mature organization (people) to adopt this approach.

Bottom-line, having an architect or architecture is necessary, but not sufficient for success.

Myth: Architecture is a deductive process; Architects are deductive thinkers

A deductive process is top-down. It’s data driven. It’s logical. It’s rational. It’s normative. Most architects connect their priorities to a business vision or a customer use-case. They would collect data about use-cases and current state of problems to deduce a new target state. An execution plan is built to reach this target state.

Alternatively, an inductive process is bottom-up. It’s heuristic. It’s collaborative. An inductive process is one where patterns emerge and induce architecture decisions. The assumption in such a process is that use-cases are known but not understood, and architecture decisions should not be finalized until “rubber hits the road”. Action-biased & fail-fast, & change acceptance culture.

Example: Deductive

We analyzed all potential use-cases. We need to create Service-X, Service-Y and Service-Z; as deduced from the use-cases. These services shall be compliant to the SOA architecture style. The implementation shall use this as guidance and architecture will govern non-compliance to this guidance.

Example: Inductive.

We will start with use-case-1; We will implement a thin slice of use-case-1, and this process creates Service-A. We continue this process for use-case-2. To meet use-case-1 and use-case-2, we need to modify Service-A to Service-A1 and create new Service-B. This process leads to (induces) services A1 & B. Continuing this approach, when we are done, we will have services A5, B2, C3 and D1.

Some agile practices, also use the term “intentional architecture” for deductive and “emergent design” for inductive; and recommend a little bit of both to evolve architecture.

Architecture is a process of planning. Planning is deductive by design. Therefore, architecture is a deductive process; and architects architecting architecture are deductive thinkers. This statement is an error in deductive reasoning.

A modern architect needs to embrace both deductive and inductive thinking in architecture. The architect has to create a reference target & plan, but also modify from feedback. The architect needs to understand the perils of each type of thinking and adapt

Deductive could be time consuming (analysis-paralysis)
Inductive could create a mess (architectural debt)

Both methods could lead to waste. Balance is critical.

Personally, my natural bias is deductive. My adapted style is inductive. I do get stressed out when I am pure inductive. I have not yet found my balance; but have realized that balance is critical.

Myth: Architecture cannot be tested or measured

The typical architecture representations are diagrams, abstracts, views, and perspectives that are documented formally or informally.

However, a modern architect does not stop there. These representations start at version 0.1 and gets iterated through testing, measuring and validation.

Example: Interface Definitions (a closely observed architectural artifact) goes through several versions.

How to test architecture?

Need to inspect the architecture, to determine, what needs to be tested.

Test the concept/assumptions: Search-ability is a quality attribute for an image archive. Do a spike to test the assumptions about what/how. E.g. Spike: Storing meta-data of an image as tags on the file image is sufficient and efficient for the query use-cases; Alternative is to store the tags as a JSON document in NoSQL DB with indexes for efficient search; Alternative is to store tags as a JSON document in ElasticSearch or Apache Solr. This will force sampling the search use-cases and validate that the approach selected will indeed meet the use-cases with desired performance & cost attributes.
Test the interface definition: Interface Granularity & Availability is a quality attribute. Create a mock API and push for consumers to consume the API, to test implicit or unsaid requirements. Iterate the API’s quickly, until, details are flushed out; this will help meet the DOR (Definition of Ready) requirements for the interface definition before significant coding is done to implement the interface, and integration issues surface.

How to measure architecture?

Three things to look out for coupling, cohesion, and complexity. These days, people also put naming into this bucket.

If developers measure coupling and cohesion of classes, architects measure coupling and cohesion of services (*) and teams. The goal is to minimise coupling and maximise cohesion. High coupling between services (e.g. use of common database, use of common cache data, direct dependency) is indicative of low modularity. High coupling between teams developing services (and continuously in meetings to discuss interfaces) is indicative of potential integration issues. In some cases, micro services may be a good answer, and in others it may be a very bad idea! High cohesion for a single responsibility (including deployment, startup and shutdown lifecycle) is a great thing! Coupling cannot be completely avoided, dependencies is a reality. For sanity of the system, its necessary to track the coupling and cohesion, if not numerically, it can also be done visually. If there are couplings from platform services to application services, it’s a RED Flag. If the build system, can generate an inventory of services and the service graph, this graph can be used to measure the coupling and cohesion.

* A service could be a RESTful service, or a UI Component in the UI container, or products in a solution.

Complexity is a different beast. This one is hard to measure. It can be felt alright. If there is complexity, a good indicator is teams trying to push it out from their scope to someone else. Example:

Platform Architect: Failure re-try is an application concern.

Application Architect: Best effort and guaranteed delivery is a QOS that platform service should provide. I want to fire & forget.

Platform Architect: OK. This will introduce complexities in my service that I had not planned for. In that case, I can only provide you an asynchronous interface. Will notify you.

Application Architect: Huh. Can you design the interface to have both synchronous and asynchronous? The asynchronous model will create complexities in my workflow.

In such areas, I personally think, consistency is important. We should not have one platform service make re-try an application concern, and the other takes on the burden.

This is a place where the system architect must step in to define guidelines. Each case is different. And. this must be dealt with case-by-case.

If you can feel complexity, it can be visualised & addressed (if not numerically measured). This can be done by enumerating RISKS.

Testing and counting is a time-tested best practice to validate. This is architectural governance.

Myth: The “architect” role will cease to exist by 2025

The things (rumblings) that I have been hearing, and is getting amplified every passing day:

Google does not have people titled architects. Sr. Engineers play the role under the covers. Prevents an unhealthy conflict between people designated “engineers” and “architects”
The role / title “architect” is very limiting and restrictive. Generally, it’s perceived as a consultant, or someone doing very high level design. It’s not seen as somebody with planning or problem solving skills.
Agile practitioners promote a team culture, where someone in the team is the “architecture owner”, but there is nobody titled “architect”.
Most of the architecturally relevant concerns are getting automated. E.g. API Gateway addresses the concern of API management, Architectural patterns are well known and documented, Cloud simplifies infrastructure concerns and delivers it as a service. A technically competent leader or engineer can make the choices, and hence we don’t need the architect.
There is sufficient training (online MOOC) to train engineers to pick up new skills (as required, on-the-fly); the need for architects (seniors) coming in to help engineers (juniors) is going away. There are many role models, who want to be senior engineers.
It’s prudent to have the team design and architect for a customers problem, than be fed the design and architecture to implement. The latter will de-motivate the team, and the former could improve engagement & innovation from all team members.
The new era is going to be very cost restrictive. Small teams have to build large stuff. We cannot afford an architect, and since generally, architects are not hands-on, having an architect on the team is overhead. If we really need architecture guidance beyond, we may reach out to an architecture consultancy to seek inputs.
Look @ UX. It’s all about UX Design. There is nobody for UX Architecture. Maybe UX Research. This will eventually happen to software as well.
It’s very difficult to define the role of an architect. It’s more difficult to separate it from an engineer. Can the roles be combined? Can the manager also play the role of the architect?
In the industry, most enterprise architecture initiatives have failed. Architecture initiatives were started to address the increasing complexity of the portfolio (IT systems, products), and alignment with business goals. It seems that standardization and rationalization are not good things to pursue, in this rapidly changing technology world. It’s going to get more messy. Architects will fail.

OK. 10 statements are enough for this blog post. There is truth in all the statements above.

This argument is similar to the one raging about DevOps. Thought leaders are saying “DevOps is a culture thing. Don’t create DevOps teams. Don’t create DevOps engineer. Don’t create DevOps career path. Create a team of developers and operations experts to build a DevOps culture in both species”

Lots can happen in next 5-10 years. 2025!! Whether there would be people titled architects or not, “architecture” and it’s “patterns and practices” are here to stay. System decomposition, making choices, planning and sequencing development, and solving problems would continue to remain. Tools and technologies will improve, making software development simpler and more professional.

In 2025, we may see a mix. Titled or not. All titles may go away in the spirit of teaming!

Architects. Engineers. Experts. Consultants.

Humans will continue to classify!

Myth: Making Architecture Decisions quickly impacts software quality (negatively)

Unable to make a decision quickly? If you are analyzing options, assessing current state, making trade-offs, communicating to stakeholders, incorporating feedback, achieving consensus, and/or validating ideas; its clear that your work type is of a software architect. Titled or not.

Traditionally, in waterfall projects, architecture was an upfront exercise, and a time consuming one. The architect had to think through requirements, system views, x-cutting perspectives, and generate architectural descriptions for review and/or communication purposes. In today’s world, architecture is still documented, not formally (unless regulatory or corporate mandate); but remains minimal. As the system evolves, and new concerns emerge, architecture evolves & adapts. Not continuously, but continually.

So, it’s necessary to make decisions quickly.

Which cloud? Eh. Google. AWS. Azure. Eh. Eh. AWS!
Which NoSQL Database? Eh. MongoDB, Couchbase, DynamoDB. Eh. Eh. MongoDB
Which Messaging Service? Eh. Kafka, AWS SQS, RabbitMQ. Eh. Eh. SQS!
Where is interface definition for this component? Eh. Will get you version 0.01 by EOD.

Yay! Go build.

Oops we don’t have MongoDB Skills! Eh. Eh. DynamoDB. Go build!

Oops we are paying too much for DynamoDB in the cloud for development. Eh. Eh. Use DynamoDB on developer machine. No testing in cloud, and cloud resources should be consumed only when absolutely necessary. Be a miser. Spend more than $10, and it will come from your salary!

This is the norm in the industry today. Speed in everything. As the saying goes: “A good product, is a great product, if fast enough”

Some architects “feel” that such an approach will have an impact to software quality. Search for quality on google, and you get this … Quality software is reasonably bug or defect free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. ISO 8402-1986 standard defines quality as “the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs.”

Given this definition of quality, it just seems that its more of pursuit to quality (just like pursuit to happiness).

Eh. Defect free?
Eh. Delivered on time?
Eh. Delivered within budget?
Eh. Meets explicit requirements?
Eh. Meets implicit requirements?

Pretty damn tough! How to correlate speed of architecture decision to this definition?

If architecture decisions can be made with speed, and pivoted (if found to be problematic) with speed, then architectural impact to software quality could be minimized?

Yes. Definitely.
No. If there are too many pivots (statistically > 3). Spinning will loose time.
Maybe. If architecture changes are not prioritized alongside features.

For a modern architect,

All analysis has to be done yesterday, and choices have to be made for projects ASAP AND/OR
The team culture has to adopt architectural changes to meet new needs or architecture changes due to bad but quick decision.

Improved delivery platforms/ecosystems, and improved libraries in the industry, will help to build above and around, with some quality attributes taken care by design. The remaining ones can be time-tested through quick failures (pivots).

By Bayes Theorem,

B = Software Quality is BAD.
E = Architecture decisions are made quickly.
p (B/E) = p(B) * p (E/B) / p (E)

Go find evidence and compute!