Delegation

Ask, Don’t tell :::: Tell, Don’t Ask :::: Don’t Ask, Don’t Tell

For those of us in software, we know about the two programming paradigms that are used widely in the industry – Object-Oriented and Functional. The object-oriented paradigm is usually termed as the “Tell, Don’t Ask” model, where the data and behavior are kept together. The functions in the object-oriented paradigm change state of the object (i.e., cause side effects). The functional paradigm is usually termed the “Ask, Don’t Tell” model, where the functions don’t change state and cause no side effects. The software security implementations favor the “Don’t Ask, Don’t Tell” – or weakly the principle of least privileges.

These software models are just copied over from the management/leadership principles. It’s all about delegation.

  1. Tell, Don’t Ask: Delegates responsibility and authority.
  2. Ask, Don’t Tell: Does not delegate responsibility or authority.
  3. Don’t Ask, Don’t Tell: Intentional Non-transparency.

The management coaches will scream: “You should delegate responsibility, but not accountability.”

The agile coaches will correct the statement: “You should delegate responsibility and authority, but not accountability.”

This one everybody agrees – The accountability lies squarely on the first line leader a.k.a. the project manager. However, the project manager is encouraged not to micro-manage but to delegate responsibility and authority to the scrum teams.

Not all scrum teams are alike because not all people are alike and have different natures (due to the nature/nurture mix) that drive their behaviors. Some people follow commands, whereas others question the status quo. Some people focus on problems, and others focus on solutions. When a scrum team is formed, and people with different backgrounds are put together, delegating responsibility and authority can be challenging. Some scrum teams will accept responsibility but will defer the authority to the leadership. Other scrum teams will not accept responsibility without authority.

Example: The architect is accountable for defining the architecture, and the scrum team is responsible for implementing the architecture. In this model, responsibility is delegated, but the authority to make architectural choices are not. Depending on the culture (largely nurture) you come from, the scrum teams would be either happy or revolt.

Not all people are made equal or nurtured equally. Some people have lived prosperously and always made their choices. Others have lived in a command control structure and always had to make the commander’s choice their own. When people of different natures are put together in a scrum team, the team takes a while to form, norm, and perform. Delegating to such scrum teams takes patience.

Hence, agile coaches ask not to break the scrum teams. Don’t allocate people to projects; assign projects to scrum teams. This principle also means that we don’t treat people like resources (i.e., people resources allocated to projects, but we assign projects to people teams). So, treat people like people (just like you treat code like code and infrastructure as code).

Summary: The project manager must delegate responsibility and gradually delegate authority (depending upon the scrum team the project manager has to deal with). Delegating authority creates an egalitarian environment that people (even from working democracies) will need to adapt.

Activities vs. Outcomes

The operation is successful but the patient died – A day in life of a doctor

It’s classic. The outcome expected by the patient’s relatives from a heart surgery is that the patient’s condition improves. The surgical team performs many activities (wheeling the patient into the surgery room, anesthesia for the patient, monitoring vitals, and many more) pre-operative, intra-operative, and post-operative. Eventually, If the patient does not recover, the team will still claim that all the activities were successful.

Successful activities don’t necessarily lead to a successful outcome. Successful activities are necessary but not a sufficient condition.

In a software development context, to reach a goal, a team performs many activities. They may spend many sprints busily doing activities. The burn charts will show that work is being done and accepted, but alas, the outcome may not be in sight for many months. In Agile, we break down a business EPIC into Features, Features into Stories, and perform tasks to claim a story. The customer (or proxy) defines the outcome in the EPIC, and the acceptance of features, stories, and tasks are just activities. Successful completion of the features, stories and tasks does not guarantee a successful outcome.

The Feature burn charts look good, and teams like to showcase these charts to demonstrate progress. Progress is necessary but not sufficient for an outcome.

A few things can happen when the team completes all the features in an EPIC

  1. Successful Outcome: The team accepts all features of an EPIC, and the customer (or proxy) accepts the EPIC.
  2. Partial Successful Outcome: The team accepts all features of an EPIC, and the customer (or proxy) gives new inputs or flags issues to the team. The EPIC is not accepted, and issues are added to the EPIC, or the existing EPIC is accepted, and new EPICs are created to handle new inputs (scope increase).
  3. Failed Outcome: The team accepts all features of an EPIC, and the customer (or proxy) does not accept the EPIC, and the team has to significantly re-plan.

If the customer (or proxy) is engaged continuously, #3 is an anomaly, but it can happen. A Partial successful outcome (#2 above) would be the most likely outcome unless the EPIC were very well defined, tiny, and non-ambiguous. But we are agile to deal with ambiguities. To handle new features (scope creep), the teams create a new EPIC V2.0. To handle issues (bugs), the teams create issues in the EPIC and plan to close this debt (at least some of them) before new features are prioritized, and the EPIC can be (reasonably) accepted.

Going back to our heart surgery, the team may have planned the exact features in the surgery (e.g., Stent the artery), but during the surgery, they may find anomalies that need to be taken care of other than just stenting the artery. These anomalies take time (effort) to fix, and the surgery time may increase. The surgeon may also detect anomalies that might require a new surgery (new EPIC) to handle during the surgery. After the surgery, the patient may have BP stability issues (hypotension) and is on NOR (Norepinephrine) to maintain blood pressure and stabilized in the ICU before being discharged (discharge: A successful outcome).

Summary: Activities lead to outcomes. This activity completion is necessary but not sufficient. Successful activities may lead to failed outcomes. Organizing and Planning work (in EPICs/Features/Stories) is important for efficiency and predictability. However, in most practical cases, things go wrong. Sometimes, more work has to be done that causes the effort in a feature to shoot up, OR more work has to be done before a feature can be claimed to be done, OR new requests prop up after an EPIC is claimed.

There is a need to separate “organizing work” and “seeking outcomes” so that both efficiency metrics (lead metrics: say/do) and outcome metrics (lag metrics: KPIs) are tracked.

Data about Data

As a Data Engineer, I want to be able to understand the data vocabulary, so that I can communicate about the data more meaningfully and find tools to deal with the data for computingData Engineer

Let’s start with this: Binary Data, Non-binary Data, Structured Data, Unstructured Data, Semi-structured Data, Panel Data, Image Data, Text Data, Audio Data, Categorical Data, Discreet Data, Continuous Data, Ordinal Data, Numerical Data, Nominal Data, Interval Data, Sequence Data, Time-series Data, Data Transformation, Data Extraction, Data Load, High Volume Data, High Velocity Data, Streaming Data, Batch Data, Data Variety, Data Veracity, Data Value, Data Trends, Data Seasonality, Data Correlation, Data Noise, Data Indexes, Data Schema, BIG Data, JSON Data, Document Data, Relational Data, Graph Data, Spatial Data, Multi-dimensional Data, BLOCK Data, Clean Data, Dirty Data, Data Augmentation, Data Imputation, Data Model, Object (Blob) Data, Key-value Data, Data Mapping, Data Filtering, Data Aggregation, Data Lake, Data Mart, Data Warehouse, Database, Data Lakehouse, Data Quality, Data Catalog, Data Source, Data Sink, Data Masking, Data Privacy

Now let’s go here: High volume time-series unstructured image data, High velocity semi-structured data with trends and seasonality without correlation, High volume Image data with Pexels Data source masked and stored in Data Lake as the Data Sink.

The vocabulary is daunting for a beginner. These 10 categories (ways of bucketizing) would be a good place to start:

  1. Data Representation for Computing: How is Data Represented in a Computer?
    • Binary Data, Non-binary Data
  2. Data Structure & Semantics: How well is the data structured?
    • Structured Data, Unstructured Data, Semi-structured Data
    • Sequence Data, Time-series Data
    • Panel Data
    • Image Data, Text Data, Audio Data
  3. Data Measurement Scale: How can data be reasoned with and measured?
    • Categorial Data, Nominal Data, Ordinal Data
    • Discreet Data, Interval Data, Numerical Data, Continuous Data
  4. Data Processing: How is the data processed?
    • Streaming Data, Batch Data
    • Data Filtering, Data Mapping, Data Aggregation
    • Clean Data, Dirty Data
    • Data Transformation, Data Extraction, Data Load
    • Data Augmentation, Data Imputation
  5. Data Attributes: How can data be broadly characterized?
    • Velocity, Volume, Veracity, Value, Variety
  6. Data Patterns: What are the patterns found in data?
    • Time-series Data Patterns: Trends, Seasonality, Correlation, Noise
  7. Data Relations: What are the relationships within data?
    • Relational Data, Graph Data, Document Data (Key-value Data, JSON Data)
    • Multi-dimensional Data, Spatial Data
  8. Data Storage Types:
    • Block Data, Object (Blob) Data
  9. Data Management Systems:
    • Filesystem, Database, Data Lake, Data Mart, Data Warehouse, Data Lakehouse
    • Data Indexes
  10. Data Governance, Security, Privacy:
    • Data Catalog, Data Quality, Data Schema, Data Model
    • Data Masking, Data Privacy

More blogs to deep dive into each category and the challenges involved. Let’s peel this onion.

Tunnel Vision

When I drive through a tunnel with my kid, there are two points of excitement – entry point and exit point. When entering a new tunnel, it’s always a “Yay!!” The exit feeling depends on how long we were inside. It’s either the expression – “Finally some light” or “Oh, no, we are out.”

You get my point. We love tunnels. We like to see the light at the end of the tunnel.

We not only love tunnels, but we love to tunnel. Tunneling helps us focus, and there is a focus threshold after which we need to see some light.

Sprinting in agile is Tunneling. After a two-week sprint, we might have the expression, “Oh, no, we are out too soon.”, and after a four-week sprint, we might have the expression, “Finally, some light.”

A two weeks sprint seems to be a global average of adequate time in a sprint. While that is an indicator, the team must choose their sprint duration.

Not all people are alike. Some people like to be first finishers – Tunneling helps them to get from A to Z fastest. However, we remember our journeys for unplanned explorations. Can we visit that lake nearby? Can we bathe in that waterfall? Can we take a different road?

In software projects, the expectation from engineers/architects is that they build a path from point A to Z as fast as possible (tunnel), but the way should be open to exploration by the user. E.g., Build a mobile user interface to update my health parameters, but let me explore new services in the user interface to improve my health.

If you are agile, you can introduce innovation experiments to explore users unwritten needs, making your own journey not feel like a long tunnel.

Takeaway: We need black-box tunnels, exciting tunnels, and open roadways, and in agile software parlance, that loosely equates to sprints, spikes, and MVPs.

Go Slow to Go Fast

“I am adopting Agile; finally, I can tell my customers that since we are going to be agile, the delivery will be late. The agile coach told me that to go fast; you need to slow down. If the customer has questions, I will ask the agile coach to help convince the customer” – Project manager

Sometimes, projects need to be rescued. They have either messed up the quality, or schedule, or both. They had some “accurate estimations” done 12 months back for the entire project. Now they have to prove that their estimations were not wrong irrespective of the scope creep, requirements ambiguities, technology risks that popped up, and COVID impacting day-2-day life.

Such projects can be rescued without agile principles. The team can get a stock of current realities and re-plan, and continue this process until successful. There are very successful waterfall projects. There are failed waterfall projects that were rescued with waterfall. A non-agile method to rescue the project will still ask you to stop, re-plan, and continue, i.e., go slow (stop-think) to go fast.

An agile method to rescue the project will also ask you to stop, re-plan, and continue iterations. i.e., deliver a small “good” quality feature increment that does not break the product, then continue. While some team members have a sprint tunnel vision, others will look beyond a sprint to ensure that they have “ready” features to add to the product when they meet their DOD, and their stories are accepted. After some sprints, the team knows more about its velocity and can predict a schedule (with a guaranteed quantum of quality). The backlog would also be groomed (prioritized, broken-down, detailed) in parallel by the product owners while the team is sprinting on the stories that were picked up. Agile enables teams to continue on features that are “ready” and not “halt.” If the reason for distress is “quality,” then the stories could be “debt” that must be paid off until we can build some more.

Agile is not an excuse to stop. Even when you are agile, you can accumulate debt that results in an inferior quality product, and prioritization/judgment must ensure that the debt is not out of control. There is always good debt (to gain speed) and bad debt (than hampers quality); and some intersection of the two. Even in agile, there is an in-built “stop” with iterations and a “STOP” requested by the team/customer.

Takeaway: Agile or non-agile a project in distress requires to stop, think, and continue.

A best practice that I have used to prevent projects to get into a bad state is to build an architectural runway (intentional architecture) outside the sprinting zone. “Readiness” to sprint is critical for success of the team.