Kashmir

It’s Tvisha’s summer vacation, and due to work pressures, we could not take her out of Bangalore in April. So she was bugging us to get out of Bangalore. After comparing Leh/Ladakh, Sikkim, Andamans, and Kashmir, we finally settled on Kashmir. I had been to Kashmir when I was very young and had fond memories of the place. I was unsure whether it was safe to visit Kashmir, and after much online research and talking to the travel agents, we zeroed in on Kashmir.

Some friends were going on a Bangalore-Leh drive (9K+ Kms), and others were going to monasteries in Sikkim. We had recently been to monasteries in Bhutan, and the girls were not in the mood for another long drive! In December 2021, we enjoyed a seven-day long drive (Bangalore-Coonoor-Kotagiri-Coimbatore-Mahabalipuram-Bangalore). The weatherman predicted rains in Andamans, and the daughter wanted to make Olaf! So, Kashmir was inevitable!

After comparing rates and dates with our favorite travel agent (SOTC), we finally booked our trip with the MakeMyTrip holidays.


Day 1 – Bangalore to Srinagar

There is a direct Indigo Flight (6E 797) from Bangalore to Srinagar that starts in Bangalore at a convenient time (~9:15 AM) and reaches Srinagar post noon (~2:00 PM) with a brief halt at Amritsar.

When the plane landed at the Srinagar airport, all passengers were in awe of Srinagar’s beautiful hills (some capped with snow). At the airport, our driver greeted us, and he instantly realized that we South Indians were in Kashmir to see snow in summer! He promised us ample snow in Sonmarg and Gulmarg, making Tvisha very happy!

It was a short drive from the airport to our first hotel in Srinagar. It was not a classic hotel but a houseboat. The houseboat we stayed in was Naaz Kashmir (https://www.naazkashmir.com/) and was located in Nageen Lake (and not Dal Lake). All the lakes in Srinagar are connected, but Nageen Lake is less commercial and crowded.

The rest of the day was free for us to enjoy the houseboat! We spent our time taking photos, watching fishermen and birds catch fish, eating pakoras, listening to the music of the water and birds, listening to the prayers from different surrounding Mosques, dressing up in a traditional dress, and a short shikara ride.

It was an abrupt stop to the fast life we are used to, staring back at nature and adoring its beauty. We talked to each other as a family and were not immersed in our gadgets! However, Tvisha was excited to show off her first day in Kashmir to her friends in a WhatsApp call!

Naaz Kashmir served us well – candlelight dinners, food per our needs, and recommendations about Shikara rides. So we took their advice and decided to take the 4:30 AM four hours Shikara ride from Nageen Lake to Dal Lake.

Staying in a houseboat is like sharing a house with other equally clueless and excited families.


Day 2 – Srinagar (Shikara Ride)

We kept the alarm to wake us up at 4:00 AM to prepare for our 4:30 AM shikara ride. We woke up to the alarm and morning prayers at the lake. It reminded me of my pre-college days when 4:00 AM had become a routine for studies. The caretakers were already up and knocked on our doors to ensure that we were ready and sent us on the shikara with some hot Kashmiri tea (Kahwa) and snacks.

It was bitter cold (for us) and pleasant for the locals. We tugged ourselves into the blanket available in the shikara. Our little braveheart sandwiched herself conveniently between her parents, refused to step out of the blanket throughout the ride, enjoyed the frequent warm rubs, and did not hesitate to nap. The shikara moved slowly, confidently, and thoughtfully through the lake.

The cold air on the face and the calming sound of the shikara moving in the lake is an unforgettable experience. Most of the locals and birds were up at 4:30 AM!! As we rowed through the lake, we could see the homes of the locals – men and women at work. Everyone that met the eye returned a welcoming smile.

We spotted a water snake, various colored water lilies, and birds like eagles, geese, mallards, pochards, gadwalls, pintails, waders, coots, and the common teal. The shikara rider helped us identify the birds.

We could see water vapor causing fog in some places due to the difference in water temperatures and the surroundings.

The shikara rider explained to us in detail the unique farming done by the locals in the lake, i.e., the process in which they grow vegetables. These farmers grow carrots, radishes, turnips, and other vegetables in the soil that floats on flora beds.

Such floating gardens are maintained all over the lake, and the farmers move through their plots on boats carrying their harvest to the lake’s market area. The rider took us through the floating gardens and the lotus farms to the floating vegetable market on dal lake. We bought some flowers and seeds and had some hot Kahwa (Kashmiri Tea) and snacks packed for us by the Naaz Kashmir at this market.

The rider then took us to the dal lake (pronounced as दल and not दाल), and at this time of the day, it was empty except for the locals fishing out the water weeds to compost them for use in their farms. It was a beautiful sunrise to watch at the dal lake, wade through the markets (Meena Bazaar), and spot traditional homes. The ride back was slow, and the gentle swaying motion of the shikara ride put us to sleep for a few minutes.


Day 2 – Sonmarg (सोनमर्ग and not सोनमIर्ग)

At around 9:30 AM, after breakfast, we started driving to Sonmarg, a hill station in the Ganderbal district.

It was about two hours drive. The roads were not too good, and there were traffic jams in a few places. If we had left about an hour earlier, we might have saved about 1/2 hour of jam time. It was visitor traffic. However, after the early morning Shikara ride, we needed some time to fuel and freshen ourselves before our next adventure.

As we reached closer to Sonmarg, we could see the snow-capped mountains, feel the drop in temperature, and breathe the superior quality air. Sonmarg was much cooler than Srinagar but fortunately pleasant (even for our visitor skin).

During the drive, we could see streams of water – the tributaries of Jhelum – Lidder, Sind, and Neelum. The sight of white water forcing itself down the mountains was strangely peaceful. It reminded us of our stay in Salt Lake City, Utah.

The driver told us that during the winter months, heavy snowfall blocks NH-1. So a tunnel is being built to keep the road open year-round. We found trekkers ready to trek from Sonmarg to Leh (“the rooftop of the world”) and groups wanting to drive through the Zoji La pass. Our driver informed us that driving through the Zoji La pass is a must-do off-road adventure. However, he recommended we not take our city car and take a four-wheel drive as the roads are narrow and rocky. It seems it’s a trek worth doing! So, now this one is also added to the backlog!

However, our goal for today was relatively simple. We either trek or take ponies to the Thajiwas glacier. This glacier is a favorite summer destination at Sonmarg.

My daughter is always excited and happy around animals. We were not sure whether we (“The Adults”) needed ponies; however, the agents there convinced us that the pony ride is a “must-do” in Sonmarg. We gave in; however, it would have been a simple to moderate walk/trek and less burden on the animals in hindsight. The ponies were expensive, about 2750 INR / person (we negotiated out more than 50% of the original demand. Better negotiators got it at around 1500 INR / person). We also got a photographer for us to be more hands-free. The pony ride was uphill, downhill, and through cold water streams.

The scenery was picture perfect, a silvery scene set against green meadows and a clear blue sky. The views were captivating, and we outsourced the picture capture activity to our photographer and instead enjoyed the views. The air was too fresh to keep our masks on.

At the glacier, Tvisha finally made her Olaf! She was throwing snowballs at us in all directions, even when we were haggling with the locals to reduce the sledding costs! She was able to go up and down the snow and was soaking in the happiness, unlike us. Then, after an uphill trek in the snow, we sat down on a rock and came down sledding. The weather was not too cold, and if it were not for the time to return to Srinagar and limited food options, we would have spent more time up at the glacier.

We encountered the local police (to the surprise of the locals), who helped us reduce the cost of sledding from 3500/- to 500/- INR (though we ended up paying 1000 INR per person for sledding). It’s funny that the locals keep saying (क्या आप खुश है, हमे भी खुश कीजिये) “Are you happy? Make us happy!”, a method to get more money from the visitors. These people make money only during the summer months (visitor months) and try to make the most of it. COVID & CONFLICT closures have been hard on them. We found these people to be cheerful, happy, and helpful. So, we did not hesitate to give (tips) more than we thought was reasonable!

We stopped at a roadside restaurant to have some delicious chole-puri and dal-makhani for Tvisha on our way back. We reached back around 7:00 PM for another candlelight dinner at Naaz Kashmir. The owners of Naaz Kashmir had moved our luggage out of the room as they thought that we booked for only one night, and they realized that there was a communication issue between MakeMyTrip and their reservation team. They made up for it by giving us a superior room, a chocolate cake for Tvisha, and several apologies.

We took a warm bath and crashed soon after!

We were woken up by Tvisha mid-night as she started vomiting and was feeling very sick! She had no temperature but did not look well. We were not sure whether it was the altitude, change in food/water, or a stomach bug. Thankfully, we had carried medicines – “Enterogermina” and “Calpol” in our first aid kit!


Day 3 – Pahalgam

We started around 9:45 AM after thanking the caretakers for their service at Naaz Kashmir and completing the check-out formalities. Tvisha was sick and only managed to eat mangoes for breakfast. The drive to Pahalgam was long but on more convenient and motorable roads. We wanted to stop for Kahwa and visit apple farms, but Tvisha would only sleep in the car. So, we drove straight to our hotel, the Radisson Golf Resort.

The Pahalgam scenery was unique, with white water streams and a backdrop of pine trees and snow-capped mountains. The temperature was pleasant and leaning towards too cool.

All we could do on day-3 was check in and rest. Tvisha slept all afternoon and night. She was running a slight fever, so we consulted our family doctor, and she diagnosed her to have the stomach flu. So we requested the hotel to get us medicines (the medicine shop was ~2KMS far), and they helped us.

The trip managers advised us to visit Abu Valley, Betaab Valley, Chandanwari, and Baisaran valley (Mini-Switzerland). However, we had lost 1/2 a day and had to check out by 1:00 PM the next day (Day-4). So we decided that if Tvisha feels better on Day-4, we will do the Baisaran valley.

Pahalgam (First Village) gets its name from Hindi “पहला गांव” and Shiva devotees visit this place frequently in summer to pilgrimage to the Amarnath Cave for the Darshan of the only ice stalagmite Shiva Linga. The pilgrimage usually starts from Chandanwari but was closed due to road work during our travel.


Day 4 – Pahalgam

Tvisha woke up ready for her next adventure; however, I felt queasy in my stomach. It was my turn to fall sick. However, Dolo came to my rescue, and after an insignificant breakfast, we all jumped up on horses to visit the Baisaran valley. This trek is captivating, and it’s better to trek than take horses. However, a just recovered Tvisha and a queasy daddy were not in any state to trek. So, horses again!

Baisaran Valley is a hilltop green meadow dotted with dense pine forests and surrounded by snowcapped mountains. This famous offbeat tourist place is excellent for those wanting to spend a quiet time in the company of nature. It also serves as a campsite for trekkers going to Tulian Lake. Some of the famous tourist points you can see en route to Baisaran are Pahalgam Old Village, Kashmir Valley Point, and Deon Valley Point. You can also enjoy panoramic sights of Pahalgam town & Lidder Valley from here.

We returned to our hotel after about four hours (~1:00 PM) ride to Baisaran. We completed our checkout and had a delicious rainbow trout tandoori (a local delicacy) for lunch. The driver told us that butter or olive fry is better. After a sumptuous meal, we departed by car to Gulmarg.


Day 4 – Pahalgam to Gulmarg (गुलमर्ग )

It took about 3.5 hours from Pahalgam to reach Gulmarg. Gulmarg is a ski destination and is famous for winter sports. It is called the meadow of flowers in summers. Our driver informed us that they have to use chains on wheels in winter, and the snowfall in Gulmarg is heavy. The beauty of Gulmarg was different (in a good sense) than Pahalgam and Sonmarg. The first thing that strikes out is the lush green meadows (in summers).

We camped in the Hilltop Hotel. At first glance, the hotel seems jaded, faded, and under maintenance. While that is true, the rooms were well designed, and the in-room dining service was good. The service at this hotel was better than any of our previous experiences in Kashmir, even though they were ok at room cleaning services, breakfast variety, bathroom accessories, changing towels, or responding to your hails. We were satisfied that we could have a warm bath, eat something edible, and reach on time for the gondola ride the following day. The hotel is close to the gondola ride and ice skating rink.

Dolo carried me only this far, and I had slight chills and crashed for the night. I hoped, wished, and prayed that the fever would help kill the virus/bacteria (stomach bug) for me to enjoy Gulmarg the next day.


Day 5 – Gulmarg – Gondola Ride to Kongdoori

The chills were gone in the morning; however, I was still queasy. So, I trusted my best friend (Dolo) and braved the gondola.

MakeMyTrip was able to arrange gondola tickets for the first stage (Base Station to Kongdoori). The tour guide told us that the second stage tickets (Kongdoori to Apharwat) were unavailable (sold out). However, in hindsight, we did not regret not being able to do the second stage.

The gondola wait lines and wait times are infamous. People start queuing at 7:00 AM, even though the ride opens at 9:30 AM. We reached the queue at around 8:00 AM and found our tour guide. The people standing in the line entertained themselves by fighting with others who tried to move ahead. Words and punches were flying until the ride opened at 9:30 AM. The locals were also amused at the sight. The trip was about 10 minutes, and the wait time to board the ride was 2 hours. Dolo kept me on my feet.

The gondola ride is short, and the views are terrific. The valley is picturesque, and we can spot the snowcapped Himalayas, Apharwat, and Mud houses from the ride. We could also see the unlucky visitors (those who could not get a gondola ticket) trekking or using horses to climb up to Kongdoori. This trek is a moderate to challenging hike.

At Kongdoori, people had to queue again to ride to Apharwat, and the queue was equally long. We were happy that we don’t have to wait in another queue.

Gondola Ride to Kongdoori (Friend: Dolo)

At Kongdoori, the horse owners were bugging us to take a horse ride to the waterfall. However, we discussed it with our tour guide and decided to trek. Trekking (and not horse riding) was the best decision. We could stop to look at the multi-colored flowers in the meadows, jump over streams, trek with the goats, stop to hear the sound of silence, spot lizards, take photos, and experience rocky trails.

When we reached the waterfall, we had to trek in snow to get up to the mouth of the waterfalls. The snow trek was challenging.

However, the mountain water was delicious and pure. We drank from the waterfall, and this water tasted better than any mineral or filtered water. So we filled a bottle to quench our thirst for the return journey.

If I travel again to destinations like these, I will remind myself to buy shoes with some grip.

Finally, we sledded downhill and relished on some delicious maggie cooked in the mountains waters. Strangely, maggie gave us the strength to trek back to the gondola ride station.

We took a different route to see the valley and meadows from a different perspective. This route was shorter and required us to climb uphill and roll downhill.

Again, the view was picture perfect!

After reaching the base station, we rushed to feed ourselves a late lunch. The lunch was good. We did not have any energy left to do the ATV rides and decided to skip them and relax in the room. Then, in the evening, we went down to the cafeteria to have some snacks. Our legs could not tolerate any more walking and would only walk back in the direction of the hotel room. So, we snuggled back into the room and watched the evening walkers from the comfort of the room.

Days are long in Gulmarg, and it’s bright even at 7:00 P.M.

I recovered from the stomach bug (Thanks! Enterogermina), and now it was time for my wife to fall sick to the same bug! She had a better immune reaction to the bug than my daughter or me; however, she sought help from Dolo and Enterogermina to fight off the bug.

Day 6 – Back to Srinagar

Gulmarg is about ~50Kms from Srinagar. So, the return journey was short. We woke up late and lazy, and left for Srinagar after a late breakfast.

We stopped to see apple farms and drink delicious green apple juice. We tasted various homemade pickles and bought lotus stem pickles from the farmer. Lotus stem (कमल-ककड़ी), locally known as “Nadru” is grown in shallow parts of water bodies like ponds and lakes and is a vastly enjoyed ingredient in Kashmiri cuisine.

We also stopped to have some premium Kahwa and buy some dry fruits (Walnuts) and condiments (Kesar).

We checked into Radisson Srinagar, the first hotel in Kashmir, where we found women employees. My wife had a heart-to-heart talk about women empowerment with the ladies there!

After a good lunch, we headed to see the oldest temple in Kashmir, the Shankaracharya Temple, dedicated to Lord Shiva. The temple is a monument of national importance and is protected by ASI (Archeological Society of India). There are many steps to climb, and the view of Kashmir valley from the hilltop is superb. It was very windy up the hill and pleasant. Unfortunately, cameras were not allowed for a picture remembrance. After blessings from Lord Shiva, we decided to stroll the gardens of Srinagar.

Unfortunately, due to the holiday rush and this day being a Sunday (many locals were out sightseeing), we saw only the Botanical Garden and the Chashme Shahi. We missed the Tulips as the Tulip Garden was closed a few days back (Peak season: April). The botanical garden was a nice walk, and the flowers in Chashme Shahi were exquisite. My daughter enjoyed taking photos of several flowers.

We had earlier decided to ride the Shikara again at Dal Lake; however, we decided to dart back to the hotel, looking at the rush and weather. Finally, we finished the day with a lavish buffet dinner.

Day 7 – Back to Home @ Bangalore

The only eventful activity was the security checks at the Srinagar Airport. We must step out of our cars at least a kilometer before the airport and have to get ourselves, the car, and the bags checked.

We left Srinagar entirely mesmerized by the beauty of Kashmir, and I decided to pen this down in a blog (for us) so that this never fades from (our) memory. So this blog is my first travel blog.

After a few more doses of Enterogermina and home food, my wife got better. We have returned to our workaholic ways and keep discussing our Kashmir trip with friends and family.

Revolutionizing SDLC with AI Agents

AI is Rewiring the Software Lifecycle.

The AI landscape has shifted tectonically. We aren’t just talking about tools anymore; we are talking about teammates. We’ve moved from “Autocomplete” to “Auto-complete-my-entire-sprint.”

AI Agents have evolved into autonomous entities that can perceive, decide, and act. Think of them less like a calculator and more like a hyper-efficient intern who never sleeps, occasionally hallucinates, but generally gets 80% of the grunt work done before you’ve finished your morning coffee.

Let’s explore how these agents are dismantling and rebuilding the Agile software development lifecycle (SDLC), moving from high-level Themes down to the nitty-gritty Tasks, and we—the humans—can orchestrate this new digital workforce.

Themes to Tasks

In the traditional Agile world, we break things down:

Themes > Epics > Features > User Stories > Tasks.

AI is advertised only at the bottom—helping you write the code for the Task. However, distinct AI Agents specialize in every layer of this pyramid.

Strategy Layer (Themes & Epics)

The Role: The Architect / Product Strategist

The Tool: Claude Code / ChatGPT (Reasoning Models)

The Vibe: “Deep Thought” At this altitude, you aren’t looking for code; you’re looking for reasoning. You input a messy, vague requirement like “We need to modernize our auth system.” An agent like Claude Code doesn’t just spit out Python code. It acts like a Lead Architect. It analyzes your current stack, drafts an Architecture Decision Record (ADR), simulates trade-offs (Monolith vs. Microservices), and even flags risks (FMEA).

Translation Layer (Features & Stories)

The Role: The Product Owner / Business Analyst

The Tool: Jira AI / Notion AI / Productboard

The Vibe: “The Organizer” Here, agents take those high-level architectural blueprints and slice them into agile-ready artifacts. They convert technical specs into User Stories with clear Acceptance Criteria (Given-When-Then).

Execution Layer (Tasks and Code)

The Role: The 10x Developer

The Tool: GitHub Copilot / Cursor / Lovable

The Vibe: “The Builder” This is where the rubber meets the road. The old way: You type a function name, and AI suggests the body. The agentic way: You use Cursor or Windsurf. You say, “Refactor this entire module to use the Factory pattern and update the unit tests.” The agent analyzes the file structure, determines the necessary edits across multiple files, and executes them by writing code.

Hype Curve of Productivity

1 – Beware of Vapourcoins.

Measuring “Time Saved” or “Lines of AI code generated” is a vanity metric (or vapourcoins). It doesn’t matter if you saved 2 hours coding if you spent 4 hours debugging it later.

Real Productivity = Speed + Quality + Security = Good Engineering

The Fix: Use the time saved by AI to do the things you usually skip: rigorous unit testing, security modeling (OWASP checks), reviews, and documentation.

2 – Measure Productivity by Lines Deleted, Not Added.

AI makes it easy to generate 10,000 lines of code in a day. This is widely celebrated as “productivity.” It is actually technical debt. More code = more bugs, more maintenance, more drag.

The Fix: Dedicate specific “Janitor Sprints” where AI is used exclusively to identify dead code, simplify logic, and reduce the codebase size while maintaining functionality. Build prompts that leverage AI to refactor AI-generated code into more concise, efficient logic. Build prompts that use AI to refactor AI-generated code into reusable libraries/frameworks. Explore platformization and clean-up in Janitor Sprints.

3 – J Curve of Productivity

Engineers will waste hours “fighting” the prompt to get it to do exactly what they want (“Prompt Golfing”). They will spend time debugging hallucinations.

The Curve:

Months 1-2: Productivity -10% (Learning curve, distraction).

Months 3-4: Productivity +10% (Finding the groove).

Month 6+: Productivity +40% (Workflow is established).

The Fix: Don’t panic in Month 2 and cancel the licenses. You are in the “Valley of Despair” before the “Slope of Enlightenment.”

AI Patterns & Practices

1 – People Mentorship: AI-aware Tech Lead

Junior developers use AI to handle 100% of their work. They never struggle through a bug, so they never learn the underlying system. In 2 years, you will have “Senior” developers who don’t know how the system works.

The Fix: AI-aware Tech lead should mandate “Explain-to-me”. If a Junior submits AI-generated code, she must be able to explain every single line during the code review. If they can’t explain it, the PR is rejected.

2 – What happens in the company, Stays in the company.

Engineers paste proprietary schemas, API keys, or PII (Personally Identifiable Information) into public chatbots like standard ChatGPT or Claude. Data leakage is the fastest way to get an AI program shut down by Legal/InfoSec.

The Fix: Use Enterprise instances (ChatGPT Enterprise). If using open tools, use local sanitization scripts that strip keys/secrets before the prompt is sent to the AI tool.

3 – Checkpointing: The solution to accidental loss of logic

AI can drift. If you let an agent code for 4 hours without checking in, you might end up with a masterpiece of nonsense. You might also lose the last working version.

Lost Tokens = Wasted Money

The Fix: Commit frequently (every 30-60 mins). Treat AI code like a junior dev’s code—trust but verify. Don’t do too much without a good version commit.

4 – Treat Prompts as Code.

Stop typing the exact prompt 50 times.

The Fix: Treat your prompts like code. Version Control, Optimize, Share. Build a “Platform Prompt Library” so your team isn’t reinventing the wheel every sprint. E.g., Dockerfile generation best-practices prompt, Template Microservices generation/updation best-practices prompt, etc. Use these as context/constraints. Check-in prompts along with code in PRs. Prompt AI to continuously build/maintain prompts for autonomous execution, using only English.

5 – Context is King.

To make agents truly useful, they need to know your world. We are seeing a move toward Model Context Protocol (MCP) servers (like Context7). These allow you to fetch live, version-specific documentation and internal code patterns directly into the agent’s brain, reducing hallucinations and context-switching.

6 – Don’t run a Ferrari in a School Zone.

Giving every developer access to the most expensive model (e.g., Claude 4.5 Sonnet or GPT-5) for every single task is like taking a helicopter to buy groceries. It destroys the ROI of AI adoption. Match the Model to the Complexity.

The Fix: Low-Stakes (Formatting, Unit Tests, Boilerplate): Use “Flash” or “Mini” models (e.g., GPT -4 Mini, Claude Haiku). They are fast and virtually free. High-Stakes (Architecture, Debugging, Refactoring): Use “Reasoning” models (Claude 4.5 Sonnet).

7 – AI Code is Guilty Until Proven Innocent

AI code always looks perfect. It compiles, it has comments, and the variable names are beautiful. This leads to “Reviewer Fatigue,” where humans gloss over the logic because the syntax is clean.

The Fix: Implement a rule: “No AI PR without a generated explanation.” Force the AI to explain why it wrote the code in the PR description. If the explanation doesn’t make sense, the code is likely hallucinated. In code reviews, start looking for business logic flaws and security gaps. Don’t skip code reviews.

8 – Avoid Integration Tax

You let the AI write 5 distinct microservices across 5 separate chat sessions or separate teams. Each one looks perfect in isolation. When you try to wire them together, nothing fits. The data schemas are slightly off, the error handling is inconsistent, and the libraries are different versions. You spend 3 weeks “integrating” what took 3 hours to generate.

The Fix: Interface-First Development. Use AI to define APIs, Data Schemas (JSON/Avro), and Contracts before a single line of code is generated. Develop contract tests and govern the contracts in the version control system. Feed these “contracts” to AI as constraints (in prompts).

9 – AI Roles

Traditionally, engineers on an agile team took on roles such as architecture owner, product owner, DevOps engineer, developer, and tester. Some teams invent new roles, e.g., AI librarian, PromptOps Lead, etc. This is bloat!

The Fix: Stick to a fungible set of traditional Agile roles. The AI Librarian (or system context manager) is the architecture owner’s responsibility, and the PromptOps Lead is the scrum master’s responsibility. Do not add more bloat.

10 – The Vibe Coding Danger Zone

The team starts coding based on “vibes”—prompting the AI until the error message disappears or the UI “feels” right, without reading or understanding the underlying logic. This is compounded by AI Sycophancy: when you ask, “Should we fix this race condition with a global variable?”, the AI—trained to be helpful and agreeable—replies, “Yes, that is an excellent solution!” just to please you. You end up with “Fragileware”: code that works on the happy path but is architecturally rotten.

The Fix: Institutional Skepticism. Do not skip traditional reviews. Use “Devil’s Advocate Prompts” to roast a decision or code using a different model (or a new session). Review every generated test and create test manifests before generating tests. Build tests to roast code. No PR accepted without unit tests.

The 2025 Toolkit: Battle of the Bots

The AgentThe PersonalityUse for
Claude CodeThe IntellectualComplex reasoning, system design, architecture, and “thinking through” a problem. It creates the plan.
GitHub CopilotThe Enterprise StandardSafe, integrated, reliable. It resides in your IDE and is aware of your enterprise context. Great for standard coding tasks.
CursorThe DisruptorAn AI-first IDE. It feels like the AI is driving and you are navigating. Excellent for full-stack execution.
Lovable / v0The Artist“Make it pop.” Rapid UI/UX prototyping. You describe a dashboard; they build the React components on the fly.
Table 1: Battle of Bots

One size rarely fits all. A tool that excels at generating React components might hallucinate wildly when tasked with debugging C++ firmware. Based on current experience, here is the best-in-class stack broken down by role and domain.

Function🏆 Gold Standard🥈 The Challenger🥉 The Specialist
Architecture & DesignClaude CodeChatGPT (OpenAI)Miro AI
Coding & RefactoringGitHub CopilotClaude CodeCursor
Full-Stack BuildCursorReplitBolt.new
UI / FrontendLovablev0 by VercelCursor
Testing & QAClaude CodeGitHub CopilotTestim / Katalon
Docs & RequirementsClaude CodeNotion AIMintlify
Table 2: SDLC Stack
Phase🏆 The Tool📝 The Role
Threat Modeling
(Design Phase)
Claude Code / ChatGPTThe Architect.
Paste your system design or PRD and ask: “Run a STRIDE analysis on this architecture and list the top 5 attack vectors.” LLMs excel at spotting logical gaps humans miss.
Detection
(Commit/Build Phase)
Snyk (DeepCode) / GitHub Advanced SecurityThe Watchdog.
These tools use Symbolic AI (not just LLMs) to scan code for patterns. They are far less prone to “hallucinations” than a Chatbot. Use them to flag the issues.
Remediation
(Fix Phase)
GitHub Copilot Autofix / Nullify.aiThe Surgeon.
Once a bug is found, Generative AI shines at fixing it. Copilot Autofix can now explain the vulnerability found by CodeQL and automatically generate the patched code.
Table 3: Security – Security – Security
DomainSpecific Focus🏆 The Power Tool🥈 The Alternative / Specialist
Web & MobileFrontend UILovablev0 by Vercel (Best for React/Tailwind)
Full-Stack IDECursorBolt.new (Browser-based)
Backend LogicClaude CodeGitHub Copilot
Mobile AppsLovableReplit
Embedded & SystemsC / C++ / RustGitHub CopilotTabnine (On-prem capable)
RTOS & FirmwareGitHub CopilotClaude Code (Best for spec analysis)
Hardware TestingClaude CodeVectorCAST AI
Cloud, Ops & DataInfrastructure (IaC)Claude CodeGitHub Copilot
KubernetesK8sGPTClaude Code (Manifest generation)
Data EngineeringGitHub CopilotDataRobot
Data Analysis/BIClaude CodeThoughtSpot AI
Table 4: Domain Specific Powertools

Final Thoughts

The AI agents of 2025 are like world-class virtuosos—technically flawless, capable of playing any note at any speed. But a room full of virtuosos without a leader isn’t a symphony; it’s just noise.

As we move forward, the most successful engineers won’t be the ones who can play the loudest instrument, but the ones who can conduct the ensemble. We are moving from being the Violinist (focused on a single line of code) to being the Conductor (focused on the entire score).

So, step up to the podium. Pick your section leads, define the tempo, and stop trying to play every instrument yourself. Let the agents hit the notes; you create the music. Own the outcome.

Coorg

Our family’s summer vacation began with a road trip to Coorg, timed perfectly with the end of my daughter’s school year. She’s always loved Coorg’s cooler climate and rope activities making it an ideal destination for our getaway.

This visit was special—we arrived during the coffee flowering season, something we hadn’t experienced in our previous trips. Rather than rushing between tourist attractions, we focused on immersing ourselves in the region’s tranquility and the facilities at the resort. The lush green landscapes provided a welcome contrast to Bangalore’s busy traffic.

The fragrance of coffee blossoms filled the air as we spent our days simply enjoying the peaceful surroundings, creating memories in this serene hill station.

The Journey

The drive from Bangalore to Coorg took approximately 7 hours. We started our day early, leaving home at 7:30 AM and picking up my parents by 8:15 AM. Our route took us via Mysore, though we didn’t enter the city itself, instead taking the direct road to Coorg.

Unlike our previous trips where we would traditionally stop for lunch at the Novotel hotel in Mysore, this time we decided to continue straight to our destination. Bangalore’s morning traffic necessitated a breakfast stop in the city before we could reach the highway.

Once on the Bangalore-Mysore highway, the journey became much smoother. The excellent road conditions allowed for comfortable cruising, with most vehicles maintaining speeds under 100 KMPH, though some drivers tested both speed limits and the patience of fellow travelers. We noticed that the number of food and refueling options along the route had increased significantly in recent months.

We drove non-stop on the highway stretch, but after passing Mysore and heading toward Coorg, we needed a break. We stopped at McDonald’s for burgers and coffee, which gave us the energy to continue. The road quality from Mysore to Coorg wasn’t as good as the highway we had just left. Along this stretch, we made another stop to buy jackfruits – a fruit with a strong smell but delicious taste that’s definitely an acquired preference. This detour provided enough sustenance to complete our journey to Coorg.

We had reservations at the Coorg Marriott Resort and Spa. Following Google Maps’ directions led us through extremely narrow plantation roads during the final stretch. While the scenery was beautiful, we worried about getting stuck. We did encounter a vehicle coming from the opposite direction, but thankfully that driver skillfully reversed to a point where I could maneuver past. I later discovered there were alternative routes available – a lesson that trusting Google Maps isn’t always advisable, especially when it directs a car onto what seems like a bike path. Perhaps someday navigation systems will better differentiate between vehicle types!

Once we arrived, one of the first things that struck me was the unique aroma in the air — a jasmine-like scent coming from the coffee flowers. We were lucky enough to visit during the coffee blooming season, and the sight of coffee plants with tall, flower-covered stems was nothing short of mesmerizing.

Coffe @ Coorg

Coorg is renowned for its coffee, cultivating both Robusta and Arabica varieties. Each bean type offers distinct characteristics:

Robusta lives up to its name with a stronger, more robust profile. It delivers bitter notes with earthy or woody aftertastes and possesses a fuller body. Robusta contains higher caffeine levels than Arabica but has lower acidity, sugar, and oil content.

Arabica produces a lighter-bodied coffee with sweeter, floral flavors.

Most coffee brands blend these varieties in different ratios. Traditional South Indian coffee typically contains 70% Robusta and 30% Arabica. Premium cappuccinos, such as those served at Third Wave Coffee, generally use mostly Arabica beans as they complement milk better. While Robusta plants have larger leaves than Arabica, their beans are actually smaller.

The local coffee possesses a unique flavor profile influenced by Coorg’s rainforest environment. I was intrigued to learn about the region’s traditional practice of growing coffee under trees, unlike the open-field plantations found elsewhere. This canopy provides essential shade and protection for the coffee plants, allowing them to thrive harmoniously with their surroundings. Since coffee plants still require sunlight, farmers strategically trim the taller trees to allow filtered light to reach the coffee bushes below.

Coorg coffee plants follow a deliberate growth timeline before reaching their productive potential. Farmers must be patient, as the first yields only appear after 2-3 years of planting. Full production capacity isn’t achieved until the plant reaches 5-7 years of age.

These plants are remarkably long-lived, with most coffee plants in Coorg surviving well beyond 50 years. Under ideal conditions, some even reach 80 years of age while still producing coffee cherries.

Plantation management requires ongoing attention. Farmers regularly remove volunteer seedlings that sprout from fallen seeds to maintain proper spacing and organization within the plantation. Trimming is an essential practice that improves overall yield, reduces excessive foliage, promotes lateral branching where coffee cherries develop, and creates more accessible plants for efficient harvesting.

This careful cultivation process contributes to Coorg’s reputation for high-quality coffee production. Generations of farmers have refined these management techniques over decades.

Coffee flowers are one of Coorg’s seasonal treasures, appearing typically once a year. The blossoms are small, white, star-shaped flowers that cluster along coffee branches, resembling jasmine in appearance. Their fragrance is exceptionally sweet and delicate – often described as a blend of jasmine, orange blossom, and honey with subtle vanilla undertones. The flowering period is brief but memorable, lasting only 2-3 weeks. In Coorg, this typically occurs between February and April, depending on weather conditions and the timing of the first pre-monsoon showers. The scent is most intense in the early morning when the flowers release their fragrance in waves that can carry for surprising distances. About 6-9 months after flowering, these blossoms will develop into the coffee cherries that contain the beans.

Beyond coffee, Coorg is a treasure trove of spices. A highlight of my visit was exploring local ingredients like pepper, which is extensively cultivated in the region. Coorgi pepper stands out for its exceptionally robust flavor, making it distinctive.

A Relaxing Stay

Our accommodation at the Marriott Resort & Spa in Coorg exceeded expectations. We chose a plantation villa that provided the perfect retreat after our long journey. The comfortable beds ensured restful nights, while the spacious balcony offered an ideal setting for morning meditation overlooking the lush plantation landscape.

The resort’s well-maintained pathways provided excellent routes for our morning walks, allowing us to explore the property while enjoying the crisp Coorg air. The bathrooms were impressively large—luxuriously so—with high-end fixtures and amenities that enhanced our comfort.

Situated in the heart of Coorg, the resort struck a perfect balance between luxury and natural beauty. We made the most of our stay by exploring the scenic surroundings, indulging in the resort’s delicious cuisine, and spending leisurely hours by the inviting swimming pool. Taking refreshing dips in the pool became a daily highlight, offering welcome relaxation amid Coorg’s serene environment.

Coorg—Where Peace Comes with a Side of Coffee

Our trip to Coorg wasn’t about frantically checking tourist attractions off a list. Instead, we embraced the simple pleasures: savoring great food, enjoying the natural surroundings, and spending quality time with family without the usual distractions.

Coorg’s coffee was a genuine highlight. Rich with distinctive local character, it made our morning cups special. The regional spices added unexpected depth to every meal, while the fresh fruits offered sweet rewards after our relaxing walks through the plantations.

The resort provided just the right atmosphere—a place where we could truly disconnect and find that elusive sense of calm that seems to evaporate in our daily lives. The days unfolded gentler, measured in conversations and quiet moments rather than schedules and obligations.

Coorg delivers beautifully if you’re seeking a getaway where crowds are scarce, and nature takes center stage. It’s the kind of place where you can finally exhale completely and remember what relaxation actually feels like—something increasingly rare in our hyperconnected world.

Vietnam

Vietnam had always been on my bucket list, but we hadn’t planned to visit it anytime soon. However, as 2024 drew to a close, perhaps inspired by the spirit of “revenge travel” or a spur-of-the-moment decision, we decided to make it happen—just in time. Our hectic work and travel schedules in November and December had left little room for early planning, but the allure of Vietnam proved irresistible, and we finally set our sights on this incredible destination.

This destination was initially tagged as a “plan-it-yourself” trip, but the last-minute nature of our decision made us turn to a holiday agent—MakeMyTrip. They were highly responsive and efficiently planned the key attractions for us, ensuring we didn’t miss the important highlights.

We managed to get our e-VISA just in time, though we were already considering a Plan B. The process typically takes 5-8 days and requires specific documents: a scanned, colored copy of the passport’s front and back pages with at least six months of validity, a recent photo (35 x 45 mm) with a white background (no visible teeth or glasses), and a PDF of the round-trip tickets. While it was a scramble, the heavy lifting and follow-up were expertly handled by the MakeMyTrip VISA team, leaving us with nothing to do but worry and wait.

While there are direct flights available, our last-minute travel plans meant we had to reach Hanoi with a layover in Bangkok.

Day 1: Hanoi – Where the Old Meets the New

We stayed at the Hanoi Daewoo Hotel, situated at 360 Kim Ma Street in the Ba Dinh District. The hotel provided a peaceful retreat in the heart of the city. Although it isn’t located directly in Hanoi’s bustling Old Quarter—approximately 4 to 5 kilometers away in the Hoàn Kiếm District—it offers convenient access to key attractions. A quick Grab ride, typically 15 to 20 minutes, easily takes you to the Old Quarter. After a tiring overnight journey with a layover in Bangkok, we were eager to rest yet excited to start exploring Hanoi.

Lotte Center Skydeck

Since we couldn’t get an early check-in, we started our day by visiting the Lotte Center Skydeck, conveniently located near the Hanoi Daewoo Hotel. The Skydeck offers breathtaking panoramic views of the city and is a must-visit for its stunning scenery. The highlight is the Skywalk, a glass-floored section that provides a thrilling experience and a unique perspective of Hanoi. Additionally, there are coffee shops where you can enjoy a classic Vietnamese coffee (more on that later) and some VR games that entertain the kids and add to the fun for everyone.

Hanoi Train Street

After visiting the Skydeck, checking into our room, and enjoying lunch at the hotel, we headed to Hanoi’s Train Street (HTS). Online, we found out there was a train scheduled at 3:30 PM. HTS is a unique and thrilling attraction where a narrow residential street comes alive as a train passes just inches away from homes and cafes. Lined with colorful buildings and bustling with local life, it’s a fascinating place to watch the train squeeze through while sipping Vietnamese elixirs (coffee, beer) and munching Vietnamese food from one of the cozy cafes. This blend of daily life and adventure makes it a must-see in Hanoi.

9:00 PM Train @ Hanoi Train Street

Unfortunately, we missed the 3:30 PM train due to traffic, and by the time we arrived, it had already passed. Determined not to miss the experience, we tried again on another night, and the photo we captured was of the nighttime train. The atmosphere was electrifying as the train approached—the warning bell rang, shopkeepers hurriedly moved their chairs indoors, guards cleared people off the tracks, and some even placed beer bottle caps on the rails to flatten them as the train sped by. When the train finally roared past, the crowd erupted in cheers and applause, awestruck by the thrilling spectacle.

If you’re wondering whether to visit this place during the day or at night, I’d highly recommend going at night! The glowing lanterns and the lively, international noisy crowd make it a truly magical experience. Be sure to check the train schedule in advance or ask the café owners for timings. One café owner told us there was a train at 8:30 PM, but the train we ended up seeing arrived at 9:00 PM. Initially, we thought it was a ploy to keep us there longer to eat, drink, and spend more, but it turned out to be worth the wait!

Lotte Aquarium

After visiting Hanoi Train Street (HTS), we headed to the Lotte Aquarium, having purchased a combined ticket for both the Skydeck and the aquarium. Located within the Lotte Center near West Lake, just a 20-minute drive from the Daewoo Hotel, this modern aquarium is an excellent destination for families. It offers an immersive experience showcasing diverse marine life, from vibrant coral reefs to mesmerizing jellyfish displays. A highlight for my child was participating in pearl harvesting, where she watched a pearl oyster being opened to reveal a pearl, which could then be crafted into jewelry. She also enjoyed the hands-on experience of touching marine creatures like shellfish, crabs, and starfish. The aquarium fosters a magical connection with the underwater world, making it a fun and educational outing for all ages!

The Dinner Disaster

The food in Vietnamese malls and hotels is predominantly non-vegetarian, with beef and pork being common ingredients. Both my wife and I stick to seafood and chicken but avoid red meat, while my daughter eats eggs and is just starting to experiment with chicken—though she won’t go near seafood or red meat. This made finding suitable food in the malls quite challenging, as most dishes contained beef or pork. In the end, we settled for ice cream instead.

Communication in Vietnam posed another challenge, as English is not widely spoken or understood. We quickly adapted by using Google Translate to communicate, a skill we relied on heavily. The girls were eager to try the iconic Vietnamese Pho, but given the limitations, we returned to our hotel and ordered chicken and vegetarian Pho instead as part of the dinner buffet. 80% of the buffet was out of reach!

Day 2: Ha Long Bay – Nature’s masterpiece

The day-long trip to Ha Long Bay began early, with a 7:30 AM meeting time at a designated point (for us, it was the Hanoi Opera House). A car transported us from our hotel to the meeting point, and punctuality was key—everyone adhered to the schedule, with a call from the driver/guide, 10 minutes in advance to ensure we were ready. From the meeting point, buses or limousines carried passengers to Halong Bay. At each stop, tour guides stepped off the bus to locate their assigned travelers, calling out names, scanning faces, and matching passengers with their groups—a lively process that felt like a cheerful treasure hunt.

The journey to Halong Bay shared with strangers (an international crowd from Japan, Taiwan, Mainland China, Australia, France, India, Germany, South Africa, and Jamaica), made for a lively and engaging travel experience. The tour guides, brimming with humor and charm, kept us entertained with amusing anecdotes, intriguing historical tidbits, and a detailed overview of the day’s itinerary. Their lively storytelling added a unique touch to the ride, making it both informative and enjoyable. However, when the tour guide took a break from narrating, the bus quickly fell into a peaceful silence as most passengers drifted off to sleep.

Cruise & Lunch

Tuan Chau Harbor

After a 2.5-hour drive, with a couple of planned and emergency pit stops, we arrived at Tuan Chau Harbor to board the boat and began our excursion exploring the stunning beauty of this UNESCO World Heritage Site. Onboard, we enjoyed a set menu lunch featuring a variety of Halong’s special dishes. Passengers were seated based on their dietary preferences, which ranged from vegan and vegetarian to non-vegetarian with specific preferences like eggs, beef, pork, or seafood. The guide carefully arranged seating to prevent conflicts, especially avoiding solo travelers claiming the best spots—a surprising number of whom were on the trip.

The food was excellent, and my wife and I opted for the seafood menu, which was a highlight for us. However, my daughter didn’t enjoy the vegetarian options and decided to rely on chips and chocolate for the rest of the day. Even that proved tricky, as many snacks in Vietnam are flavored with pork, beef, or seafood, making her scrutinize every label carefully before digging in.

TiTop Island

After lunch, we headed up to the deck to enjoy the stunning views of the islands and capture some photos. Our next stop was TiTop Island, where visitors could choose to swim or hike to the top for the best view. We opted for the trek, eager to take in the panoramic scenery. Each limestone island in Halong Bay is uniquely named based on its appearance, such as Chicken Island, adding to the charm of the experience. The hike to the top was steep, with over 500 steps, but the mesmerizing view made every step worth it. At the summit, we treated ourselves to refreshing coconut water, snapped some memorable pictures, and soaked in the breathtaking landscape before heading back down to rejoin the cruise.

Luan Cave

Our first stop after the hike was Luon Cave, located on Bo Hon Island. The area surrounding Luon Cave is rich with well-known attractions in Halong Bay, making it a hub of natural beauty. Directly in front of the cave is Turtle Island, while Heaven Gate lies to its right, adding to the picturesque setting.

Visitors have several options to explore Luon Cave: paddling through its waters by kayak, gliding on a traditional bamboo boat, or opting for a speedboat tour to see nearby attractions. We chose the speedboat, and it turned out to be an exhilarating experience. The driver skillfully maneuvered the boat, creating an adrenaline rush as we sped across the water. Along the way, our guide pointed out fascinating landmarks like Dragon Cave, Stone carved Lotus, and Mermaid, each adding a touch of wonder to the journey. It was a thrilling and memorable way to experience the beauty of Halong Bay.

Sung Sot Cave

Our next destination was the famous Sung Sot Cave, or the “Surprising Cave,” located on Bo Hon Island. As we approached, our tour guide shared fascinating details about the cave’s history and its unique features, preparing us for the experience. However, we were greeted by a long line of eager tourists—proof of the cave’s popularity as one of Halong Bay’s must-see attractions. Sung Sot Cave is a natural marvel, consisting of three chambers: a small entry chamber, a middle chamber, and an expansive, grand chamber that lives up to its name. Spanning a 2-kilometer walk, the cave is adorned with stunning stalagmites and stalactites that have been shaped over millennia by dripping water and time. The formations take on fascinating shapes and textures, sparking the imagination as you stroll through the illuminated path. The grand chamber, the highlight of the cave, is particularly breathtaking, with its vast space and dramatic formations. It’s no wonder the cave is named “Surprising”—its sheer scale and beauty leave visitors awestruck. Despite the crowd, the experience was well worth it, as we explored one of the most beautiful caves in Halong Bay.

Sunset

The return to the harbor was serene, a perfect end to our day at Halong Bay. As the sun set, the golden light reflected on emerald waters, casting a warm glow over the limestone islands. A gentle breeze carried a sense of tranquility, washing away urban stress. Standing on the deck, surrounded by nature’s beauty, we felt a profound peace—a magical moment that makes Halong Bay unforgettable.

Day 3: Ninh Binh Tour

The day began even earlier than Day 2, as we prepared to meet a new group of strangers at a different meeting point. Our guide, Mr. Luca, was a bit of a newcomer to managing a large bus tour, having mostly handled smaller limousine groups before. The itinerary for the day included a 110 km drive filled with diverse activities planned by the tour team: a visit to a historic temple, a cycling adventure, a serene boat ride, and a trek through stunning landscapes.

Dinh King Temple

The first stop was a temple. The Dinh King Temple is located in the ancient capital of Hoa Lu and was built in honor of Emperor Dinh Tien Hoang, who founded the Dinh Dynasty in the 10th century. The temple reflects the spiritual depth of Vietnam’s early dynasties. Intricate carvings of dragons, phoenixes, and mythical creatures adorn the temple, symbolizing strength, prosperity, and protection.

Vietnamese culture has deep-rooted connections to kingship and spirituality, with temples serving as both places of worship and symbols of reverence for past rulers. The five elements—metal, wood, water, fire, and earth—play a significant role in traditional Vietnamese cosmology, representing harmony between humanity and the natural world. The Vietnamese people are believed to embody characteristics of these elements based on their birth year. These elements shape compatibility in relationships, especially marriage. For example, water nourishes wood, making individuals of these elements a harmonious match, while fire and water may clash.

Cycling and Lunch

Our second stop was for a short cycling tour followed by lunch. Cycling was a fun way to work up an appetite while exploring the land like traditional Vietnamese locals. However, the cycle was a bit too large for my daughter, so she decided to piggyback while I pedaled through the scenic route. She did try it when there was no traffic! Along the way, there were plenty of photographers (paparazzi) ready to capture the moment, making us feel like celebrities!

Exploring the countryside on a bicycle offered a unique perspective, reminiscent of our cycling adventure in the paddy fields of Bali. Though this tour was brief, lasting about 30 minutes, we spent a memorable time winding through peaceful villages. The activity got our metabolism going, and we rewarded ourselves with a delicious Vietnamese lunch. A highlight of the meal was the fresh spring rolls—thin rice paper wraps filled with vegetarian or other tasty fillings. Healthy, light, and full of flavor, they were a perfect treat after the ride.

Trang An Grottoes

The afternoon was spent on a tranquil ride in traditional sampan boats, their unique design adding to the charm of the experience. These flat-bottomed wooden boats are rowed by local women (always talking and making jokes) and can accommodate up to four passengers. The ride felt like being part of a school of fish, effortlessly flowing together in perfect rhythm through the water. The cool weather enhanced the magic as we glided through the Trang An Caves, a mesmerizing network of interconnected caves and waterways surrounded by dramatic limestone mountains. However, caution was necessary—low cave ceilings meant you could easily bump your head if not careful!

Along the way, we stopped at ancient sites like Cao Son Temple and Suoi Tien Temple, nestled amidst lush greenery. These stops not only offered a chance to explore the spiritual history of the area but also served as a welcome restroom break during the two-hour journey.

Trang An’s ethereal beauty has also earned it international recognition as a filming location for Kong: Skull Island. Its otherworldly landscapes bring to life the feeling of stepping onto a movie set, blending stunning natural scenery, rich history, and a touch of cinematic magic.

Ngoa Long (Lying Dragon) mountain

The Ngoa Long Mountain, located in the Mua Cave area, offers one of the most stunning panoramic views in Ninh Binh. A popular trekking spot, it’s often said, “A trek a day keeps stress away!” Our guide cheerfully estimated 20 minutes to climb up, 20 minutes to soak in the views, and 20 minutes to descend—but the reality is never quite so balanced. The climb is steep, with two routes: a steeper left side boasting spectacular vistas and a slightly easier right side. We opted for the more challenging left route, navigating short, steep steps that tested our endurance.

Before tackling the ascent, we took a leisurely walk around the tranquil lotus ponds, capturing some beautiful photos. The climb was rewarding, and at the summit, the breathtaking view of rice fields, limestone karsts, and the surrounding landscape made every effort worthwhile. Ngoa Long Mountain is truly a trekker’s delight and a photographer’s paradise.

Day 4: Da Nang

We had an early morning flight to Da Nang (Flight VJ 503: Hanoi [HAN] to Da Nang [DAD], departing at 7:20 AM), which meant a hectic night of packing and an early scramble to the airport. Adding to the chaos, we had squeezed in a night visit to Hanoi Train Street—an experience we didn’t want to miss.

Sleep was scarce, coming in short bursts during the car ride, at the airport, and on the flight. Still, those quick naps were enough to recharge us for the day ahead. Our walking pace had shifted to “turtle mode,” and every step was a not-so-gentle reminder that our muscles were staging a protest after the Day 3 climb!

Once in Da Nang, we checked into Holiday Beach Resort, located on a beautiful stretch of My Khe beach. With no specific plans, we started the day with breakfast and a relaxing stroll along the shore. The waves were perfect for surfing, and we spent time watching skilled surfers and beginners alike tackling the high swells. We were not keen on jumping into the cold waters of the south china sea.

While soaking in the peaceful beach atmosphere, we spoke with our tour guide, who enthusiastically suggested visiting Hoi An. Her advice? “Just grab a taxi and go!” With no set plans, it felt like the perfect spontaneous next step.

Hoi An: Coconut Village

We visited the Hoi An Coconut Village and had an amazing time exploring its lush coconut groves. Riding in traditional round basket boats, expertly steered by local rowers (mostly men), we navigated through a maze of winding waterways. The guides entertained us with impressive boat tricks and even fished out crabs, adding to the excitement. The peaceful setting of the groves was beautifully complemented by the lively music and cheerful atmosphere. To make the experience even more special, we had a fun photo shoot, creating lasting memories of this vibrant slice of Vietnamese culture.

Hoi An: Old Town

After visiting the Coconut Village, we hopped into another taxi and headed to Hoi An Old Town. The charming streets welcomed us with their iconic yellow buildings and bustling riverside vibe. Since most shops in Hoi An only accept cash, our first stop was an ATM before we began exploring.

We wandered through the streets, passing boatkeepers along the river and inviting us for the 10:00 PM lantern-lighting rides. Along the way, we stopped for ice cream and enjoyed a delicious Vietnamese lunch at Miss Ly Café. The town seemed to understand Indian travelers well—the kind waitress went out of her way to offer true vegetarian dishes alongside seafood and chicken options, which we really appreciated.

Of course, we couldn’t resist picking up a few souvenirs, but the highlight for the kids (and us!) was the smoky ice cream, which felt like a show in itself. We also tried some street fruits—mango and jackfruit—despite our kids’ humorous warnings not to. To end the evening, we watched a lively dance performance by the riverside, soaking in the cultural charm of Hoi An. It was truly a day to remember!

Smoking ice cream in Hoi An comes surrounded by liquid nitrogen vapor, creating a dramatic “smoking” effect as you eat it. The cool, foggy spectacle makes it feel like more than just dessert—it’s an experience! It’s a must-try for anyone looking to add a little flair to their culinary adventures in Hoi An.

Day 5: Ba Na Hills

It was the last day of the year, and we started early, joining a group of fellow travelers. This group was primarily “Indian,” but not everyone was from India. In Da Nang, the convenience of buses picking you up directly from your hotel was a nice touch. The ride to Ba Na Hills was scenic, and soon we were on the cable car, ascending through misty clouds with breathtaking views of the Mo Stream below.

The English of our Vietnamese tour guide took some effort to understand, but with patience on both sides, we managed just fine. Like school kids, we were assigned group numbers and had to respond to roll calls, adding a touch of humor to the experience.

Ba Na Hills, originally developed in 1919 as a French hill station, has since transformed into a vibrant tourist destination. It felt like a mix of Disneyland and Ocean Park Hong Kong. Nestled atop the Trường Sơn Mountains, it offered a fascinating blend of natural beauty and man-made attractions. The cloudy, cold, and rainy weather added an ethereal charm, making it feel as if we were walking among the clouds. Thankfully, we had packed umbrellas to stay prepared.

Our adventure began with the iconic Golden Bridge, held aloft by the massive “Hands of God.” We arrived early enough to snap some photos before the crowds descended. Getting a clear shot, however, was nearly impossible with so many visitors vying for the same goal. Professional photographers stationed there had software to remove other people from your photos, but their services came at a steep price.

Expertly guided by our tour leader, we explored a variety of attractions, from the charming French Village to the peaceful 27-meter Buddha statue and the vibrant Flower Garden. Each stop felt like stepping into a different world, creating a truly diverse and exciting experience. We even took a tram ride to watch a 7D film, which transported us to fascinating places around the globe with its immersive visuals.

By lunchtime, my daughter was craving the comfort of a simple Indian meal. To help us beat the cold, we were treated to complimentary beers (truly elixirs from heaven). Our guide directed us to an Indian restaurant offering a buffet, which included plain dal and rice—just what my daughter wanted—alongside some pasta for variety.

After lunch, although we had some time left to explore Fantasy Park, the biting cold and windy weather (despite a forecast of 12°C, it felt much colder due to the rain and wind) forced us to seek warmth. We found ourselves, like many other families, huddled around a heater to warm our hands.

Ultimately, we decided to skip both the Fantasy Park and Linh Chua Linh Tu Temple. Instead, we opted to descend via the cable car and returned to the hotel by 5:30 PM, choosing comfort over braving the harsh weather in Ba Na Hills.

Happy New Year, 2025!

As the evening approached, we hoped the rain wouldn’t dampen our New Year’s plans—and luckily, it didn’t! We strolled around Da Nang, indulging in more Indian food for my daughter, sipping on unique Vietnamese coffees, getting a foot massage, and savoring fresh seafood, including a large crab. It was a perfect way to close out the year, blending familiar comforts with local flavors and experiences.

The streets were buzzing with life, packed with cafés serving every cuisine you can imagine, massage spas ranging from cozy to extravagant, and shopping marts eager to satisfy every whim of the visiting crowds. Every other corner seemed to have an Indian restaurant blaring ’90s Bollywood hits, adding a nostalgic desi touch to the lively atmosphere.

Big names like Sheraton and Marriott are racing to stake their claim along the beach, hinting at an even glitzier future for Da Nang. Meanwhile, the present was alive with loud party music echoing from various hotspots, each vying to pull in crowds for the midnight countdown.

Unfortunately, the famous 45-minute Da Nang fireworks show was canceled this year for environmental reasons, leaving some disappointed. But not all was lost—many families opted for a quieter celebration, strolling along the beach where small shops lit up the night with minor fireworks to keep the festive spirit alive. It was a vibrant mix of noise, nostalgia, and seaside serenity—a fitting end to the year!

We strolled back to our room, quietly responded to several New Year messages, and enjoyed a much-needed nap to prepare for another long (but last) day of travel.

Day 6: Hue

We reluctantly dragged ourselves out of bed for an early drive to Hue City. Hue is pronounced “h-way,” with an emphasis on the “h” sound at the start, followed by a drawn-out “u,” similar to saying “way.”

Once again, we were picked up and found ourselves among a group of sleepy international travelers, exchanging cheerful New Year wishes with everyone. Hue is renowned for its rich history and cultural significance, making it a must-visit destination.

The journey itself was an experience, taking us through the Hai Van Tunnel, the longest and most modern tunnel in Southeast Asia. This marvel of engineering not only made the drive smoother but also added a touch of awe to the trip.

Hue’s dialect is often considered more elegant and poetic compared to other regions in Vietnam. It’s said that the people of Hue speak with a soft, lilting tone that reflects the city’s historical ties to Vietnam’s royal past.

Lap An Lagoon

We made our first stop at Lap An Lagoon, located west of Lang Co Bay. This picturesque lagoon is surrounded by mountains and is home to several pearl farms. We spent some time taking in the serene views, sipping salt coffee, and watching local pearl farmers separating pearl oysters before continuing our journey.

Khai Dinh Tomb

Upon arriving in Hue, our first visit was to the Khai Dinh Tomb, one of the most beautiful and intricate royal tombs of the Nguyen Dynasty. The stunning architecture combines traditional Vietnamese and European elements, making it a truly unique historical site. The detailed carvings and mosaics were a testament to the grandeur of the Nguyen kings.

Many of the intricate carvings and mosaics in Khai Dinh Tomb were reportedly created by artists using their feet—a technique viewed as a bad omen in Vietnamese culture. There are two fascinating stories behind this unusual technique. One suggests that the artists used their feet as an act of protest against Emperor Khai Dinh, who raised taxes to fund the construction of the tomb. Another version claims the method was purely practical—ceiling paintings and carvings were done using feet so the artists could lie back and see the artwork as they worked, ensuring better precision and perspective.

Towering at the top of 127 steps, the tomb exudes grandeur. The structure is covered in blackened concrete, which gives it an imposing presence against the lush green backdrop of the mountains. The interior, however, bursts into vibrant life with colorful glass and porcelain mosaics adorning the walls. The most striking feature is the statue of Emperor Khai Dinh, cast in bronze and seated beneath a celestial ceiling painting that depicts a lively dragon amidst clouds.

Thien Mu Pagoda

Post-lunch (at a local restaurant), we visited the Thien Mu Pagoda, Hue’s oldest and most beautiful pagoda. This iconic site, perched on a hill overlooking the Perfume River, is a symbol of Hue’s spiritual heritage but also a serene retreat that offered a peaceful pause in our journey.

The name “Thien” means “heavenly” or “celestial,” while “Mu” translates to “lady,” together symbolizing a divine feminine spirit. According to legend, an old woman appeared on the hill where the pagoda now stands, prophesying that a great leader would build a temple here to bring peace and prosperity to the region. Inspired by this, Lord Nguyen Hoang ordered the construction of the pagoda in 1601, marking the beginning of its storied history.

The pagoda is also tied to a tragic love story in local folklore. Legend has it that couples who visit the pagoda together—especially those holding hands—might face separation afterward. While this remains a superstition, it adds an intriguing and mystical element to the otherwise serene and spiritual site. As our tour guide jokingly remarked, “If you’re looking for separation, just hold hands here!”

The Phuoc Duyen Tower, a seven-story pagoda, stands as an iconic symbol of Hue here. Each of its seven stories represents a step on the Buddha’s path to spiritual awakening. This concept is tied to the legend of Buddha’s seven steps at birth, where each step was marked by a blooming lotus flower, symbolizing purity and enlightenment.

There were many bonsai trees in the gardens here. Miniature trees, commonly known as bonsai, are referred to as cây cảnh or “ornamental trees” in Vietnamese culture. These living art forms are meticulously shaped to balance natural beauty and artistic vision, showcasing the grower’s skill and patience. Bonsai symbolizes harmony, balance, and interconnectedness, often representing nature’s grandeur in a small form. For many, caring for bonsai is a meditative practice fostering mindfulness and inner peace. In Vietnamese and East Asian traditions, they are associated with Feng Shui, believed to attract positive energy, prosperity, and good fortune. Historically, bonsai also signified wealth and status, often adorning the homes of nobles and scholars. Whether for artistic, spiritual, or cultural reasons, bonsai remains cherished worldwide.

Hue Citadel

Our last destination was the Hue Citadel, a UNESCO World Cultural Heritage site since 1993. This massive complex, with its walls, palaces, and gates, holds the legacy of the 13 Nguyen dynasty kings who ruled for over 140 years. Despite enduring the ravages of time and war, the Citadel continues to preserve its historical significance and charm.

Approximately 40% of Vietnamese people have the surname Nguyen, making it by far the most common surname in Vietnam. The prevalence of this surname is linked to the Nguyen Dynasty, the last imperial dynasty of Vietnam, which ruled from 1802 to 1945. During this time, many people adopted the surname as a sign of loyalty or to align themselves with the ruling class.

The undisputed highlight of the day? Our royal visit to the royal toilet! Who knew history could be so flush with grandeur?

After a two-hour walk exploring Hue’s rich history and culture, exhaustion caught up with us, and we quickly fell asleep on the bus ride back to Da Nang. The driver smoothly navigated the route, and we arrived at our hotel by 6:30 PM, marking the official end of our holiday adventures.

Last Evening

To celebrate our final evening in Vietnam, we indulged in a feast of fresh seafood by the beach, including a mouthwatering lobster, and savored one last round of aromatic Vietnamese coffee—a fitting farewell to the culinary delights of this vibrant country.

Special Mention: Vietnamese Coffee

Vietnamese coffee culture is incredibly diverse, with a wide variety of options to explore. Popular choices include Cà Phê Sữa Đá (iced coffee with milk), Cà Phê Đen (black coffee), Cà Phê Trứng (egg coffee), Cà Phê Dừa (coconut coffee), Bạc Xỉu, and the luxurious Cà Phê Chồn (weasel coffee). Another unique offering is Avocado Coffee, a creative blend of flavors. Each type reflects the rich coffee heritage of Vietnam and offers something for every palate. You can have coffee hot or iced!

Done Done

If you’ve reached here, congratulations! This travelogue was penned down immediately after our trip, before the busy workdays started rolling in and the memories began to blur.

One thing that still makes us laugh is how I kept calling “Da Nang”Ha Dang”, completely forgetting the real name. The names of the places we visited—Ha Noi, Hoi An, Ninh Binh, Ha Long, Da Nang, Ba Na, Hue, Sung Sot, Bo Hon, Luon, Trang An, Cao Son, Suoi Tien, Ngoa Long, Thien Mu, and Lap An—felt like tongue twisters at times. I couldn’t help but mix up Da Nang with Ha Dang, much to everyone’s amusement! My daughter made sure I was properly “punished” for my slip-up with endless giggles. We googled Ha Dang, and found that to be a real person’s name!

I suggest you put Vietnam on your bucket list! Cheers

Embracing Uncertainty

Picture a bamboo grove in a storm. While mighty oaks crack and fall, bamboo stems bow to the ground yet spring back unbroken. This simple truth about flexibility defeating rigidity lies at the heart of handling uncertainty. Today’s world spins faster than ever – markets transform overnight, technologies reshape entire industries, and a single event can send ripples across the global economy.

Ancient Wisdom

“The mind is very restless, turbulent, strong and obstinate, O Krishna. It appears to me that it is more difficult to control than the wind.” – Arjuna 

When facing uncertainty, ancient texts offer surprising clarity. The Bhagavad Gita speaks of focusing on actions rather than outcomes – karma (choices, actions, causation) over karmaphala (fruit, outcomes). This isn’t just philosophy; it’s practical advice for today’s leaders. When you focus solely on results, fear can paralyze you. When you focus on taking the right actions, clarity emerges.

The Dance of Adaptation

Leadership is a dance with uncertainty. Sometimes you lead, sometimes you follow, but always you must stay in motion. The ancient concept of dharma teaches us to act with integrity without clinging to specific outcomes.

“In a gentle way, you can shake the world.” – Mahatma Gandhi

Like bamboo, successful organizations build resilience through flexibility. They maintain strong roots – their core values and principles – while adapting their strategies to changing winds. This balance between steadfastness and adaptability creates lasting success.

Seeing through Maya’s Veil

In Sanskrit philosophy, Maya doesn’t simply mean illusion (a reality distortion field). Think of it like watching shadows on a cave wall – what we see is real, but it’s not the complete truth. In leadership, this concept is surprisingly practical.

The Upanishads teach about “neti neti” – Sanskrit for “neither this, nor that” – a powerful method for finding truth by eliminating what isn’t true. Instead of immediately grasping at solutions when facing uncertainty, we can systematically eliminate what we know isn’t working or true.

“When you have eliminated the impossible, whatever remains, however improbable, must be the truth.” – Sherlock Holmes

Today’s leaders can practice this by asking deeper questions (why, when, what, which, where, who): What assumptions are we making? What aspects of this situation am I misleading? Which assumptions need challenging? What alternatives haven’t we considered? What possibilities lie beyond our current view? etc.

When we stop clinging to our first interpretations and systematically examine our assumptions, new opportunities often emerge from unexpected directions.

“The important thing is not to stop questioning. Curiosity has its own reason for existence” – Albert Einstein

The Empty Cup

An old Zen story tells of a master pouring tea into a student’s already full cup until it overflows. The lesson? We cannot receive new insights when our minds are already full of preconceptions. In times of uncertainty, we must first empty our cups – let go of what we think we know to discover what might be.

Final Thoughts

Uncertainty isn’t the leader’s enemy – it’s the space where innovation happens. Like bamboo in the storm, true strength lies not in standing rigid against change, but in dancing with it. By embracing uncertainty while staying true to our principles, we transform challenges into opportunities for growth.

Leaders who thrive in uncertainty don’t try to control the uncontrollable. Instead, they build resilient organizations that bend but don’t break, adapt but don’t compromise their values, and turn change into opportunity. This way of leading isn’t just about survival – it’s about finding grace in chaos and strength in flexibility.

“In the middle of difficulty lies opportunity.” – Albert Einstein

WHY THIS BLOG?

I was talking to leaders taking their year-end breaks, and I sensed the weight of uncertainty (organization changes, inability to make lasting changes, frequent pivots, and idea conflicts) wearing them down. Hence, this blog.

While I don’t have a perfect solution, some things have helped me in my journey: Taking action instead of freezing. Staying true to your values. Being flexible in your plans but firm in your integrity. Trusting your team. Listening and Respecting Ideas. Keeping your mind healthy – not letting it trap you in victim thinking and impossible scenarios. This last one (keeping the mind healthy) is/was difficult for me to practice sometimes, but being in these cycles multiple times is helping me/has helped me.

Leadership isn’t about having answers to everything. It’s about staying grounded while keeping flexible.

Generative AI in Healthcare

Today’s popular ChatBots have evolved to the point where they can ‘search’ the internet to build a profile of a person. While the idea of profiling might seem unsettling, what’s even more fascinating is their ability to dynamically integrate with external tools. This means these bots don’t just acquire context—they actively use it to take meaningful actions. And that’s a game-changer. In industries like healthcare, where inaction and delays can have serious consequences, this capability to bridge information and execution is nothing short of revolutionary.

Insights Are Necessary But Not Sufficient

We can design a world-class computer vision algorithm to detect potholes on roads—no doubt about that. The real value of such technology lies in its ability to prevent small potholes from escalating into traffic nightmares. But here’s the catch: insight without action is as good as scrap metal. Knowing there’s a pothole on Road A is useful, but if it’s not fixed, that insight is wasted. This is where Generative AI steps in, not just to detect problems but to close the loop by orchestrating solutions. Imagine a team of AI agents—a detector, a work approver, a fixer, and even a finance approver—all collaborating to decide which potholes get priority. Given limited budgets and resources, the challenge becomes an optimization problem: which roads to repair to maximize safety, improve traffic flow, and minimize frustration. With high volumes of insights, this kind of collaborative decision-making can cut through the ‘insights fatigue’ and turn knowledge into action.

Healthcare faces a similar dilemma. There’s a heavy emphasis on monitoring—producing insights that are supposed to drive action—but the gap between knowing and doing often remains wide. Take the case of a diabetic patient struggling to lower their HbA1c from 8.5. The problem might not be awareness but action: lifestyle changes like regular walks, strength training, better diet choices, or even adhering to medication schedules. The truth is, most people know what to do; the challenge lies in execution.

This is where a hyper-personalized approach becomes critical—like having a health coach who not only reminds you to do your sit-ups but makes sure you squeeze in 10 while you’re still on the call. Habits, once formed, become second nature (think brushing your teeth). But doctors don’t have the bandwidth to coach every patient. Enter Generative AI, which can act as a virtual co-pilot for healthcare professionals. Imagine a digital assistant that mirrors a doctor’s conversational style while incorporating a coach’s motivational touch. This AI can identify when a patient is straying off course, focus discussions on actionable lifestyle changes, and tackle one problem at a time. If medication compliance isn’t the issue, it can hone in on diet or exercise—whatever the patient needs most. Generative AI brings the promise to move insights to actions.

Generative AI Use Case Sampler

Generative AI has the potential to revolutionize healthcare by enabling smarter, more personalized, and action-oriented solutions. By leveraging diverse modalities like text, images, audio, and video, this technology can bridge gaps in patient care, education, and operational efficiency. Here are some potential use cases:

Text-to-Text: Personalized Discharge Summaries
Generative AI can generate hyper-personalized discharge summaries tailored to individual patients. By pulling data from EMRs and provider-recommended templates, these summaries can be presented in clear, actionable language, preferably in the patient’s preferred language. From follow-up instructions to medication schedules, this empowers patients to take control of their recovery with confidence.

Image-to-Text: Radiology Reports Made Smarter
Deep learning has already enabled precise interpretation of radiology images. Generative AI can take this further by generating highly personalized reports for different audiences, such as PCPs, patients, or specialists. By reducing radiologists’ workload and improving turnaround times, it can ensure quicker and more efficient delivery of critical insights. Moreover, Generative AI can allow radiologists to query images for specific details or compare similar images by vectorizing (creating embeddings) the image and reports.

Text-to-Image: Visualizing Patient Education
Generative AI can help patients better understand their diagnoses and treatment options by creating personalized visualizations. A picture truly is worth a thousand words, especially in healthcare, where complex concepts can be challenging to convey through text or speech alone.

Image-to-Image: AI-Powered Image Reconstruction
Generative AI can enhance the quality of medical images through advanced reconstruction techniques, improving both speed and resolution. This capability can boost diagnostic accuracy and provide healthcare professionals with the clarity needed for more informed decisions.

Text-to-Speech: Accessibility for All
For patients with disabilities or language barriers, text-to-speech technology can make medical information more accessible. Whether through audio outputs or Braille conversion, Generative AI ensures inclusivity in patient communication and care delivery.

Audio-to-Text: Seamless Medical Transcription
Generative AI can convert conversations—between doctors and patients or among clinicians—into structured medical records. This technology streamlines documentation, allowing healthcare professionals to focus on patient care while simultaneously generating structured, actionable records.

Text-to-Video: Transforming Medical Education
Generative AI can transform training and education by generating personalized, easy-to-follow video content. From simplifying complex medical topics to creating interactive learning experiences, it offers a more engaging way to educate both patients and healthcare professionals.

Text-2-Text: Generative AI

A Text-2-Text generator can be used to build a story to recite to a child to help her develop better study habits. With a simple prompt, Generative AI (courtesy: ChatGPT) can craft a compelling story—like one inspired by Harry Potter—that captures the child’s imagination while embedding useful lessons. For example, a magical artifact like a “Focus Fuser” might serve as the centerpiece of the story, teaching readers how small, consistent actions can lead to meaningful progress.

Now shift to a healthcare setting. Medical jargon, while essential for precision, often confuses patients. Generative AI can take complex radiology reports and translate them into clear, patient-friendly language. For example, instead of saying, “Impression: The findings are consistent with a moderate right-sided pleural effusion,” the AI might explain: “The results of your imaging test show a moderate amount of fluid in the space around your right lung, known as a pleural effusion. There’s no sign of pneumonia or a collapsed lung. Your doctor may recommend further tests to get more clarity if needed.”

This translation doesn’t just simplify language—it empowers patients to understand their health better, enabling them to make informed decisions.

Text-2-Image: Generative AI

A Text-2-Image Generator can recreate familiar concepts with stunning precision and creativity. Take the classic representation of the “Three Wise Monkeys”—each embodying the principles of “see no evil, hear no evil, speak no evil.” With a simple prompt, AI can generate a vivid image of three monkeys, each covering their eyes, ears, or mouth, perfectly capturing the symbolic meaning behind the concept.

This same technology demonstrates its utility in healthcare through advanced imaging capabilities. For instance, it can generate a synthetic frontal chest X-ray (CXR) image of a patient with specific features, such as simple pleural effusion and focal airspace opacity. This isn’t just an exercise in realism—it supports medical education, research, and even diagnostic validation. The ability to generate medically accurate images empowers healthcare professionals and researchers to innovate and enhance patient care. It also supports the cause of patient privacy.

Image-2-Text: Generative AI

The image-to-text generator capability is a game-changer for accessibility, enriching experiences for visually impaired individuals and enabling deeper engagement with visual content through natural language. Auto-generation of key features from an image can enable search (E.g., get me pictures with a blue lake and snow-capped mountains).

On the other hand, in a healthcare radiology context, Generative AI can generate detailed radiology reports (E.g. CXR 2 views – AP/LAT) such as: “The heart size and mediastinal contours are normal. There is no evidence of focal airspace consolidation, pleural effusion, or pneumothorax. No acute bony abnormalities are observed, although degenerative changes are present in the thoracic spine.”. This not only reduces the workload on radiologists but also enhances accuracy and efficiency in delivering diagnostic insights. By automating the creation of such reports, AI enables healthcare professionals to focus more on patient care while ensuring timely and precise interpretations.

Reasoning is Complex

(*) The above reasoning is not to show any tool or technology in a Bad light.

Reasoning can be a tricky beast, as you’ve probably noticed by now. While a human might have gone for answers like, ‘Maybe the male prisoner was released!’ or ‘Plot twist—he escaped!’, AI takes a different approach. It sticks to the context you give it, like a dog chasing a stick. But here’s the catch: if you want AI to keep doing its mind-blowing, jaw-dropping, ‘WOW’ job, you’ve got to toss it the right stick—clear context and solid rules. Otherwise, you might end up with answers as wild as a soap opera plot!

Closing Quote

‘The machine does not think, but it reveals our thinking.’

Generative AI quality is proportional to the quality of the information and context we give it—whether it’s clear or confusing. This means the responsibility isn’t just on the AI (or its builders) but also on us (the users) to ensure it is used in a way that reflects important values. When used wisely, it can help us improve care, build stronger connections, and create a fairer world for everyone.

2024: Emerging Technology Trends

This blog is my attempt to make sense of the transformative technology trends shaping 2024, organizing them into a structure that helps me—and hopefully you—grasp their impact. From sweeping macro shifts to granular micro innovations, I’ve distilled my observations and reflections into this post to explore how these trends are reshaping our world. My goal is to spark ideas and inspire curiosity as we navigate the ever-evolving frontier of technology.

From Star Trek Dreams to Today’s Realities

As a lifelong fan of Star Trek, I often find myself marveling at how much of its futuristic vision has seamlessly blended into our everyday lives. What was once the realm of science fiction is now science fact, and it’s astonishing to see how the imaginative worlds of Gene Roddenberry have inspired generations of innovators. Let me take you on a journey through some of these Star Trek dreams and their counterparts in today’s technology:

  1. Communicators → Smartphones: Captain Kirk’s communicator is today’s smartphone, letting us call, navigate, shop, and even control our homes.
  2. PADDs → Tablets: Starfleet’s sleek devices are now tablets like iPads and Kindles, offering portable, powerful access to information.
  3. Universal Translators → Google Translate: Star Trek’s universal translator lives on in Google Translate, breaking language barriers in real-time.
  4. Tricorders → Portable Medical Scanners: Dr. McCoy’s tricorder inspired tools like GE’s VScan, revolutionizing portable medical diagnostics.
  5. Replicators → 3D Printing: Captain Picard’s replicator echoes in 3D printing, creating tools, prosthetics, and more layer by layer.
  6. Holodeck → Virtual and Augmented Reality: The Holodeck is here with VR and AR, immersing us in gaming, collaboration, and virtual experiences.
  7. Voice-Activated Computers → Siri and Alexa: Starfleet’s voice-activated computers are now Siri, Alexa, and Google Assistant, responding to commands daily.
  8. Data → AI like ChatGPT: The android Data foresaw today’s AI, like ChatGPT (+Robotics), transforming creativity, problem-solving, and workflows.
  9. Memory Banks → Cloud Computing and Big Data: Starfleet’s vast knowledge banks mirror today’s cloud computing and data lakes, offering limitless storage.

Star Trek’s genius lies not just in its speculative tech but in how it inspires us to push the boundaries of the possible. We may not have cloaking devices, or the Prime Directive fully figured out yet, but the trajectory is clear—we’re steadily turning sci-fi into sci-reality. The question is no longer if but when.

A Quote from a modern-day philosopher and visionary

“Imagine a digital product as the spirit animal of a human, guiding and empowering her in her journey. The question isn’t just how we maintain these companions, but how we nurture them to thrive alongside us in the pursuit of seamlessness.”

— CHATGPT (Prompt Engineered by Nitin Mallya)

Every post needs a thought-provoking quote, right? But in the age of AI, why not ask ChatGPT, our modern-day philosopher? A little prompt engineering led to a quote that resonated with a Star Trek twist.

Remember Chakotay from Star Trek: Voyager and his guiding spirit animal? It was more than mystical—it was a companion, helping him navigate challenges. Today, AI is becoming just that: a copilot, not just a tool, but a partner empowering us to explore and solve problems.

Another Quote

There’s a quote I love: “A product/service that gets better after we sell it.” I’m not sure who first said it, but it perfectly captures the magic of today’s connected technologies. It’s not just about creating products; it’s about creating experiences that evolve and improve over time.

For a salesperson, this quote highlights the thrill of the sell. For a product manager or engineer, it’s all about delivering the better. But the real brilliance lies in the platform/ecosystem—the invisible engine that keeps adding value long after the sale is made.

In the world of hardware, this might mean something tangible, like adding a dashcam to your car. But in software, the possibilities are limitless. Think of apps that refine workflows, tools that make complex tasks effortless, or AI systems that transform product usage. The game has shifted; it’s about enhancing every moment of the user’s experience over the product’s entire lifecycle.

Quantum Digitalization: The Journey to Hyper-Personalization

The trajectory of technology often feels like a dance between the micro and the macro, the tangible and the intangible. I call this journey Quantum Digitalization—a fusion of precision and expansiveness that is reshaping how we interact, innovate, and solve problems. It’s not just a trend; it’s a profound transformation unfolding in stages, each one moving us closer to a world where hyper-personalization isn’t merely possible—it’s inevitable.

I found inspiration in the AI paper, “Attention is All You Need,” and I’ve borrowed its clarity to describe the stages of this evolution. Each stage has a theme and a guiding principle that frames the technology and its impact.

Digitize: Data Is All You Need

The first step in this journey is to Digitize, or as we might say in India, Digify. This stage is about making sense of chaos—bringing structure to the unstructured and transforming information into a usable form. It’s the shift from paper to pixels, where workflows are streamlined, knowledge is centralized, and insights become attainable. However, the magic of digitization isn’t in collecting more data; it’s in collecting the right data. Too much can lead to a data swamp—overwhelming and unusable. Too little creates gaps that even the smartest algorithms cannot bridge. This stage is the foundation—the roots that support the tree of innovation. Without a strong base, the rest of the journey is shaky at best.

Digitalize: Experience Is All You Need

Once the data is in place, the next step is to Digitalize. This stage is often misunderstood. It’s not about Digitizing; it’s about creating meaningful, engaging experiences. Technology shifts here from being a tool to becoming a partner, enhancing how people interact with systems and processes. This stage is about connection—devices talking to each other, systems understanding users, and interactions that feel intuitive. Imagine seamless experiences, where diagnostics, decisions, and delivery happen almost invisibly. It’s no longer just about solving problems; it’s about delighting users. Unfortunately, many still confuse digitalization with digitization or equate it solely with software. But digitalization is much more—it’s about creating ecosystems where experience becomes the driver of adoption and innovation.

Qubitize: Accuracy is all you need

As the journey advances, we step into the stage of Qubitize, where the guiding theme is Accuracy is all you need. This is the realm of quantum technologies, which bring a level of precision, speed, and security that traditional systems simply cannot match. It’s here that the boundaries of what is possible begin to dissolve, unlocking a future once confined to science fiction.

At the heart of Qubitize are three transformative domains: quantum sensing, quantum computing, and quantum communications. Each represents a leap forward, reshaping how we measure, compute, and interact in ways that redefine accuracy.

Quantum Sensing is revolutionizing how we perceive the physical world. Traditional sensors—no matter how advanced—have limitations in precision and sensitivity. Quantum sensing taps into phenomena like quantum superposition and entanglement to deliver ultra-precise measurements of time, magnetic fields, gravity, and more. For example, quantum-enabled imaging devices could detect changes at the molecular level, opening doors to breakthroughs in fields like healthcare, geophysics, and environmental monitoring. Imagine medical scans with resolutions so high that they can detect diseases at their earliest stages or environmental sensors that can map underground resources with pinpoint accuracy. This level of sensing not only enhances accuracy but also expands what we can observe and understand.

Quantum Computing takes problem-solving to a whole new dimension. Unlike classical computing, where bits are either 0 or 1, quantum bits—or qubits—exist in multiple states simultaneously, enabling computations at unprecedented speeds. This allows quantum computers to solve complex problems, like optimization or molecular modeling, that would take classical systems centuries to process. For instance, quantum algorithms can revolutionize logistics by finding the most efficient routes in real time, or they can accelerate drug discovery by simulating molecular interactions at a scale and speed previously unimaginable. Quantum computing isn’t just faster; it’s smarter, enabling us to approach problems in ways that were previously inconceivable.

Quantum Communication offers a paradigm shift in data security and transfer. At its core is Quantum Key Distribution (QKD), which uses the principles of quantum mechanics to create encryption keys that are theoretically unbreakable. By leveraging entangled particles, QKD ensures that any attempt to intercept or tamper with the transmission is immediately detectable, making it a cornerstone for ultra-secure communications. Beyond security, quantum communication networks are laying the groundwork for a quantum internet—an interconnected web of quantum devices that could transform how information is shared globally. This leap will redefine the concept of trust in digital systems, safeguarding sensitive information in ways that traditional cryptographic methods cannot match.

Qigitalize: Seamlessness Is All You Need

Finally, we arrive at Qigitalize—the frontier where the physical and digital worlds blur into one seamless continuum. This stage is about fluidity, where boundaries dissolve, and technology adapts to us rather than the other way around. Imagine ecosystems where robots autonomously handle tasks, collaborating with humans only when necessary. Picture a world where virtual and real merge effortlessly, enabling interactions that are frictionless and immersive. Collaboration transcends barriers, and hyper-personalization becomes a given. It’s a vision of a world that feels intuitive, connected, and limitless.

AI: A Microtrend Shaping the Future

AI is in the midst of an explosion—a rapid evolution transforming how we build, use, and interact with intelligent systems. Let’s dive into how AI is expanding its influence in four transformative dimensions: learning styles, tasks, data modalities, and system designs.

Learning Styles: How AI Masters the World

AI’s ability to learn has evolved dramatically. It began with supervised learning, where labeled data taught machines to distinguish between cats and dogs. While groundbreaking, this method relied heavily on expensive, annotated datasets. Then came unsupervised learning, revealing patterns in unlabeled data, like customer segmentation. Semi-supervised learning bridged the gap, blending smaller labeled datasets with large unlabeled ones for efficiency. Reinforcement learning marked another leap, enabling AI to learn through trial and error by interacting with environments—think Chess or Go. But the game-changer is self-supervised learning, where AI generates its own labels from data structures, powering massive models like GPT. Combine that with transfer learning, which reuses pre-trained models for new tasks, and federated learning, which keeps data private while training across decentralized systems, and you see AI’s growing adaptability. These advances are turning AI into an increasingly self-reliant, scalable force.

Learning Tasks: From Assistants to Creators

AI has transcended its origins as an assistant. Initially focused on classification (e.g., “Is this a cat?”) and regression (e.g., “What will sales be next quarter?”), AI has now become a creator. Generative AI systems can write essays, design artwork, or even produce videos from simple prompts. This creative power is revolutionizing industries, from marketing and entertainment to education and healthcare. But generative AI is just the beginning. Multimodal capabilities allow AI to integrate text, images, and audio seamlessly. Imagine an AI that can describe a picture in words, generate a matching soundscape, or create immersive VR experiences—all from a single prompt. This evolution isn’t just about automating tasks; it’s about transforming AI into a collaborator, pushing the boundaries of creativity and innovation.

Data Modalities: Beyond Text/Images

As AI grows, it’s venturing into new data territories. Vision and language remain the bedrock, powering breakthroughs in computer vision, natural language processing, and search engines. AI’s ability to interpret haptics—touch and pressure data—is revolutionizing robotics and prosthetics, enabling machines to “feel.” Meanwhile, spectral data, which captures wavelengths beyond the visible spectrum, is transforming agriculture, defense, and medical imaging. These new modalities are extending AI’s reach into uncharted territories, creating once unimaginable possibilities.

AI Systems: From Assistance to Autonomy

AI systems are evolving at a pace that’s reshaping the boundaries of what machines can do. What’s most fascinating is how our trust in these systems has grown, allowing them to take on tasks that were once the sole domain of humans. This evolution is about more than efficiency—it’s about unlocking entirely new possibilities.

Let’s start with semi-autonomous systems. These are like reliable coworkers who need just a little oversight. Take drones, for example. They can map terrain, deliver packages, and navigate through complex environments on their own. Yet, for critical decisions—like rerouting in extreme weather or avoiding restricted airspace—they defer to human judgment. This “autonomy with a safety net” allows machines to handle repetitive or high-risk tasks while keeping us in the loop for truly important calls.

Then, there are multi-agent systems, which take teamwork to a whole new level. Imagine a city where AI agents manage traffic lights, not individually, but as a coordinated network. These systems communicate in real-time, adjusting signals across intersections to ensure the smoothest flow of vehicles. The result? Reduced congestion, lower emissions, and happier commuters. It’s like having an army of super-intelligent traffic controllers working in perfect harmony. But multi-agent systems don’t stop at collaboration—they can also compete. In financial markets, trading bots operate as independent agents, analyzing trends, predicting outcomes, and executing trades faster than any human could. Some even collaborate to manipulate market conditions for mutual gain, demonstrating just how dynamic and adaptive these systems can be.

The next step is fully autonomous systems. These are the true independents, capable of sensing, deciding, and acting entirely on their own. Self-driving cars are a prime example: they don’t just follow a programmed route—they interpret traffic signs, anticipate pedestrian movements, and adjust to road conditions in real-time. Industrial robots are similarly autonomous, assembling products, identifying defects, and even scheduling their own maintenance. But here’s where the challenge lies: when we hand over complete control to machines, the stakes are higher than ever. Safety protocols must be flawless, and decision-making processes must be transparent and accountable. We’re making great strides, but full autonomy still demands rigorous refinement.

And now, for the most exciting part? economic participation. AI is no longer just about doing what we already do, only faster or better—it’s about exploring new frontiers. Take generative AI systems in intraday trading. These systems analyze vast amounts of market data, identify patterns, and execute strategies faster than any human trader. What’s even more remarkable is their ability to adapt in real-time, learning from market fluctuations and refining their tactics on the fly. These agents aren’t just assistants; they’re active participants in the economy, creating wealth independently.

Beyond finance, autonomous systems are venturing into creative and strategic territories. Imagine an AI agent tasked with designing marketing campaigns. It could analyze audience data, draft ad copy, create visuals, and even determine the optimal time to launch—all without human intervention. Or consider healthcare, where AI agents might coordinate treatments for patients, consulting with doctors only for the most complex cases. These examples aren’t just theoretical—they’re already in motion, signaling a profound shift in how we think about machine intelligence.

This evolution is about more than technology—it’s about redefining what machines can do for us. AI is no longer confined to automation or assistance. It’s becoming a collaborator, a decision-maker, and even an innovator in its own right. And here’s the kicker: we’re only scratching the surface of what’s possible. As these systems grow more capable, adaptive, and independent, they’ll continue to push the boundaries of intelligence, creativity, and impact. This is the dawn of a new era, where machines don’t just follow—they lead. And the journey has only just begun.

How can this be regulated!?

Balancing Act of AI Regulation

Transparency | Equity | Accountability | Privacy | Safety | Responsible | Ethical | Trustable

Regulating AI is a delicate balancing act—a tightrope walk between fostering innovation and ensuring safety. Governing bodies are tasked with a daunting challenge: to embrace the transformative potential of AI while protecting society from its unintended consequences. It starts with risk classification, determining which systems are harmless tools and which might pose significant risks, like surveillance technologies with the potential for misuse. Coupled with this is the need for transparency and explainability—because if even the creators can’t fully understand their models, how can users or regulators trust them?

Data privacy is another cornerstone of responsible AI regulation. With AI systems handling sensitive information, particularly in fields like healthcare, strict safeguards are essential to prevent misuse and protect personal data. Beyond that, regulators must confront harmful applications head-on, banning systems that perpetuate discrimination or exploitation, such as social scoring mechanisms. These measures aren’t about stifling progress; they’re about building a foundation of trust, ensuring that AI not only thrives responsibly but also paves the way for sustainable growth. By navigating this tightrope, we can unlock AI’s full potential while keeping its promises aligned with human values.

Summary

Technology in 2024 is transforming at an unprecedented pace, bringing us closer to the Star Trek dream of seamless human-machine collaboration.

AI micro trend is pushing boundaries, evolving from simple assistants to autonomous decision-makers. Multi-agent systems optimize cities, while generative AI creates solutions in real time, transitioning from insights to actions. Machines are no longer just tools—they’re collaborators, turning science fiction into reality and revolutionizing how we work, live, and connect.

The Quantum Digitalization macro trend is all about redefining hyper-personalization, moving from digitizing workflows to creating immersive, Qigitalized ecosystems where the physical and digital worlds blend effortlessly. Powered by quantum sensing, computing, and communication, this trend is driving unprecedented accuracy, security, and innovation.

“Trend is fashionable. Fashions change. Old trends come back. But when they return, they’re never quite the same—they carry the weight of the past, reimagined for the present. A trend reborn isn’t just a repeat; it’s a remix, blending nostalgia with innovation, and reminding us that in the cycle of change, the new is often just the old with better timing. What will trend next?”

Data Descriptors (Stats, Relations, Patterns)

Data analysts look for descriptors in data to generate insights.

For a Data aggregator, descriptive attributes of data like size, speed, heterogeneity, lineage, provenance, and usefulness are essential to decide the storage infrastructure scale, data life cycle, and data quality. These aggregator-oriented descriptions are black-box perspectives.

For a Data analyst, descriptive statistics, patterns, and relationships are essential to generate actionable insights. These analyst-oriented descriptions are white-box perspectives. The analysts then use inferential methods to test various hypotheses.

Descriptive Statistics

Data analysts usually work with a significant sample of homogenous records to statistically analyze features. The typical descriptive statistics are – measures of location, measures of center, measures of skewness, and measures of spread.

E.g., A 23 member cricket team of three different states has players of the following ages:

Karnataka: [19,19,20,20,20,21,21,21,21,22,22,22,22,22,22,23,23,23,23,24,24,24,25,25]

Kerala: [19,19,20,20,20,21,21,21,22,22,22,22,23,23,23,23,23,24,24,24,24,24,24]

Maharashtra: [19,19,19,19,19,19,20,20,20,20,20,21,21,21,21,22,22,22,23,23,24,24,25]

Numbers represented this way does not help us detect patterns or explain the data. So, it’s typical to see the tabular distribution view:

AGEKarnatakaKeralaMaharashtra
19226
20335
21434
22543
23452
24362
25201
Age Distribution of State Players

This distribution view is better. So, we would like to see measures of center for this data. These are usually – MEAN, MEDIAN, and MODE.

  • MEAN is the average (Sum Total / # Total)
  • MEDIAN is the middle number
  • MODE is the highest frequency number
MeasureKarnatakaKeralaMaharashtra
MEAN2222.121
MEDIAN222221
MODE222419
Measures of Center

This description is much better. So, we would like to see this graphically to understand the skewness.

Measuring skewness

The age distribution is symmetrical for Karnataka, skewed to the left for Kerala, and skewed to the right for Maharashtra. The data analyst may infer that Karnataka prefers a good mix of ages, Kerala prefers player experience, and Maharashtra prefers the young.

The data analyst may also be interested in standard deviation, i.e., a measure of spread. The standard deviation symbol is sigma (σ) for a sample and is the MEAN distance from the mean value of all values in the sample. Since a distance can be positive or negative, the distance is squared, and the result is square-rooted.

MeasureKarnatakaKeralaMaharashtra
Standard Deviation1.81.71.8
Measure of Spread

In our example, a measure of location (quartiles, percentiles) is also of interest to the data analyst.

PercentileKarnatakaKeralaMaharashtra
25 Percentile212119.5
50 Percentile222221
75 Percentile2323.522
100 Percentile252425
Measure of Location

The table above shows that the 50 percentile value is the median, and the 100 percentile is the maximum value. This location measure is helpful if the values were scores (like in an exam).

Combining statistics and display to explain the data is the art of descriptive statistics. There are several statistics beyond the ones described in this short blog post that could be useful for data analysis.

Time-series Data Patterns

The time-series data has trends, variations and noise.

  1. A trend is the general direction (up, down, flat) in data over time.
  2. Cyclicity variation is the cyclic peaks and troughs in data over time.
  3. Seasonality variation is the periodic predictability of a peak/trough in data.
  4. Noise is meaningless information in data.

The diagrams below provide a visual explanation:

“Ice cream sales are trending upward,” claims an excited ice-cream salesman.

“Used Car value is trending downward,” warns the car salesman

Every business has up and down cycles, but my business is trending upwards,” states a businessman.

It’s the end of the month, so, Salary and EMI season in user accounts, so the transaction volume will be high,” claims the banker.

“There is some white noise in the data,” declared the data scientist.

Data Relationships

Data analysts seek to understand relationships between different features in a data set using statistical regression analysis.

There could be a causal (cause and effect) relationship or simply a correlation. This relationship analysis helps to build predictors.

A simple measure of linear relationship is the correlation coefficient. The measure is not relevant for non-linear relationships. Correlation coefficient of two variables x and y is is calculates as:

correlation(x, y) = covariance(x, y) / (std-dev(x) * std-dev(y))

It’s a number that in the range [-1,1]. Any number closer to zero implies no correlation, and closer to either extremity means higher linear correlation.

  • Negative one (-1) means negatively linearly correlated
  • Positive one (1) means positively linearly correlated
  • Zero (0) means no correlation

Example: Let’s take this random sample.

XYY1Y2Y3
13-383-100
28-8108250
315-15146-50
424-2498150
535-35231-50
648-48220155
763-63170-125
880-80100-150
999-99228-12
10120-120234190
Sample Data
X and YX and Y1X and Y2X and Y3
1-10.60
Correlation coefficient

Visually, we can see that as X increases, Y increases linearly, and Y1 decreases linearly. Hence, the correlation coefficient is positive (1) and negative (-1), respectively. There is no linear relation between X and Y3, and hence, the correlation is 0. The relationship between X and Y2 is somewhere in between with a positive correlation coefficient.

Scatter plot X against (Y, Y1, Y2, Y3)

If X is the number of hours bowler practices and Y2 is the number of wickets, then the correlation between the two can be considered positive.

If X is the number of hours bowler practices and Y3 is the audience popularity score, then the correlation between the two can be considered negligible.

If X is the number of years a leader leads a nation, and Y or Y1 is his popularity index, then the correlation between the two can be considered linearly increasing or decreasing, respectively.

Summary

Data analysts analyze data to generate insights. Insights could be about understanding the past or using the past to predict the (near) future. Using statistics and visualization, the data analysts describe the data and find relationships and patterns. These are then used to tell the story or take actions informed by data.

V’s of Data

Volume, Velocity, Variety, Veracity, Value, Variability, Visibility, Visualization, Volatility, Viability

What are the 3C’s of Leadership? “Competence, Commitment, and Character,” said the wise.

What are the 3C’s of Thinking? “Critical, Creative, and Collaborative,” said the wise.

What are the 3C’s of Marketing? “Customer, Competitors, and Company,” said the wise.

What are the 3C’s of Managing Team Performance? “Cultivate, Calibrate, and Celebrate,” said the wise.

What are the 3C’s of Data? “Consistency, Correctness, and Completeness,” said the wise; “Clean, Current, and Compliant,” said the more intelligent; “Clear, Complete, and Connected,” said the smartest.

“Depends,” said the Architect. Technologists describe data properties in the context of use. Gartner coined the 3V’s – Volume, Velocity, and Variety to create hype around BIG Data. These V’s have grown in volume 🙂

  • 5V’s: Volume, Velocity, Variety, Veracity, and Value
  • 7 V’s: Volume, Velocity, Variety, Veracity, Value, Visualization, and Visibility

This ‘V’ model seems like blind men describing an elephant. A humble engineer uses better words to describe data properties.

Volume: Multi-Dimensional, Size

“Volume” is typically understood in three dimensions. Data is multi-dimensional and stored as bytes—a disk volume stores data of all sizes. Data does not have volume! It has dimensions and size.

A person’s record may include age, weight, height, eye color, and other dimensions. The size of the record may be 24 bytes. When a BILLION person records are stored, the size is 24 BILLION bytes.

Velocity: Speed, Motion

Engineers understand the term velocity as a vector and not a scalar.

A heart rate monitor may generate data at different speeds, e.g., 82 beats per minute. I can’t say my heart rate is 82 beats per minute to the northwest. Hence, heart rate is a speed. It’s not heart velocity. I can say that a car is traveling 35 kilometers per hour to the northwest. The velocity of the vehicle is 35KMPH NW.

Data does not have direction; hence it does not have velocity. Data in motion has speed.

Variety: Heterogeneity

The word variety is used to describe differences in an object type, e.g., egg curry varieties, pancake varieties, sofa varieties, tv varieties, image data format varieties (jpg, jpeg, bmp), and data structure varieties (structured, unstructured, semi-structured). Data variety is abstract and is a marketecture term.

Heterogeneity is preferred because it explicitly states that:

  1. Data has types (E.g., String, Integer, Float, Boolean)
  2. Composite types are created by composing other data types (E.g., A Person Type)
  3. Composite types could be structured, unstructured, or semi-structured (E.g., A Person Type is semi-structured as the person’s address is a String type)
  4. Collections contain the same or different data types.
  5. Types, Composition, and Collections apply to all data (BIG or not).

Veracity: Lineage, Provenance

Veracity means Accurate, Precise, and Truthfulness.

Let’s say that a weighing scale reports the weight of a person as 81.5 KG. Is this accurate? Is the weighing scale calibrated? If the same person measures her weight on another weighing scale, the reported weight might be 81.45 KG. The truth may be 81.455 KG.

Data represent facts, and when new facts are available, the truth may change. Data cannot be truthful; it’s just facts. Meaning or truthfulness is derived using a method.

Lineage and provenance meta-data about Data enables engineers’ to decorate the fact with other useful facts:
1. Primary Source of Data
2. Users or Systems that contributed to Data
3. Date and Time of Data collection
4. Data creation method
5. Data collection method

Value: Useful

If Data is a bunch of facts, how can it be valuable? Understandably, the information generated from data by analyzing the facts is valuable. Data (facts) can either be useful to create valuable information or useless and discarded. We associate a cost to a brick and a value to a house. Data is like bricks used to build valuable information/knowledge.

Summary

I did not go into every V, but you get the drill. If an interviewer asks you about 5V’s in an interview, I request you to give the standard marketecture answer for their sanity. The engineer’s vocabulary is not universal; technical journals publish articles in the sales/marketing vocabulary. As engineers/architects, we have to remember the fundamental descriptive properties of data so that the marketecture vocabulary does not fool us. However, we have to internalize the marketecture vocabulary and be internally consistent with engineering principles.

It’s not a surprise that Gartner invented the hype cycle.

Data Aggregation (Map, Filter, Reduce)

Data engineers think in batches!

Thinking in batches reminds me of a famous childhood story.

Once upon a time, a long, long time ago, there was a kind and gentle King. He ruled people beyond the horizon, and his subjects loved him.

One day, a tired-looking village man came to the King and said, “Dear King, help us. I am from a village beyond the horizon. It’s been raining for several days. My village chief asked me to fetch help from you before disaster strikes. It took me five days to walk to the Kingdom, and I am tired but glad that I could deliver this message to you.”

“I am glad that you came for help. I will send Suppandi, my loyal Chief of Defence, to assess the damage and then send help,” said the King. “Suppandi, you have your orders. Now, go. Assess the damage, report to me, and help,” ordered the King.

Suppandi left to the village beyond the horizon on his fastest horse. When he reached the town, the town was flooded, and Suppandi felt the urge to return to the King quickly to inform him about the floods. So, he drove his horse faster and reached the Kingdom in 1/2 day. He went to the King and told him. “Dear King, the village is flooded. I went in a day and came back in 1/2 day to give you this information.”

Suppandi was pleased with himself. However, the King wanted more information. “Suppandi, please tell me whether people in the village have food, are children hurt? What can we do more to help?”

“I will find out, Dear King,” said Suppandi. He left again on his fastest horse. This time he reached in 1/2 day. He figured that people don’t have food, and many children are hurt and homeless. He raced back to the Kingdom. “Dear King, I reached in 1/2 day and came back in another 1/2. The villagers don’t have food to eat, and they are hungry. Several children are hurt and need medical attention,” said Suppandi.

This time the King had more questions. “Dear Suppandi, what did the village chief say? What can we do for him?”

“Dear King, I will find out. Let me leave to the village immediately,” said Suppandi.

Chanakya was eagerly listening in to the conversation. He told Suppandi, “Dear Suppandi, you must be tired. Let me take over. Take some rest.”

Immediately, Chanakya ordered his men to collect food, water, clothes, medicines, and doctors. He asked for the fastest horses, and along with several men and doctors, he left for the village beyond the horizon. When he reached, the town was flooded, and people were on their home terraces. He found several houses destroyed and hungry kids taking shelter under the trees, and many wounded villagers.

He ordered his men to save the villagers skirting the flood, protect all children, feed them, and take them to a safe place. He also called the doctors to attend to the wounds.

The men built a temporary home outside the village to give shelter to the homeless. They waited for a few days for the rain and flood to subside. When it was bright and sunny, Chanakya, his men, and the villagers cleaned the village, re-built the homes, and deposited enough food and grains for six months before saying goodbye.

Chanakya reached the Kingdom and immediately reported to the King. The King was anxious. He said, “Chanakya, you were gone for two weeks with no message from you. I was worried. Did you speak to the village Chief?”

“Dear King, Yes, on your behalf, I spoke to the village chief. I found that the village was flooded, so we rescued all the villagers, attended to the wounded, fed them, re-built their homes, and left food and grains for six months. The people have lost their belongings in flood, but all of them are safe, and they have sent their wishes and blessings for your timely help,” said Chanakya.

The King was pleased. “Chanakya, I should have sent you earlier. You are a batch thinker! Thank you,” said the King.

Suppandi was disappointed. He had worked hard to drive to the village and report to the King as instructed, but Chanakya gets all the praises. To this date, he still does not understand and is hurt.

Most non-data engineers are like Suppandi; they use programming constructs like “for,” “if,” “while,” and “do” on remote data. Most data engineers are like Chanakya; they use the programming constructs like “map,” “filter,” “reduce,” and “forEach.” Programming with data is always functional/declarative, while traditional programming is imperative.

There is nothing wrong with acting like Suppandi; he is the Chief of Defence. But, some cases require Chanakya thinking. In architectural language, Suppandi actions move data to algorithms, and Chanakya actions move algorithms to data. The latter works better when there is a distance and cost-to-travel between data and algorithms.

This difference in thinking is why data engineers use SQL, and traditional engineers use C#/Java. SQL uses declarative commands that are sent to the database to pipeline a set of actions on data. The conventional programming languages have caught up to the declarative programming paradigm by supporting lambda functions (arrow functions), and map/filter/reduce style functions on data collections. The map/filter/reduce style functions allow compilers/interpreters to leverage the underlying parallel compute backbone (the expensive eight-core CPU) or use a set of inexpensive machines for parallel computing. They are abstracting away parallelism from the programmer. The programmer helps the compiler/interpreter to identify speed-improvement opportunities by explicitly programming declaratively.

Mapping

Instead of iterating over a collection one at a time, a map is a function to apply another function to all elements of a collection. The map function may split the collection into parts to distribute to different cores/machines. The underlying collection remains immutable. In general, mapping could mean one-2-one, one-2-many, and many-2-one; and is the process of applying a relation (function) to map an element in the domain with an element in the range. In the case of computing, mapping does not change the size of the collection.

E.g., [1,2,-1,-2] => [1,4,1,4] using the squared relation is a many-2-one mapping

var numbers = [1, 2, -1, -2];
var x = numbers.map(x => x ** 2);
console.log(x);
[1,4,1,4]

E.g., [1,2,-1,-2] => [2,3,0,-1] using the plus one relation is a one-2-one mapping

var numbers = [1, 2, -1, -2];
var x = numbers.map(x => x + 1);
console.log(x);
[2, 3, 0, -1]

E.g., [1,2,-1,-2] using the plus one and squared relation is a one-2-many mapping

var numbers = [1, 2, -1, -2];
var x = numbers.map(x => [x + 1, x ** 2]);
console.log(x);
[[2, 1], [3, 4], [0, 1], [-1, 4]]

E.g., An SQL Example of a one-2-one mapping

SELECT Upper(ContactName)
FROM Customers
MARIA ANDERS
ANA TRUJILLO
ANTONIO MORENO
THOMAS HARDY

Filtering

Instead of iterating over a collection one at a time, a filter is a function to return a subset of elements that match criteria. The filter function may split the collection into parts to distribute to different cores/machines. The underlying collection remains immutable. Examples:

var numbers = [1, 2, -1, -2];
var x = numbers.filter(x => x > 0);
console.log(x);
[1, 2]
SELECT *
FROM Customers
WHERE Country="USA"

Number of Records: 13

CustomerIDCustomerNameContactNameAddressCityPostalCodeCountry
32Great Lakes Food MarketHoward Snyder2732 Baker Blvd.Eugene97403USA
36Hungry Coyote Import StoreYoshi LatimerCity Center Plaza 516 Main St.Elgin97827USA
43Lazy K Kountry StoreJohn Steel12 Orchestra TerraceWalla Walla99362USA
45Let’s Stop N ShopJaime Yorres87 Polk St. Suite 5San Francisco94117USA

Reduce

Instead of iterating over a collection one at a time, a reduce is a function to return a single value. The reduce function may split the collection into parts to distribute to different cores/machines. The underlying collection remains immutable. Examples:

var numbers = [1, 2, -1, -2];
var x = numbers.reduce((sum,x) => sum + x, 0);
console.log(x);
0
SELECT count(*)
FROM Customers
Number of Records: 1
count(*)
91

Pipelining

When multiple actions need to be performed on the data then it’s a norm to pipeline the actions. Examples:

var numbers = [1, 2, -1, -2];
var x = numbers
  .map(x => x + 1) //[2,3,0,-1]
  .filter(x => x > 0) //[2,3]
  .map(x => x ** 2) //[4,9]
  .reduce((sum, x) => sum + x, 0) //13
console.log(x);
13
SELECT Country, Upper(Country), count(*)
FROM Customers
WHERE Country LIKE "A%"        
GROUP BY Country
Number of Records: 2
Country Upper(Country) count(*)
Argentina ARGENTINA 3
Austria AUSTRIA 2

Takeaway

Data Engineers use Chanakya thinking to get work done in batches. Even streaming data is processed in mini-batches (windows). Actions on data are pipelined and expressed declaratively. The underlying compiler/interpreter abstracts away parallel computing (single device, multiple devices) from the programmer.

Think in Batches for Data.