Mapping your Cloud Ecosystem
Author's note: This is article is next in a series, so will build upon the work below:
The "AI Strategy Framework" guides your organization's journey in the Age of AI
Crafting the Executive Vision and an Actionable Roadmap for your AI Strategy
I mentioned the "Reference Ecosystem" in my recent piece, Principles of Ecosystem-Oriented Architecture: The future of enterprise cloud, data, and AI. If you've not read it, add it to the others I listed above, and consider it the essential prequel to the below. "Ecosystem Map" is both one of the 25 dimensions of the AI Strategy Framework and a foundational concept in ecosystem-oriented architecture (EOA), which makes this article doubly important reading for strategic thinkers on both fronts.
Let's nail down these key terms, here, so that we're clear on the differences between them:
An Ecosystem Map is a high-level architectural diagram of an organization’s cloud ecosystem, and something that every organization must create at the start and continuously evolve as they progress on their journey;
The Reference Ecosystem is a specific ecosystem map that Ana Welch and I have created as a composite of many different cloud ecosystems across many different industries.
The "City" that is your Cloud
The “map” metaphor is instructive here. It is used to distinguish an ecosystem map from the various forms of architectural diagrams, nearly all of which tend to include more technical minutiae than a typical ecosystem map. Whereas an architectural diagram provides specific parameters for specific technical solutions, an ecosystem map presents a higher-level, more visionary view of an organization’s cloud ecosystem.
To make an analogy to architecture in the physical world:
Solution architecture provides schematics - floor plan, dimensions, electrical wiring, ventilation, plumbing - from which a building is constructed;
Enterprise architecture provides plans for specific neighborhoods or systems such as a subway or electrical grid;
Ecosystem architecture and, by extension, an ecosystem map shows us the entire city.
This analogy is fundamental to understanding and practicing EOA. Thinking of an organization’s cloud ecosystem as a city, we then conceptualize the next-level-down component parts of the ecosystem as “neighborhoods” (we might have also called them “boroughs”). Cities the world over are pieced together this way: Downtown, Seaport, Southie, etc. in Boston; Greenwich, Soho, Canary Wharf, etc. in London (though you’re forgiven if you thought I was talking about New York until you got to “Canary Wharf”); Palermo, Recoleta, Puerto Madero, etc. in Buenos Aires; Norrmalm, Gamla Stan, Kungsholmen, etc. in Stockholm. The list goes on.
Each of these neighborhoods share the quality of dividing their city into smaller pieces, each often with their own distinct culture, aesthetic, and purpose.
Like cities, ecosystem maps are constantly evolving and changing. To prevent your ecosystem from becoming overcrowded, stagnant, or unable to meet the needs of its expanding 'population,' it's essential to revisit, revise, and adapt your Ecosystem Map on a regular basis.
We apply this same concept to EOA, clustering technologies and workloads devoted to similar purposes into distinct neighborhoods, and tying them together with the flow of data, logic, and actions—our roads, subway lines, waterworks, electricity, etc.—to build coherent cities in the cloud.
To do this, we compared the cloud ecosystems across real-world organizations in different industries and geographies to compile their common features and best practices. We then produced the “Reference Ecosystem”, that is to say, a notional cloud ecosystem that is a composite of the cases we considered. This reference ecosystem is the ideal standard, the prototypical notion of what a cloud ecosystem might look like.
Variation surely exists amongst individual organizations and sizes, industries, geographies, and other considerations such as regulatory requirements. To compensate for this variation, we then spent nearly two years applying our reference ecosystem to real-world scenarios, testing the composite model against actual guiding principles, business objectives, and requirements. We incorporated lessons learned from these scenarios to improve and refine the Reference Ecosystem over time, and were pleased to see how durable the model really is. Indeed, over time more and more of the ecosystems we mapped began to converge on and look more like our best practice Reference Ecosystem. We interpret this as evidence of its durability and flexibility to meet the needs of various organizational profiles.
Unfortunately (at least as of 5 December 2024), LinkedIn articles strangely do not support expanding images, so this one can be tough on the eyes. That said, you can find an expanded version on page 24 of the whitepaper available here.
Neighborhoods in your Ecosystem Map
In general, we find that most cloud ecosystems are home to six major neighborhoods.
Core Platform Services
The Core Platform Services Neighborhood includes many of the infrastructure, security, governance, management, and monitoring services used across the ecosystem. Largely synonymous with a “cloud landing zone”, the Reference Ecosystem above shows (left to right) Purview, Entra ID (formerly Azure Active Directory), Key Vaults, Azure Monitor, Azure DevOps, Security Center, Application Insights, and Power Platform Managed Environments as examples.
Integration
The Integration Neighborhood includes services for (left to right in the diagram) technology-specific integration services such as OneLake shortcuts in Microsoft Fabric and virtual tables in Microsoft Dataverse, event-driven integration services such as Event Grid and Service Bus, use of APIs via API Management, logic-driven integration such as Logic Apps and Azure Functions, batch integration relying on Azure Data Factory, and a generic master data management (MDM) solution.
Core Business Systems
The Core Business Systems Neighborhood includes the “Tier 1” business applications common in many organizations such as ERP, CRM, HRMS, etc. The Reference Ecosystem uses custom applications built atop Azure SQL and Power Platform, CRM in Dynamics 365, ERP in SAP, and HR in Workday as illustrative examples of the many solutions organizations rely on in their core business systems neighborhood.
Application Portfolio
The Application Portfolio Neighborhood may include “Tier 1” applications but often include solutions aimed at smaller audiences or more niche business processes of the Tier 2 (“business important) and Tier 3 (“productivity”) variety. EOA in the Microsoft context ought to rely heavily on Power Platform in the Application Portfolio Neighborhood, though as you see above, we also highly favor Power Platform for core business systems.
Unstructured Data
The Unstructured Data Neighborhood includes services and storage for documents, files, photos, videos, etc. often housed in legacy network file storage, SharePoint, or Azure Blob Storage. Many of the Microsoft 365 services will operate here, noting that we strongly recommend using SharePoint as a data service for the Tier 1 or Tier 2 workloads found in other neighborhoods.
Data Distribution
The Data Distribution Neighborhood provides for data consolidation in (for example) OneLake, and for all manner of “downstream” data distribution such as search, APIs, data warehousing, analytical workloads, and use in AI-driven scenarios. We’ve also decided to place Microsoft 365 Copilot in the Data Distribution neighborhood as it—like many other scenario-specific Copilots that also reside there—relies on consuming, interpreting, and otherwise distributing data in response to user prompts. Note that M365 Copilot is hydrated with data from Microsoft 365 via the Microsoft Graph, and may be configured to consume data from elsewhere in the data estate, as well.
A Practical Example
The example below is one of my recent favorites produced in collaboration with a global enterprise organization as part of their AI and broader cloud strategy. I've heavily anonymized it, but it remains highly instructive.
As with the previous diagram, you can overcome LinkedIn's odd lack of support for expanded images by taking a look at page 22 of the whitepaper available here (which is not the same as the previous whitepaper I mentioned).
Note that:
Specific AI and other data products are identified in the Data Distribution Neighborhood (bottom of the box on the far right);
The ecosystem map shows how data will flow from the organizations’ core business systems (including some yet to be identified, which is just fine) and application portfolio such that it hydrates various data distribution points, including but not limited to AI workloads;
Migration of legacy applications and legacy data warehouses is identified as a priority.
Mapping the cloud ecosystem is a key element of both your AI strategy and ecosystem-oriented architecture (which is very much built for the age of AI) because the efficacy of any AI workload is directly related to the quality of the data upon which the workload’s model is trained or augmented. You can geek out with me about generative AI acting on enterprise data in this previous piece. Mapping, evolving, and maintaining the organization’s cloud ecosystem map provides the essential high-level technical architecture underpinning our AI strategy.
There's a lot here, so I'll take on the various pieces of this in future articles. Consider this your foundational introduction, though, to both the Reference Ecosystem and to ecosystem maps more broadly. Creating yours needs to feature prominently as you craft your strategy. Updating and evolving it over time is an indispensable discipline.