Next Generation Enterprise Architecture: Self-describing Enterprise Architectures

About a decade ago, together with Marin Zec and Florian Matthes, I published the Enterprise Architecture (EA) Visualization Tool Survey (https://wwwmatthes.in.tum.de/pages/6u8f5ki1t2yz/EAVTS2014-Enterprise-Architecture-Visualization-Tool-Survey).

As a management consultant, I come across Enterprise Architecture tools here and there in different contexts. Most of the time, I see the results as Excel exports. Depending on the data quality, completeness, etc. it can be a helpful source for all kinds of projects: e.g. cost cutting, harmonization during an M&A, etc.

Hands down: The tools did not change much, so as the underlying problems! Even in the most mature organization I provided my services to, the EA tool/database was just not up to date. And of course they already planned a transition to yet another new EA tool.

Do not try to document the past

Enterprise Architects, Architecture Boards and roles/committees alike never get tired of approaching product owners to maintain their IT product/application within the EA tool. Most prominent examples are

• Business Processes and Capabilities supported by the application
• Data stored in the tool and Data flows between the applications (and thus also through the processes)
• Technology Stack used to realized the application
• Infrastructure: Server the application runs on and potentially even IP address information
• …

No matter what you are trying to document, it is highly likely that you look at outdated or inaccurate data. Consultants deal with that every day. This comes with the nature of the process: software is first adapted (in most of the cases an existing application is just modified, not a new introduced), time is precious and short, changes are deployed after a product increment is ready. Documentation may or may not be updated after deployment. 

No matter which methodology your organization follows, documenting an EA is documenting a moving target. After decades of EA tools and initiatives, it is time to say: Do not try to document the past.

The idea of automation

Back in the days, we already had that vision if we would just do it clever enough, we can get much of the information required automatically through infrastructure monitoring tools or process modeling tools.

Knowing that there is a considerable gap between the process models stored in tools or slides and day-to-day reality, just colliding all required information and working out the differences/collisions and inconsistencies would not do the trick either.

With today’s cloud-based stacks, one gets a good overview on the technical aspects. As soon as more people-based aspects come in, self-describing systems are rather a vision than done in practice.

Toward self-describing Enterprise Architectures

Knowing the symptoms, having been there, what could be a solution? One can seek to answer that question layer by layer:

• Processes: applications know very little about what process they support. This is probably the one layer to talk more about going forward and to understand better.
• Solutions/IT Products/applications: While having many names and coming in many granularities, all IT systems are implemented using a programming language at their heart. Modern programming languages know a great deal about the program they are executing. They have transparency on the entities used/stored. Besides: some know about their runtime environment too.
• Data flows: Open API standards have become economically meaningful to the extent that modeling everything in excessive detail, such as OWL approaches, often incurs unacceptably high costs.
• Infrastructure: Cloud providers and containerization technologies know a fair deal about their setup. Among others, this property is leveraged by the community of monitoring tools.

Relating these facts with the idea of Automated Enterprise Architecture, one could attempt to collect these information and bring them all together. Since initially published, the term Automated Enterprise Architecture reached commercial EA tools — so you might want to reconsider “just collecting”.

In an era of Artificial Intelligence being clever enough to replace human workers, how can it be that we still build IT products that cannot answer simple questions like

• “What data do you produce?”
• “What data do you consume?”
• “Which other systems are you talking to?”
• …

in a uniform and processable way.

Built-in meta information

As an Enterprise Architect you probably know Prometheus and possibly had a first hand look at Grafana in one of the newer IT products. While Grafana provides a fancy way to look at it, Prometheus collects all sorts of data using a plethora of data extraction plugins. Collected data is usually up-to-date, accurate and collected in a federated way.

You want to have a view on the current Enterprise Architecture and not an outdated one? 

• Establish a commonly accepted set of questions an IT product should answer.
• Propose a metamodel that is strong enough to answer all questions.
• Provide prototypes / plugins that help/demonstrate how to implement a self-describing endpoint in the most commonly used tech stack in your organization.
• Establish the standard: Gradually make it part of the Continuous Deployment pipeline.
• See your EA model grow.

Also published on Medium.com: https://medium.com/@sascha-roth/next-generation-enterprise-architecture-self-describing-enterprise-architectures-a2628553e687