Sunday, July 16, 2017

SOA AND NONTRIVIAL CHALLENGES TO ITS REALIZATION

Time for a reality check, I think.

There is a fair amount of consensus in the literature on SOA suggesting that organizational readiness and governance are essential keys to a successful SOA operating at an enterprise level.

The problem is that this prerequisite has a near-zero probability of being implemented meaningfully in any enterprise of moderate or large size. We can have meetings. We can have vision statements. We can have architectural initiatives. We can have well-intentioned business and technology stakeholders committed in principle to SOA.

None of this matters.

An SOA can expose a trivial service -- e.g., a holiday calendar that consuming systems use to determine business days in the U.S. and in other countries. While useful, the real promise of SOA, as an architectural style, stems from the notion that we can extract object-oriented principles, such as encapsulation and reuse, from the realm of application-level runtimes and implement them in a network-mediated enterprise environment -- meaning that we can provide a functional decomposition of LOB-specific applications, identify shared functionality across the LOBs, and consolidate those functions into unique services that can be consumed across LOBs through standardized network-based communication transport and messaging protocols.

This sounds great as both an architectural objective and as a business case pitched at the enterprise level. True, there are a number of challenges to be overcome: Stove-piped applications need to be decomposed and reassembled. Stove-piped data stores may need centralization--and surely need to be refactored so that LOB-specific data are derived from an enterprise-level ontology and concomitant semantic conventions. Tools need to be selected and implemented. Etc. These challenges are significant but not insurmountable as technical exercises.

Unfortunately, these technical exercises never take place in an organizational vacuum. And this is where some nasty realities rear their heads. Consider:

1. Large enterprises are organized into LOBs. Governance, accountability, profit and loss, rewards, etc. all get defined in these terms. Yes, there are enterprise-wide governance structures. But they function more like a federated aggregation of individual constituencies than as constituent components of a monolithic organization focused on the interests of the enterprise as a whole. (Sounds pretty much like a legislative assembly, doesn't it?) Organizations like this function by striking bargains and making compromises. This is not an environment conducive to promoting a cross-LOB decomposition of applications and data stores. There are too many LOB-specific constituencies that will not see how this benefits their organizations, their personal interests, or the interests of their customers.

2. Large size presents other problems. Even in the stove-piped world, not every technology function is packaged up into the one big box. There are networks, messaging systems, source control and deployment systems, system administrators, DBAs, etc. As we all know, this engenders more complexity in implementation, problem resolution, and day-to-day application management than comparable activities executed by programmers in a development environment. Decomposing business applications into functional services expands this complexity geometrically. And I would assert that, in large enterprises, the logic of geometric complexity serves to diminish the perceived benefits of application rationalization through SOA in the eyes of the very decision makers whose stove-piped LOB responsibilities make them predisposed to stove-piped solutions in any event.

3. Finally, there is the very thorny problem of LOB-specific business complexity in cases of proprietary product lines. Increasingly, large economies in the Western world are dominated by firms that produce proprietary services and intellectual products. Commodified products in traditional manufacturing industries have moved to low-wage economies. Automating business processes attending proprietary products and services typically means automating a complex ensemble of business rules. Complexity comes in the form of contingent logic, exceptions to contingent logic, exceptions to exceptions, and data elements whose semantic meaning are coupled tightly to a given LOB. In such an environment, it may be impossible, and almost certainly cost-prohibitive, to produce a functional decomposition and correlative abstractions resulting in business services that are reusable across LOBs while meeting the business needs of those units. A service interface that is nothing more than an if-then redirect to multiple LOB-specific implementations offers no value at all.

Note. I am not claiming that these barriers to effective SOA represent irrationality. Quite the opposite. Rationality resides at the level of the individual agent. Enterprises are neither rational nor irrational. Rather, they are ensembles of (complex) rules and concomitant incentives within which rational agents act. On this view, rational agents within large enterprises, particularly key decision makers, respond to LOB-specific incentives, organizational scale, and the inherent complexity attending proprietary business processes (and the software that automates them) in ways that militate against the architectural objectives of SOA.

Does this mean SOA is impossible? No. But I would expect that you will find successful implementations mostly in the form of utilities that are simple in function and uncontroversial in scale. A holiday calendar web service comes to mind. So do utilities like the Google Docs office suite. In such cases, the problem domain is simple, well understood, and noncontroversial. My experience tells me that value-add business applications almost never reflect such criteria. And so, for the reasons given above, you are not likely to see successful SOA implementations involving such applications. This leaves room for SOA. But perhaps not for all the hype surrounding it as the next great thing in enterprise computing.

No comments: