InfoWorld SOA Exec Forum: Key Takeaways from Enterprise (Real) SOA Practitioners

Earlier this month, I had the opportunity to attend InfoWorld’s SOA Executive Forum in New York.  The SOA Executive Forum brings together enterprise SOA practitioners and contemplators, SOA industry experts, and leading SOA practice and infrastructure vendors. Through formal presentations, real-world case studies, panel discussions, a solution showcase, and table and hallway conversations, enterprise business and technology professionals learn about SOA­­––the opportunities, risks, architecture, technologies, products and practices––in order to define, or refine, their own SOA strategy.

While the forum is vendor sponsored––by BEA, HP, IBM, Intel, SOA Leaders, Cisco, Infravio, InterSystems, Layer7, and SOA Software––there was a good balance of enterprise and vendor presentations, and even some mixed panels.

The enterprise presenters included individuals from organizations such as Merrill Lynch, Verizon, Sprint-Nextel, Guardian Life Insurance Company of America, The Hartford, Ohio State University Medical Center, SABRE, Pfizer, the U.S. Army, and the District of Columbia. [Kudos to InfoWorld for recruiting a great, diverse lineup to demonstrate there is more than one way to realize SOA.]

One thing that was abundantly clear to me (and my table mates), is there is an obvious disparity between the architecture origins of SOA, and the marketing messages of SOA infrastructure providers, who claim the answer to SOA is in infrastructure (registry, repository, Web Services infrastructure, SOA fabric and/or enterprise service bus).

Those of us who have pursued SOA in the enterprise know that SOA is an architectural strategy, enabled by products, but not centered on them. As Martin Brodbeck, Pfizer’s director of Global Application Architecture, stated in a session: “SOA is a loosely-coupled architectural strategy, and needs to be both technology and vendor independent.” 

The enterprise participation and perspective is what drew me to the event. Because the best way to learn what real people are doing, is to actually go talk to them! 

Last week, I published a (free with registration) report - Best Practices, Lessons Learned, and Takeaways from Enterprise SOA Practitioners - through the PSGroup Research Service based on content (presentations and conversations) from the SOA Exec Forum.  

In this post, I excerpt some key takeaways – all from enterprise practitioners. Before I list the takeaways, let me share my report’s closing thoughts, because I think they are important:  

The enterprise presentations proved that SOA is real. Enterprises are using SOA to provide both agility and productivity for business and IT. The secret is to approach SOA as an architectural strategy. In such, make decisions based on your business, technology portfolios, and people. 

Don’t be misled by vendors who claim SOA can be bought in a box. Don’t be misled by technology snobs who claim SOA is Web Services. Remember loose-coupling, well-defined interfaces, and standards based messages. 

If you have questions on your strategy, speak to a trusted technology- and vendor-neutral advisor, and network with other enterprises. For a healthy dose of reality, attend sessions like InfoWorld’s SOA Executive Forum. 

Excerpt: Takeaways from Enterprise SOA Practitioners 

1. How Is SOA Being Justified?

When justifying SOA, the presenters used the terms re-use, agility, and productivity as follows.
 
Re-Use. SOA purists speak to re-use solely in respect to the service itself. If you create a customer account service that three lines of business use, across three channels, then you have achieved significant re-use payback. Many vendors at the forum took this purist stance. 

The IT presenters at the SOA Forum also spoke to the re-use of the providing asset. So, if you create a customer account service, which allows you to extend a legacy application asset to the Web that counts as re-use. This is true even if only one Web application consumes that service. Presenters spoke to the value of “liberating existing data.” 

A variation of re-use is shared services. In a shared service situation, an organization creates a service for external use. Consumers of a shared service may be a customer, a partner, or in the case of Google, Amazon, and Yahoo!, the great unwashed.

Agility. On the business side, agility refers to the composable nature of services. If you can easily add, remove, augment, or tweak applications and processes––using services and metadata-driven business rules, orchestrations, and policies––then you can be extremely responsive to your business. 

On the IT side, agility often refers to provider layer. The loosely-coupled nature of services and providers allows for the non-invasive replacement of underlying providers, such as application packages, legacy code, and databases. 

Productivity. Obviously, there are IT productivity gains from re-use. A couple of the presenters also called out reduced maintenance costs, since a loosely coupled solution is easier to refresh, change, and extend over time.

However, the presenters cautioned the audience not to expect immediate productivity gains. In fact, the initial transition to SOA will cause a productivity decrease, as you build out an SOA environment and skills. 

2. The Architecture in SOA Is Enterprise Architecture

All too often, the industry hype portrays SOA as Web Services-based application development. In actuality, SOA is an architectural strategy––an enterprise architectural strategy––that relies on good architectural foundations for success. The enterprise presenters made this point very clear. Two architectural dependencies the majority of the presenters called out are:

Business Architecture. Services and compositions reflect your business architecture, not your existing application portfolio or code base. You need to understand your business domains, functions, rules, processes, interactions, offers (products and services), and events in business terms. 

Information Architecture. If your information architecture is a mess, your SOA will be a mess. Every enterprise should have a canonical data model that is the definitive source of key data terms, definitions, and relationships. In a perfect world, this canonical model would be your implementation schematic. But, in the real world, the canonical model drives your semantic interpretation and data translation strategy, so services don’t just message each other, but communicate. 

Some presenters had adopted industry standard dialects, such as ACORD and XBRL, while others relied on in-house agreed-upon terms. The District of Columbia incorporated semantic Web technology, resource description framework (RDF), in its information definition and interpretation strategy.

3. Keys to Success in Your SOA Journey

The keys to success were amazingly consistent, from presenter to presenter. Not surprising, the overarching message was SOA is a journey. Success requires incremental pursuit and measurable business returns. 

SOA Strategy/Program Execution

• SOA = Architecture. Recognize that SOA is an enterprise architectural strategy, not merely an application development model. See above.

• Start Small. Pick a project that can have a measurable return, but poses no great business risk. First projects often connect the disconnected, or liberate information.

• Grow Organically. Encourage organic growth of SOA from one or two top development teams. Pick teams whose members are good technicians, want to follow standards, like to work smarter, and are influencers. Don’t start with an architectural mandate.

• Exploit Business Knowledge. Find an “uber business analyst” to be part of your long-term SOA competency center. This individual knows your business really well and has good enough knowledge of technology. This individual will facilitate the requisite service and business process definition, leading your organization away from business as usual to agility.

• Evangelize. Evangelize based on your initial successes. Spread SOA education to business partners and IT professionals.

• Govern. Establish governance and coordination mechanisms to optimize SOA. Share practices, services, and infrastructure. Coordinate service, project, and infrastructure development plans.

• Measure. Measure your results. Merrill Lynch includes re-use in a return on asset (ROA) measurement, and agility (time to market gains) in a return on investment (ROI) measurement.

Technical Execution

• Create a Strong Service Catalog. Define your services in business terms. Create course-grained services from compositions of fine-grained services.

• Communicate Your Service Catalog. Establish a repository that is the “trusted place to go” for service descriptions, interfaces, bindings and policies. The repository could be as simple as a spreadsheet, or a Wiki, or a commercial repository solution.

• Start Simple. Initial services are typically stateless and fine grained. With simple services in place, compose larger-grained, stateless services. Then, proceed to large-grained, stateful services.

• Common Practice. Services are “discovered” at design time, not runtime. During design, developers learn of a customer account service, and the interface and invocation requirements. Enterprises are not dynamically discovering and invoking services at runtime.

• Best Practice. From the onset, leverage mediation techniques. Mediators sit in between requesters and services. Use mediation to abstract common, crosscutting functionality, such as logging, security and protocol transaction out of your services.

Typically, mediation is performed by SOA and/or integration infrastructure. Examples mentioned in the presentations were SOA Fabric (Blue Titan), SOA Security Appliance (Reactivity, Layer7), enterprise service bus (Sonic, BEA’s AquaLogic) and EAI solution (WebSphere Message Broker).

• Be Standards Based. Be Smart. There was overwhelming agreement that standards are critical to service composition and communication, and infrastructure interoperability. Specific to services, the following themes emerged:

- XML and HTTP are the most important standards. All presenters incorporated XML in messaging.

- Web Services (SOAP and WSDL) are important, but not a requirement. For the sake of performance and ease of use, many implementers offer more than one technical implementation model for the same service. The models were Web Services, REST (XML/HTTP) style services, Java services, and .Net services.

- Of the advanced Web Service standards (WS-*), the most common implementations are WS-Security and BPEL.

- There was one example, from the Ohio State University Medical Center, that used a broad contingent of advanced Web Services standards, including WS-Security, WS-Policy, WS-Trust, WS-SecureConversation and WS-ReliableMessaging. The initial implementation used Microsoft’s WSE 2.0. The current implementation uses Microsoft’s Indigo.

•  Abstract Complexity. Leverage tooling and frameworks to abstract technical complexity away from developers and assemblers. This complexity includes standards implementations, technical mediation, and resource integration.

• Remember Human Element. Don’t forget the human element of SOA. People interact with services, business processes, and events through portals, rich interfaces, and subscriptions (RSS), across a range of devices: PC/browser, handhelds, and pagers.

4. Warnings and Lessons Learned

Although the presenters are successful advocates for SOA, they each faced challenges along the way. Here are some warnings and lessons learned.

Be Prepared For:

• Business Executive Skepticism. Business executives still have scars from previous IT paradigm shifts. Be prepared to face resistance. Instead of arming yourself with slides and marketure, bring tangible results. Implement a small services project—perhaps just one service connecting previous disconnected channels or domains, and share the results. Explain how SOA is a foundation, and incremental investment brings incremental gains.

• Development Community Backlash. Developers often rebel against the perceived restrictiveness of standards, and the not-invented-here (NIH) nature of services. To counteract this, educate that standards actually promote innovation. By not worrying about the “framing and plumbing,” developers can focus on the business problem at hand, and bring their creativity to bear for business advantage.

Regarding NIH syndrome, make your service developers “heroes.” As we’ve learned from open source, adoption (and acceptance) of a developer’s code, leads to community credibility and status.

• Business Community Pushback. Business agility, often through business process changes, threatens the norm, and jobs. Be sensitive to the implications. Collaborate with your business to facilitate business change; don’t define the business change from IT.

Don’t Underestimate:

• The Three C’s. SOA success requires communication, coordination, and cooperation within IT, across business units, and between IT and the business.

• Operations Impact. SOA is a hyper-distributed, horizontal architecture. Systems management and monitoring solutions are vertical or agent based. Production turnovers must contain a step to educate and hand-off to operations. Provide documentation and insight on the components (infrastructure, application, and data) and interactions required for your service-oriented business solution.

Keep in Mind:

• The SOA Readiness of Your IT Partners. Individuals working for systems integrators and development outsourcing shops do not have SOA training or experience. In outsourcing situations, you need to control the architecture and be very specific in your requirements. The output you will receive is exactly what you ask for!

• Business Process Outsourcing and SOA. If you are considering business process outsourcing (BPO), make sure your provider has an SOA strategy. If not, you will be locked out of agility, or find yourself in renegotiations.

5. Help Move the Industry Forward

The enterprise presenters had some requests of the audience to advance SOA, and more importantly, the promises of agility and productivity. I share those here, and encourage readers to help move the industry forward.

Push Your Vendors:

• Demand Service-Oriented Solutions from Your Software Vendors. An example cited was Siebel’s new Component Assembly offering.

• Demand Standards-Based Infrastructure Offerings from Your Vendors. In a service-oriented world, the infrastructure layer is also a loosely-coupled, business-driven composition. As stated by more than one presenter, “The best vendors make themselves replaceable.” 

Use Your Influence:

• Influence Standards Bodies through Your Vendor Partners. Articulate your requirements, but do not expend valuable time and resources sitting on standards committees.

• Influence College Curriculums to Teach Beyond Traditional Algorithms. The real world is composition, mix and match, and service oriented. If they balk, refer them to Ray Ozzie’s Internet Services Disruption memo. 

Comments

[Anil John]

Brenda, you mentioned ".. Use mediation to abstract common, crosscutting functionality, such as logging, security and protocol transaction out of your services" as a best practice.

What is the canonical descripiton (if there is one) of Mediation?

I've always thought of Mediation as data and protocol transformation. The majority of the items that you noted above (logging, security etc), I have always considered re-usable service functionality.

I completely agree with the intent here i.e. abstracting common functionality out to make it reusable across the Enteprise, but am seeking clarification on the usage of the terms.

[brenda]

Anil, Good question. There are an over abundance of terminology challenges related to SOA.

I'm using the term "mediation" in a broad architectural pattern sense. You could also substitute the term "interception". In this sense, mediation refers to having functionality that sits outside of a service be non-invasively applied within a service invocation in order for a request to be completed. The mediated work can be technical and data protocol transformation (as you mention) or logging and security (as I mention). I would expect the logging and security to be (as you say) implemented as services. -Brenda

[Richard Akerman]

Great summary, very information-dense.
I saw many of the same themes when I presented at an SOA Symposium in Manitoba.

http://scilib.typepad.com/science_library_pad/2005/12/presentations_f.html