SOA Case Study Contest Special Recognition Winner: FINRA for NYSE & NASD Member Regulation Merger

Earlier this week, the SOA Consortium announced that Financial Industry Regulatory Agency (FINRA) is the special recognition winner for Regulatory in the SOA Consortium | CIO magazine case study contest.  FINRA Case Highlights -- originally posted by me on SOA Consortium Insights -- follow. 

Organization Background

The Financial Industry Regulatory Authority (FINRA) is the largest independent regulator for all securities firms doing business in the United States.  Created in July 2007 through the consolidation of NASD and the member regulation, enforcement and arbitration functions of the New York Stock Exchange, FINRA is dedicated to investor protection and market integrity through effective and efficient regulation and complementary compliance and technology-based services.

Business Scenario

Merger/Consolidation: The Financial Industry Regulatory Authority (FINRA) SOA project consolidated the New York Stock Exchange Member Regulation systems with the NASD Member Regulation systems. FINRA is the largest private independent regulator for all securities firms doing business in the United States. FINRA oversees nearly 4,850 brokerage firms, about 173,000 branch offices and approximately 649,000 registered securities representatives.

The primary challenges:

1. Consolidation of the two organizations’ application portfolios that support the member regulation business.  Each application portfolio was sizable and heterogeneous. At the onset FINRA had ~160 applications and NYSE Member Regulation was supported by 86 applications.
2. Reconciliation of two sets of legacy business processes into a final-state business process.
3. Final-state business processes must seamlessly integrate new systems and existing systems from both legacy organizations. The existing systems required enhancements.
4. Business teams were distributed across the United States in district office locations.  The development team was located in New York City and the Washington D.C. area.

The objectives:

1. The final-state business processes of the merged company required seamless operation.
2. The team needed to ensure a continuity of business operations while transitioning in phases to the new final-state business processes.
3. Performance and reliability of the systems was a key requirement in maintaining core mission success.

Why SOA:

1. The size and complexity of the project required multiple teams in different locations working effectively in parallel to meet the aggressive schedule.
2. A SOA approach reduced risks presented by the large team size.
3. The end-state systems had to be flexible and provide the ability to quickly deploy changed and new business process without breaking the architecture.
4. It was anticipated that the approach would deliver significant savings in both cost and time when compared to competing approaches.

ROI

The Member Regulation function of FINRA (the new, merged regulator) benefited greatly from the new system.  Broker regulation tasks were simplified and accelerated, and delivered cost savings for the business.

The key business values achieved are:

Time to Market – Project delivery was greatly accelerated by allowing development teams to conduct parallel development of 10 major services with minimal interaction and dependencies. The service oriented approach and detailed overall vision allowed each team to rapidly deliver individual services that were seamlessly integrated and tested by the system team.

Reduced Risk – The SOA approach mitigated many of the risks associated with large development teams (100+ staff) by facilitating parallel development while minimizing team interdependencies and setting clear team responsibilities. The key to reducing risk is the early definition of business service interfaces and responsibilities.

Cost Savings – The modular SOA architecture of the new system consolidated business functions into a common set of business services that are leveraged across many business processes, resulting in cost savings for construction, deployment and maintenance of the system. 

Improved Agility – The business-centric service design and modularity of the SOA approach provides flexible deployment to support current business processes and to rapidly adapt to support future business process. Current business centric services include data sourcing, analytic surveillance and case management.

Resilience – Fault tolerant business-continuity is achieved using guaranteed message delivery, as individual business services are moved off-line for maintenance and restored.

Process Optimization – Technology duplication is eliminated through the consolidation of functionality into discrete standardized business services. This also provides a uniform approach and consistent results across all the business processes.

Project Organization

FINRA used a three-tier approach to organize the project:

Tier 1 was comprised of business analysts that spanned the various business processes.

Tier 2 consisted of 10 distinct service teams that defined the function of their service based on the business analysis and under the supervision and guidance of architecture and program management.

Each service team was individually sponsored by the organization and was a cross-functional team comprised of business analysts, architecture, development, and testing. The collaboration and alignment of technology and business staff on each individual service team was key to their success.

Tier 3 was comprised of the system architecture team and program management who provided overall vision, governance, and timeline. The overall vision provided business-centric services spanning many business processes. This naturally created emphasis on major business functions such as analytic surveillance and case management.

Lessons

1. The single most important lesson from this SOA project is that extremely large, time critical applications can utilize a SOA approach to segregate and compartmentalize common services and allow for massively parallel work by independent teams. Not only does this approach increase organizational productivity but also mitigates some of the risk presented by a large project.
2. The SOA approach gave the teams a measure of insulation, helping ensure decisions on one component did not negatively impact other components or the project.  The de-coupling allowed teams to deliver well-defined components on an aggressive time line required for project success.
3. Understanding the underlying business problems and processes is crucial to creating well-defined services that are reusable and exhibit the correct level of granularity.  The payoff for this is a flexible business process that can change and grow without changing the architecture
4. A concise architectural vision shared between system architects and application architects is key in large projects. Effective governance, along with well-defined services with clear functions and interfaces, is essential. Since the interfaces will change over time, it is important to develop a plan to handle this early.
5. For projects that use process orchestration, identify and document those processes early in the project.  This will help ensure that they operate in conjunction with the business function they automate and avoid problems that would be more costly and difficult to solve later in the project.
6. The combination of an ESB and BPM is a robust and powerful enterprise pattern. This combination greatly simplified several of the tasks of adapting existing capabilities into a new unified SOA system.

[Disclosures: The SOA Consortium is a client of my firm, Elemental Links. I was a contest judge.]