Sunday, 30 March 2014 00:00

Case Study: $250 Million Failure, Who Are You Going To Sue?

Rate this item
(1 Vote)

In order to comply with the Affordable Care Act, the State of Oregon made the decision to build its own Health Insurance Exchange (ORHIX). An online portal to allow applicants was supposed to go live October 1, 2013. As of March 30, 2014 the site was not functional and all ORHIX applications must be processed from paper applications.

This is one of two Case Studies on recent government project failures that I was asked to review. I am providing this assessment as a public service. The other assessment is on the Massachusetts Division of Unemployment Assistance.

Analysis

The in-depth analysis is restricted to the September 2012 Oracle contracts since that is the only set of original documents with enough data to make an analysis. Two audits reports were also read—one from Centers for Medicare & Medicaid Services (CMS) and the other from FirstData—and Maximus' two quality reports from November and December.

Discrepancy in PO Values (missing)

Figure 1: Discrepancy in PO Values

Lack of Clarity in POs

The September contract for Oracle was for $35.1MM with a very clear assignment to "assist" Oregon on a (T&M) basis to build the ORHIX system. This amount was for an estimated 118,477 hours of services totaling $35,094,504. There are, however, irreconcilable differences on this PO. The PO body has a narrative that defines the work is being contracted. The narrative list defines one extra area of work and the summary hours are inconsistent with those in the subsequent hourly estimates table (even though the grand total of hours for both sets are the same 188,477-hour total). When calculating the value of the PO using the labor rate and estimated hours tables found in the PO, the value should be $34,139,415.62—$955,088.38 less than what was paid to Oracle. Due to the lack of access to the Cover Oregon staff, this is unexplained.

Project Structure

The project had a number of root issues that gravely affected its ability to complete successfully. From my analysis, there were four major issues:

Oracle Contract page (missing)

Figure 2: September SOW with focus on "Assist"

  1. Hiring Oracle on a T&M basis. Time and materials contract placed all the risk for completing the project on the State of Oregon. Oracle's September 2012 contract has very little scope that they had to provide. A large majority of their service delivery lines start with the word "assist" (see Figure 2). They are clearly subservient to either the State of Oregon or a System Integrator (who ended up being the State). Oracle's contract should have been fixed-price for specific deliverables.
  2. Failing to restrict the scope to what was essential to meet the go-live requirements for October 1, 2013. The scope of the project, according to the two independent audits, was not only to develop the ORHIX, but also to include the DHS Modernization project. This created the MaX project. The decision should have been to minimize scope by trimming it to a minimally viable product for the ORHIX go-live in October 1, 2013. The DHS Modernization should have been postponed to a subsequent year and a roadmap developed to define the phase-in goals for it after a successful go-live and system stabilization.
  3. Failing to hire a system integrator with large-scale system integration experience. Projects to build systems of this size are rare. Their chances for success are low regardless of who builds them. Trying to be the integrator with little or no experience has almost no chance of success. A system integrator should have been hired on a fixed-price contract to run and manage the project.
  4. Inadequate project governance and oversight. Numerous issues raise questions on the governance of the project from only looking at the minimal contractual documents that were available. These include:
    • Lack of reaction to the Maximus reports. It is unclear how the Maximus reports could go unheeded. This underscores, however, the most serious flaws in communication, cooperation, and understanding of reality of the project.
    • Hiring Oracle on T&M. As mentioned above Hiring Oracle on T&M removed all accountability. With a contract that primary had Oracle in an assisting role gave them less accountability.
    • Use of the Dell Contract process. It is not understood why the contracts were processed through Dell (these were service contracts processed through a hardware purchasing agreement). Accord to the other audits, the practice of putting these contracts through Dell continued. This only added cost to the project.

The CMS and FirstData audits point out many other areas where governance and communication fell short. This is a major issue that must be resolved prior to moving forward with this project or others.

Maximus Quality Reports

Maximus Quality Summary (missing)

Figure 3: Maximus Quality Summary

Oregon hired Maximus to run monthly audits of the ORHIX project (Maximus refers to these as quality reports). These reports started in 2011, but only two reports were available to review—November and December of 2013. They show multiple areas where the project at high risk for protracted lengths of time. They clearly highlight and document issues early in the project. Figure 2 is one page from the December 2013 report that clearly indicates a history of issues on the project.

These reports underscore a lack of project control since they clearly show the project was in trouble from early 2013. Audits conducted by CMS and FirstData confirm that the project's oversight team failed to heed Maximus' warnings. According to FirstData, "multiple members of the [Legislative Oversight] committee told us they were completely unaware of the Maximus QA role and had not received any of the QA reports".Audit ReportsAs mentioned above, two different organizations performed audits on the project in 2014—after it was clear that the project was seriously missing its goals. These reports add to the findings here. Neither was commissioned to find root causes (CMS was a Federal audit to assess the state of the project, and FirstData was to answer seven specific questions). Both reports are very good, but stop at charting a course to prevent re-occurrence of the problem on this or other projects (this was not in their scope).No reports can be complete, and, although very valuable, FirstData's points out a couple inconsistencies that should be investigated. Three of these are:

  • They report that the decision was made to not use a system integrator in 2011. However, the contract for services with Oracle in September, 2012, clearly states:
    "[Oracle will] Assist the SI Vendor with Architecture, Design Guidelines to build interfaces to DHS systems, Federal Systems, and third-party systems for the following: "1. Real time integrations using the Oracle SOA Suite. "2. Web Services "3. Batch interfaces"
    If the decision had been made a year earlier, why would Oracle say this when the rest of the document clearly talks about work with the State?
  • The list of POs in their report does not include a PO for a Cloud Services contract signed on August 21, 2013. This is a three-year contract for $22MM of Cloud Services.
  • The completion of a complete financial audit on the project. Aside from the discrepancy noted above, FirstData enumerates 43 POs that were issued to Oracle. There are references, however, to another contracted amount prior to September 2012 that is not reflected in their list. Notably, the detail in the Dell Quote BPB11111715-R07 (Addendum Two of Oracle License and Service Agreement US-TERM-GMA-5772-30-NOV-2011) refers to an original contract in the value of $7,241,836. The POs listed by FirstData cannot be reconciled to this amount.

Recommendations

In my opinion, although Oracle is culpable in the failure, they were on a T&M contract and it was the State of Oregon and Cover Oregon's lack of governance and controls that is the biggest issue. The investigation into what happened needs to move on to find the root causes and solve them. That must to be done before any further money is spent on completing the ORHIX project or defining an alternative option. The two steps are:

Table 1: Documents Reviewed
Known or Referenced POs and Contracts
PO Date Description PO Contract Value
11/2011 Unidentified services. Document not seen. Ref US-TERM-GMA-5772-30-NOV-2011   X $7,241,836
9/2012 Services Contract. Addendum to US-TERM-GMA-5772-30-NOV-2011 X X $35,094,504
3/14/2013 Software Licenses   X Not Provided
8/21/2013 Cloud services (3-year)
  Core service   X $15,493,111
  Service Option Fees   X $3,528,848
  Enhanced security (8/1/13-7/31/14)   X $2,128,815
Total of POs listed: $63,487,114
  • A complete financial audit to show the actual and complete cost of the system. This should include money spent on ORHIX, DHS Modernization, and MaX (the combined projects). It should itemize all internal and external costs (not just Oracle).
  • A root cause analysis focused on finding solutions, rather than blame. The FirstData analysis, although well done, was commissioned to answers to seven questions, which appear to be looking for blame. Blame does not solve the problem. It is only political fodder. To find answers an audit needs to look at the following:
    1. What were the processes for each of these roles and what changes need to be implemented to make the role effective:
      1. Scope definition.
      2. Project planning (Schedules, project methodology, risk assessment, mitigation planning, assumption tracking, communications, etc.).
      3. Inter-agency and project communication.
      4. Issue resolution and escalation.
      5. System architecture and validation.
      6. System and interface development.
      7. Project Quality: System testing, test definition and approval, and regression testing.
      8. System deployment and training.
      9. Vendor selection, due diligence, and contract review.
      10. Task/resource allocation.
    2. What was the plan around project governance and what is an appropriate plan to make it work?

Constraints on Analysis

According to the March 19, 2014 Assessment Report completed by FirstData, there were 43 POs issued to Oracle with a value of $132MM. There were other vendors and subcontractors involved in the Cover Oregon project. The value of their POs and the State's internal costs are unknown. My analysis was conducted using a small subset of all contractual documents along with assessments from Maximus, CMS, and First Data. No one from the State or Oracle was consulted in making this assessment or its recommendations. This is a serious constraint on this analysis and hence it only represents are a small part of what needs to be done.The POs reviewed include Oracle's September 2012 time and materials (T&M) services agreement and the August 2013 Cloud Services purchase orders (POs). The value of these POs is $56.2MM. The former references an initial PO for $7.2MM, but it was not reviewed. No POs were seen for the software purchase; however, I reviewed the Oracle Software License agreement that listed the packages and seat counts. Following is a summary of these items.Conflict of Interest DisclaimerThis was a non-paid engagement and neither eCameron nor myself has received any form of remuneration for this effort. The genesis of the investigation based on a request from a reporter at the Oregonian who asked for an interpretation of the material that he provided. This assessment is based totally on those documents.With these items completed and addressed, this and other large projects can proceed.

Read 27089 times

Related items

  • Filling Execution Gaps: How Executives and Project Managers Turn Corporate Strategy into Successful Projects
    What Filling Execution Gaps Covers

    Filling Execution Gaps

    by Todd C. Williams
    ISBN: 978-1-5015-0640-6
    De G Press (DeGruyter), September 2017

    Project alignment, executive sponsorship, change management, governance, leadership, and common understanding. These six business issues are topics of daily discussions between executives, middle management, and project managers; they are the pivotal problems plaguing transformational leadership. Any one of these six, when improperly addressed, will hex a project's chances for success. And, they do—daily—destroying the ability companies to turn vision into value.

    Check it out on Amazon or the Filling Execution Gaps website

    Without the foundation of a common understanding of goals and core concepts, such as value being critical to success, communication stops and projects fail.

    Without change management, users fail to adopt project deliverables, value is lost, and projects fail.

    Without maintaining alignment between corporate goals and projects, projects miss their value targets and projects fail.

    Without an engaged executive sponsor, scope increases, goals drift, chaos reigns, value is lost, and projects fail.

    Without enough governance, critical connections are not made, steps are ignored, value is overlooked, and projects fail.

    Too much governance slows progress, companies cannot respond to business pressures, value drowns in bureaucracy, and projects fail.

    Without strong leadership defining the vision and value, goals are not set, essential relationships do not form, teams do not develop, essential decisions are not made, and projects fail.

  • Filling Execution Gaps: Building Success-Focused Organizations

    Executives define vision, strategy, and goals to advance the business. Projects enable companies to meet those goals. Between strategy and projects, there is a lot of work to be done—work that lays the foundation for project and operational success. Through experience and research, six common gaps exist in organizations that inhibit project success—an absence of common understanding, disengaged executive sponsors, misalignment with goals, poor change management, ineffective governance, and lackluster leadership.

  • Strategy-Execution Gaps

    The statistics on strategy execution are dismal:

    • 59% of middle managers fail at resolving conflicts in corporate strategy.
    • 45% of middle managers cannot name one of the top five corporate goals.
    • 64% of cross-department/functional issues are poorly resolved.

    And maybe as you could expect from this:

    • 53% of companies cannot react timely to new opportunities.

    You do not need to be a rocket scientist to know that this trajectory is not going to launch most companies’ latest strategic plans successfully. In fact, these data might make you feel that middle management would be better suited as test dummies for the next generation of manned space-vehicle. Granted, the data show there is a dearth of leadership in middle management, but the executive tier has a culpable hand.

  • Process Mapping

    Process is at the core of any business. It makes work predictable, repeatable, and transferable. Without it we cannot scale our businesses. However, process can be a bane to making progress. Processes that work for a $10 million company have difficulties supporting a $30 million company. Trying to scale them to a $300 million company will not only fail but not address the issues that larger companies have that were never dreamt of in a smaller organization. Processes need to be discarded, revamped, and built—all of that without creating an overburdening bureaucracy.

    Anytime you need to go someplace, you first have to know where you are. Processes are never static and your company's current state is probably far from where you think it is. Hence, the first step is mapping out you company's current state followed by defining the future state. This is more than a logical map of the process; it must also include physical maps. Whether your process is solely to provide a service (say, website development) or physical (say, manufacturing) there are logistical issues that complicate the process flow. Without fully understanding those nuances, future state processes will not reach the desired efficiencies.

    For more information about process mapping fill out the form to the left and we will get in touch with you.

  • Success vs Culture

    The other day a Latvian student contacted me for my views the connection between culture and success criteria—an important and intriguing topic. After working in Taiwan, Singapore, Korea, Japan, Israel, United States, and Canada, I wear many scars of both blatant and subtle cultural violations. I also know that within a culture one person's success is often another person's failure. So, after dispelling concerns about clicking on some random email link, I completed her survey (please feel free to take it yourself). In the process, I struck up a friendship with the student, Kristine Briežkalne, who is studying at Riga International School of Economics and Business Administration . She has some interesting views and presented me with a Venn diagram showing four frames to a project (business, client, project management, and growth perspectives) and how they intersected. As the diagram is part of her Master's thesis, I will let you ponder the how to label the overlapping areas (an eye-opening exercise).

Leave a comment

Filling Execution Gaps

Available Worldwide

Filling Exectution Gaps cover

Filling Execution Gaps is available worldwide. Below are some options.

 

PG DirectLogo
Limited Time Price $20.99
Amazon logo
Book or Kindle
Flag of the United States Canadian Flag Flag of the United Kingdom Irish Flag Deutsche Flagge
Drapeau Français Bandiera Italiana PRC flag
Japanese flag
Bandera de España
Flag of India
Bandera de México
Bandeira do Brasil
Flag of Australia
Vlag van Nederland
DeG Press Logo
Barnes and Noble Logo
Books a Million Logo
Booktopia Logo
Worldwide: Many other
book sellers worldwide.

Rescue The Problem Project

Internationally acclaimed

Image of RPP

For a signed and personalized copy in the US visit the our eCommerce website.

Amazon logo
Buy it in the United States Buy it in Canada Buy it in the United Kingdom
Buy it in Ireland Buy it in Germany Buy it in France
Buy it in Italy Buy it in the PRC
Buy it in Japan
Book sellers worldwide.

Upcoming Events

Other's References

More Info on Project Recovery

Tell me More!

Please send me more information
on fixing a failing project.

Sitemap