Cloud computing is a key driver for change in the existing IT infrastructure in organisations across the world. While cost reduction makes a strong business case for adoption of Cloud solutions, migrating to Cloud also affords significant advantages like leveraging of AI, ML and data analytics, increasing resilience and survivability of IT assets etc. It is these last two use cases that make Cloud adoption a necessity rather than an aspiration for the defence forces.
In India, the government embarked upon an ambitious Cloud initiative – called “MeghRaj” – in 2014 with the focus to accelerate delivery of e-services in the country. The Army Cloud, launched in November 2015, comprises a Central Data Centre and a Near Line Data Centre, both in Delhi, and a Disaster Recovery site for replication of its critical data. The Army is looking to expand its Cloud computing network for storing and sharing operational and personnel information such as deployments, movement of formations and health and service records. Similar initiatives are in hand in both the Indian Navy and the Indian Air Force.
While Indian defence forces actively explore the adoption of Cloud Computing and would no doubt be guided in their efforts by the policy reports prepared by the Ministry of Electronics & Information Technology (Meity) viz., “GI Cloud Strategic Direction Paper” and “GI Cloud Adoption and Implementation Roadmap”, it would be most instructive for them to analyse the US Department of Defence (DoD) Cloud initiative and draw lessons, both in technology and contract management, from its apparent failure.
JEDI Background
The US DoD issued a memorandum in 2017 that sought an early adoption of an enterprise wide Cloud services solution. The solution was to be based on commercially available Cloud services linking the warfighter with the Pentagon and was intended to support security classifications ranging from Unclassified to Top Secret. Named the Joint Enterprise Defense Infrastructure (JEDI) Cloud acquisition program, it was to be awarded for an initial period of two years with an estimated contract value of $1 million, extendable to a 10-year contract with a ceiling of $10 billion.
The contract drew interest from the marquee list of Cloud service providers that included Amazon, Oracle, IBM, Microsoft and Google. However, the program was to have a turbulent life. Google withdrew after protests from its employees on the grounds that the program conflicted with the company’s corporate values. IBM and Oracle filed pre-bid protests with the Government Accountability Office (GAO). Both the protests were dismissed by the GAO. Oracle then filed a bid protest in the US Court of Federal Claims (COFC) that was also rejected.
The aspiring contractors were not the only party contesting the contract. The program was also in the cross hairs of President Donald Trump over claims that the contract was tailor-made for Amazon. Trump, reportedly and publically, made a series of contentious statements and acerbic posts against Amazon and its founder Jeff Bezos, which were later to provide fuel for another, more substantial, bid protest.
The JEDI contract was awarded to Microsoft on October 25, 2019, but AWS (Amazon Web Services) challenged the award on November 22, 2019 in the COFC; its legal strategy included calling Trump to testify over allegations of influencing the JEDI bid evaluation and giving directions to DoD officials to “screw” Amazon out of the contract.
The COFC halted Microsoft’s work on the project on February 13, 2020, a day before the system was scheduled to go live, ruling that Amazon’s claims are reasonable and “likely to succeed on the merits of its argument that the DoD improperly evaluated” Microsoft’s offer. The JEDI contract with Microsoft was cancelled by the DoD on July 6, 2021 with the expectation that a new program called “Joint Warfighter Cloud Capability” would replace it, which would involve services from multiple vendors.
This series of articles examines the JEDI contract’s formulation, award and cancellation with the intention to provide insights into best practices and avoidable actions and omissions that would be helpful to the military technology and contract management professional.
“The greatest teacher, failure is… We are what they grow beyond.” – Yoda
Part I: Lessons in Technology Management
Lesson 1: Choice of Vendor Engagement Models – Single Cloud Service Provider Vs Multiple Cloud Service Providers
Prior to issuing the JEDI solicitation, the DoD carried out an extensive interaction with the Army, Navy, Air Force, Marine Corps, and other DoD Components, as well as industry information technology experts in order to understand the Cloud infrastructure requirements. The DoD weighed the options to source its cloud requirements from a single vendor or from multiple vendors. The decision to award the JEDI Cloud contract to a single contractor was heavily influenced by their interaction with the CIA. The CIA had forayed into the Cloud infrastructure space in 2014 with their Commercial Cloud Service (C2S) project, which had a single contractor. Using Cloud from a single contractor allowed the CIA to understand the nuances of Cloud management and by 2019 they had gained sufficient technical knowhow to go in for a multiple vendor Cloud solution. At the time of the JEDI contract solicitation, the DoD’s workforce lacked the technical proficiency to configure, secure, and optimize the benefits of Cloud computing. By acquiring an enterprise Cloud from a single contractor first, the DoD was following the CIA’s example. The JEDI Cloud Statement of Objectives explicitly stated that a single contractor JEDI Cloud would serve as a pathfinder for the DoD to understand how to deploy an enterprise Cloud at scale with security, governance, and modern architectures.
In the Indian context, while the defence services have some in-house capability in managing Cloud solutions, there exists a considerable capability gap to be able to successfully manage an enterprise-wide Cloud solution. The Ministry of Electronics & Information Technology (MeitY) has empanelled multiple Cloud Service Providers (CSPs) and government departments can procure the Cloud Services in 3 different engagement models:
- Procure Cloud Services directly from a CSP that already has in place an Implementing Agency or Internal IT Team or expertise that is responsible for managing Cloud resources.
- Procure Cloud Services through an MSP (Managed Service Provider) when it does not have an Implementing Agency or Internal IT Team or expertise that is responsible for managing Cloud resources. In this case, they procure the Cloud services from an MSP who will be responsible for managing the Cloud service offerings.
- Procurement of end-to-end services from a Systems Integrator (SI) when the Cloud services would be a part of the total services procured through an SI for a turnkey project implementation.
In the current situation, the SI/ MSP engagement route seems to be the best fit for the Defence Services. However, procurement of Cloud services through a MSP/ SI would come at an added cost and would bring with it its own contract management challenges. Thus it would be prudent for the Services to follow the JEDI example and utilize the SI/ MSP engagement model in the pilot phase and use this lead time to simultaneously build up the required in-house capability to become enabled to eventually manage an enterprise-wide Cloud independently.
Lesson 2: Area of Focus – Backbone Vs Tactical Edge
The DoD Cloud Strategy 2018 recognises that: “Data and our ability to process data at the ready are differentiators to ensure mission success.” The focus is equally on backbone networks and on tactical edge and mission needs. It seeks to empower the warfighter with data and enable informed and fast decision making in the battlefield.
“Cloud is a fundamental component of the global infrastructure that will empower the warfighter with data and is critical to maintaining our military’s technological advantage.”
By contrast, the Indian Army’s Cloud focus is still primarily on developing the backbone network, with extension of the Cloud to the tactical edge still some distance away. Adoption of a sequential approach in implementing Cloud infrastructure is a strategic and national risk as the military advantage enjoyed by our nation’s Armed Forces over our competing neighbors may decline rapidly as the adversaries develop their capabilities in this domain. For instance, in recent years the Chinese government has prioritised the development of cloud computing technology with the goal of expanding Chinese military and civilian access to Cloud computing information technology (IT) resources. India needs to leapfrog the technology ladder and bring in disruptive changes in rate of technology induction in the defence forces, to be able to retain its military advantage or strive for parity with its neighbours.
Lesson 3: Civil-Military Alignment
The US DoD JEDI solicitation saw participation from MNC giants like Amazon, IBM, Oracle and Microsoft. These companies are technological superpowers with economic heft that gives them substantial leverage in negotiations with governments and ensures they are beyond the arc of government oversight and control. To that extent, in Government to Business engagements, these companies can be treated and often conduct themselves as sovereign nations. They are likely to comply with the dictates of the government’s national strategy only as long as they are aligned with their economic interests. A case in point is the bid protest filed by AWS in the US COFC with the intention to call the US President as a witness an integral part of its legal strategy.
At the other end of the spectrum lies China. Three of China’s most prominent tech firms and cloud service providers – Alibaba, Tencent, and Baidu are entrenched in the Chinese government’s high-tech, dual-use innovation drive spanning areas like artificial intelligence, big data analysis, biometric modalities, and autonomous systems. The Chinese Military-Civil Fusion (MCF) Development Strategy implies that there is no clear demarcation between the PRC’s civilian and military economies and the technology companies have their survival and commercial interests intertwined with the national strategy. According to recent reports, the US DoD is considering expanding the list of Communist Chinese Military Companies (CCMCs) to include Alibaba Group and Tencent Holdings.
The Cloud technology landscape in India lies in between these two extremes. MeitY has empanelled 17 Cloud Services Providers (CSPs) to offer “Basic Cloud Services” to government organisations. These CSPs are a mix of MNCs like Google, Amazon, IBM, Oracle, Microsoft, Indian corporates like TCS and Reliance and PSUs like BSNL, ITI and Railtel, with varying degree of deference to national strategic interests expected. The interesting aspect is that the technological capability of these CSPs, as per popular perception, is inversely proportional to the likelihood of these companies subsuming their commercial interests when these are in conflict with the national strategic objectives. The Ministry of Defence will need to strike a fine balance in ensuring that the Armed forces’ Cloud capability is fully supported in all eventualities.
Part II: Lessons in Contract Management
The Dark Side: Conflict of Interest and Bias & Bad Faith
Among the many contentious issues surrounding the JEDI contract, the most scandalous were the ones involving allegations of Conflict of Interest and Bias & Bad Faith on part of the DoD officials involved in the bid evaluation process.
Lesson 1: Declare Conflict of Interest Upfront and in Full
The allegations of conflict of interest primarily concerned three former DoD employees, Deap Ubhi, former Product Manager, Defense Digital Service; Victor Gavin, former Deputy Assistant Secretary of the Navy for Command, Control, Communications, Computers, Intelligence, Information Operations and Space; and Stacy Cummings, Principal Deputy Assistant Secretary of Defense for Acquisition and Deputy Assistant Secretary of Defense for Acquisition Enablers. Each of these employees was involved to a greater or lesser degree in the early development of the JEDI program. Two of these former DoD employees, i.e. Gavin and Ubhi, were in talks with Amazon Web Services (AWS) for employment, while Stacy Cummings owned stocks in Microsoft at the time they were working on the JEDI project. While the COFC opinion in the protest lodged by Oracle cleared DoD of allegations of conflicts of interest, the persons involved did commit ethical violations by misrepresenting facts and not declaring conflict of interest upfront and in full. Such lapses can be seized upon by belligerent parties for their vested interests and could scuttle a procurement process.
Lesson 2: Beware of Bias and Bad Faith
AWS protested the decision to award the Contract to Microsoft in the Court of Federal Claims, arguing, among other things, that the Company was not selected due to “improper pressure” on the DoD by President Trump. Trump had suggested that he could intervene in the decision-making process for the contract, citing complaints from other competitors that the evaluation was favouring AWS unfairly. His comments had led Republican lawmakers in the House Armed Services Committee to send a letter warning Trump against intervening.
Trump carried out a very vocal and public tirade against Amazon and Jeff Bezos through a series of tweets and media comments. Articles in news papers, including The Washington Post, generated allegations that Trump may have influenced Secretary Mattis, Secretary Esper, and DoD JEDI Cloud source selection officials to prevent Amazon from winning the contract. Also in the summer of 2018, a former member of the then Defence Secretary’s staff wrote a book in which he stated that Trump had called Defence Secretary Mattis and told him to “screw Amazon” out of a chance to bid on the contract.
While the DoD OIG inquiry concluded that the DoD personnel who evaluated the contract proposals and awarded Microsoft the JEDI Cloud contract were not pressured by DoD or the White House, the US COFC declined to dismiss Amazon’s allegations of bias and bad faith. At the very least, the very public posturing by Trump did provide grounds for AWS to prepare their legal strategy to seek President Trump to testify in Court, stating that, “Basic justice requires reevaluation of proposals and a new award decision. The stakes are high. The question is whether the President of the United States should be allowed to use the budget of DoD to pursue his own personal and political ends.”
Personal conduct of public officials can at times be used by parties dissatisfied by the source selection procedure to substantiate their claims of bias and bad faith in award of a contract. Public office-bearers need to be wary of this and ensure that they are not only impartial but also appear to be impartial when handling high stakes public procurement.
Lesson 3: Stress and Communication Gap, a Recipe for Disaster Make
The DoD contract specialist working on the JEDI inadvertently shared proprietary information related to the Microsoft bid with AWS as part of the written debriefing provided to AWS. This allowed AWS to carry out a point by point comparison of its bid with the winning bid of Microsoft and use the comparison in its merit based bid protest. The debrief also contained un-redacted names of DoD employees involved in the JEDI procurement process. The error was due to cumulative factors like the communication gap between the contracting team and the White House Service, which provided legal advice on the procurement process, inadequate supervision by the Procuring Contract Officer, pressure to release the written debrief at the same time as the contract award announcement and technical glitches on the award announcement day. The contract professional would identify all or most of these factors existing in his/ her work place and reading about the JEDI episode would hopefully serve to reinforce the need to exercise caution.
The Federal Acquisition Regulation (FAR), the primary regulation governing procurements by all US government agencies, requires contracting agencies to provide information to the contractors about the selection decision and contract award.
Considering the magnitude and importance of the contract, the award debriefing was reviewed by the White House Service counsels who represent DoD in litigation. However, there was a communication gap between the WHS counsel and the contract specialist regarding the documents to be included in the debriefing to AWS. The contract specialist interpreted the WHS recommendation to include “evaluation reports” in the debriefing to AWS, to mean all the reports including AWS and Microsoft reports, whereas the intention was to only release AWS reports to AWS. He assumed that the WHS counsel would review the draft email to AWS with the attachments that the contract specialist shared with the WHS Counsel. However, the email sent by the contract specialist to WHS for review did not include any attachments, thus the WHS counsel was not able to review the documents that the contract specialist intended to share with AWS. The DoD Inspector General report on the JEDI contract states: “The WHS counsel… was unable to review the documents that the contract specialist planned to provide to AWS. However, he assumed that an experienced contract specialist would be aware of which documents to share and which not to share.”
The DoD departmental inquiry on JEDI reported that: “…the decision to notify the unsuccessful offer or and simultaneously provide a written debriefing for the strategic purpose of starting the protest clock immediately created extraordinary pressure on the contracting team and did not allow for the proper amount of time that all the tasks required.” The fallout was to compound an already complicated situation and contribute in some measure to the JEDI contract award being challenged.
Large and prestigious contracts require coordinated efforts from multiple teams within an organisation and even across organisations. There is tremendous pressure created due to tight timelines, the need to maintain confidentiality of information and the knowledge that there will be inevitable complaints to departmental and central monitoring mechanisms like the Oversight Committees and the Central Vigilance Commission. Communication gap and compartmentalised functioning can exacerbate an already delicate situation. The Project Management Office leadership must be wary of these existential challenges and meet them proactively.
Conclusion
Technology induction into hierarchical organisations with relatively rigid procurement procedures like government bodies and armed forces needs equal focus on the contract management and technology management aspects. The JEDI Cloud solicitation by the US DoD amply illustrates the pitfalls of paying inadequate attention to the contract management aspects that can negate even the most sophisticated technology management capabilities and derail the technology induction drive.
-The writer is a contract and technology management professional with more than twenty five years of experience. He has contributed to the strategic management of core technology projects of Aadhaar, the world’s largest biometric based identity system. The views expressed are personal and do not necessarily reflect the views of Raksha Anirveda
Bibliography
Inspector General, USA Department of Defence, Report on the Joint Enterprise Defense Infrastructure (JEDI) Cloud Procurement.
Congressional Research Service report “The Department of Defense’s JEDI Cloud Program”, August 2, 2019.
Amazon Web Service Incorporated versus United States of America and Microsoft Corporation, Bid Protest Case 1:19-cv-01796-PEC, in the United States Court of Federal Claims.
Opinion and Order, No. 19-1796C by Judge CAMPBELL-SMITH, in Bid-Protest; Motion to Dismiss; RCFC 12(b)(6), Amazon Web Service Incorporated versus United States of America and Microsoft Corporation.
https://www.gatewayhouse.in/national-Cloud-ai-defence/
How much disruption? Deloitte Global Outsourcing Survey 2020
Guidelines for Enablement of Government Departments for Adoption of Cloud, Ministry of Electronics & Information Technology, Government of India.
GI Cloud Reference Architecture, Ministry of Electronics & Information Technology, Government of India.
Guidelines for Procurement of Cloud Services, Ministry of Electronics & Information Technology, Government of India.
USA DoD, Office of The Secretary of Defence, Annual Report to Congress: Military and Security Developments Involving the People’s Republic of China.
Red Cloud Rising: Cloud Computing in China by Leigh Ann Ragland, Joseph McReynolds, Matthew Southerland, and James Mulvenon Research Report Prepared on Behalf of the U.S.-China Economic and Security Review Commission September 5, 2013.
Examples of Chinese Military ties to Alibaba, Tencent, and Baidu, January 12, 2021, RWR Advisory Group LLC.
