Making the Move: Key Lessons from Nasdaq MRX Cloud Migration

Nasdaq has reached a new milestone in its cloud journey as the MRX options market has fully migrated to the Amazon Web Services (AWS) cloud, leveraging co-engineered AWS Outposts. With this move, we are paving the way for the next generation of capital markets to enable greater scalability when managing unpredictable events while providing greater flexibility and resiliency.   

The completion of the migration comes one year after we announced our plans to reimagine the future of the capital markets by moving our North American markets to the cloud. It builds upon Nasdaq’s decade of work gradually and systematically moving our foundational systems to the cloud, including data distribution, revenue management, regulatory reporting and market operations systems.   

Committing to the Cloud 

While our cloud journey began in 2008, the global coronavirus pandemic solidified our conviction that we needed to accelerate the realization of our cloud ambitions and begin migrating core markets to the cloud. In 2020, our data volumes reached new records, with 60 billion messages moving through our market systems and over 200 billion records into our Nasdaq Data Warehouse, up from an average of 70 billion per day. Many of our exchange technology customers also experienced a significant surge in trading volumes, with some increasing by over 100%.

Amid this market environment with heightened, sustained volatility, we recognized the need for dynamic capacity and order flow, as well as the need to have a more geographically portable business and the ability to interact with customers in new, digitalized ways. With the cloud, we can add capacity whenever needed without impacting other workloads critical to our day-to-day operations. We can also help our customers get greater control over their service consumption with rapid scale while providing them greater flexibility and choice in how they interact with us and the services they want to consume.

Lessons Learned 

The North Star in our strategy has been to bring the cloud to our clients and the broader financial ecosystem while preserving or improving the performance profiles that we offer today on our markets and for our technology clients. As part of this cloud journey, we are also focused on bringing our clients and partners with us so that they can fully leverage the power of the cloud to gain efficiencies and business agility. To do this successfully, we are adapting our tooling for deployment, orchestration and instrumentation.

Moving markets to the cloud required the migration of an intricate web of processes, systems and native protocols that are used by multiple layers of the overall capital markets ecosystem. To successfully execute at the performance level we do today and manage a wide array of real-time market conditions, our underlying market infrastructure for the matching engine must be resilient and highly performant. All the while, we must maintain ultra-low latency transactions and handle enormous volumes of message traffic. This was an ambitious, transformational project for the capital markets, and we’re grateful to have AWS as our preferred cloud provider and partner as we forge this new path. 

Interoperability between our trading platform and the new cloud infrastructure was mission-critical as we approached our program, and we identified some key architectural and operational changes that would enable the interoperability required while capturing the benefits of the cloud. With the new infrastructure-as-a-service model, we needed to ensure that market systems could continue to operate during AWS service events as well as when disconnected from the public cloud. Our infrastructure design is based on static stability, which serves as a guiding principle to deliver resiliency. Additionally, we know physical servers and other components can experience other issues that impact availability, so we had to consider fault domains as part of our deployment.

One of the key advantages of the cloud is the ability to capitalize on modern and efficient tools and API-driven workflows for orchestration and instrumentation that enable us to automate both how infrastructure and applications are deployed and managed. Critically, prior to launching MRX in the cloud, we repeatedly tested our operational support procedures and held drills with AWS for “game-day” scenarios to help ensure everything worked as intended.

Shifting the architecture and deployment of our mission-critical technology and operational models could not have been feasible without cultural alignment, a strong collaboration with AWS and the support of our customers and broader ecosystem stakeholders.

Over the last five years, and especially the last two, our organization has undertaken a number of initiatives, including the creation of a Cloud Center of Excellence (CCOE) and several training initiatives to expand the capabilities of our team and upskill where required. However, we’ve also had to think differently, and while doing so, align our businesses, technology teams and the broader organization. 

But our vision to move markets to the cloud would not have been possible without the deep collaboration and engagement at multiple levels with AWS. From the start, AWS helped us address and mitigate risks as we worked together to pave the way for how markets will operate in the future. Together, we have enhanced product development for AWS and Nasdaq, building the Outpost design and resiliency models, both new to AWS. Throughout this process, the tight feedback loop between our engineering teams has helped us stay on target and even outperform our expectations. 

Continuing the Journey

This latest milestone has been more than a decade in the making, but we’re only just getting started. From a technology perspective, we are introducing Kubernetes, which will help enable us to continuously improve the dynamism of the trading system deployment.

We plan to move more markets to the cloud next year and will further optimize our deployments as we learn more. As we advance, we intend to offer cloud-based Colocation with appropriate regulatory approval, and other complementary services for our client community in the coming years. Additionally, we are packaging our experiences with MRX to create a market migration blueprint designed to help our Market Technology community. Made up of over 130 financial market infrastructure providers – exchanges, clearinghouses and central securities depositories -- in 50 countries, we hope to help them drive and accelerate transformation in their market ecosystems all over the world based on our combined technology capabilities, expertise and experience. Finally, as we advance the creation of a private local zone within the footprint of our New Jersey data center, we are excited to enable our customers and partners to bring their workloads in close proximity to the matching engine. 

Nasdaq firmly believes that the cloud will serve as a hub for innovation in the capital markets in the decades ahead. We will continue to deepen our collaboration and expand capabilities to create new offerings in data and analytics, anti-financial crime technology and market infrastructure solutions. Moreover, we believe that the innovation that occurs as a result of the cloud will lead to incredibly powerful change that will shape the future of our industry. 

To learn more about how Nasdaq and AWS can help you execute your cloud journey, contact us.

Other Topics

Markets US Markets Innovation Technology Data and Analytics

Magnus Haglind

Nasdaq

Magnus Haglind is Senior Vice President and Head of Products, Marketplace Technology within the Market Platforms division of Nasdaq and is responsible for roadmap and product portfolio management of a broad suite of mission critical market infrastructure solutions provided to capital markets.

Read Magnus' Bio

Marc Murphy

Nasdaq

Marc Murphy is a Senior Vice President and leads Global Connectivity Services at Nasdaq. In his role, Marc is responsible for collaborating across Nasdaq to set strategic product direction related to Software Solutions and Technology Infrastructure, including the migration of Nasdaq’s North American Markets to the cloud and the application of additional emerging technologies.

Read Marc's Bio

Nate Sammons

Nate Sammons is Vice President of Enterprise Cloud Architecture at Nasdaq. Nate has been at the forefront of cloud adoption and cloud practice at Nasdaq, beginning with an early partnership with AWS in 2012. He is responsible for enterprise cloud architecture and works globally with teams to advance their cloud adoption and knowledge.

Read Nate's Bio