Co-authors: Tim Thornton – Director, Arm Engineering, Kushal Koolwal - Sr. Manager, Software Ecosystem
As a global leader in semiconductor design and silicon intellectual property development and licensing, there is no overstating the importance of Arm’s relationships with our Electronic Design Automation (EDA) partners and the tools they develop that allow our now 30-year-old business to thrive. This close integration is not only important to the broad ecosystem of companies that licenses Arm’s IP, but it is also critical to our core business of IP development. With our global footprint of engineers, we have multiple concurrent design projects running around the clock that require the usage of various EDA tools.
Running these EDA workloads on Arm servers for designing Arm processors is a strategic initiative that we embarked on four years ago by deploying EDA tools on-prem on Arm-based servers. We call this Arm-on-Arm. The expansion of this transition from x86 based EDA tools and workflows to on-prem Arm-based servers to the Arm-based processors in the cloud is now possible due to the availability of these simulation, verification and implementation tooling from our EDA partners.
In this blog, we will highlight the journey of designing Cortex-M55 CPU in production from on-prem to Arm Neoverse based AWS Graviton2 processors. This has increased our overall throughput significantly by reducing cost and scheduling risks for critical projects, with some teams experiencing a 6x improvement. In addition, AWS Graviton2 based Amazon M6g instances provide 36% cost savings and 20% better runtimes compared to the x86 based Amazon M5 instances.
Arm’s journey for deploying EDA workloads on Arm-based processors in the cloud began with close collaborations with our EDA partners with our on-prem deployments. This began by having a shared vision with our EDA partners on benefits of running these workloads on Arm and in having a close collaboration in porting of the tools. This allowed us to be ready to gain higher throughput per server and lower overall costs with the advent of Marvel ThunderX2 systems in late 2018.
For the design of Cortex-M55, Arm leverages Mentor QuestaSim for simulation and Cadence JasperGold as the verification tool. Both these tools were made available by our partners to run on Arm servers. QuestaSim is part of Mentor Graphics’ Questa Verification Solution and is the latest tool for Functional Verification providing simulation support for latest standards of SystemC, Verilog, SystemVerilog, and VHDL.
Having a well-designed on-prem EDA deployment set the stage for Arm to begin leveraging the benefits of running Arm-on-Arm on AWS Graviton2 processors that became available in 2020. AWS custom-built the Graviton processors using 64-bit Arm Neoverse cores to deliver the best price-performance for cloud workloads running in the Amazon EC2 environment. The Graviton processors enabled us to begin testing in the cloud and got us thinking about how we should re-architect our EDA workflows for running both on-prem and in the cloud. In fall 2019, AWS launched the Graviton2 processor based on Arm’s Neoverse N1 core, and the EC2 C6g, M6g, and R6g instances it powers provide up to 40% better price and performance than comparable 5th-generation x86-based instances. These Arm-based instances naturally became the highest performance mainstream option for EDA in the cloud.
Moving to the cloud allowed us to better scale our cluster capacity in the cloud specifically since EDA workloads can be bursty. This makes provisioning enough internal EDA cluster capacity (fixed server farms) a constant challenge. Building out more internal datacenter capacity is not always the most optimal solution as there will be periods where the server resource would be under-utilized. So we started to think about how we could tune our EDA processes and workflows to take advantage of the flexibility that the cloud offers in terms of capacity. Not only getting the right CPU capacity at the right time is one of the key benefits of cloud, but you can also right size the server configuration in terms memory, networking, storage, and so on. Having the ability to get the optimal hardware configuration when you need it avoids project delays and provides maximum flexibility in starting new semiconductor design projects at lower costs.
Moving EDA workloads to the cloud is not necessarily a lift and shift of our existing tools and workflows. Our long-term objectives for running EDA jobs in the cloud are to be able to leverage multi-cloud capabilities to be able to select the best performing cloud services depending on our workload. In addition, to be able to implement cost monitoring and cost controls and also implement access and security controls, given the highly sensitive and valuable nature of our IP. Finally, everything had to work as reliably and consistently as with our on-prem cluster. Therefore, trying to lift-and-shift our existing toolchain just did not meet those requirements.
To achieve the previous goals, we developed an internal orchestration scheduler RESTful API tool to which users can submit jobs and have those jobs run in the cloud. Our tool is written to be extensible and cloud-agnostic. We use an extensive list of AWS services, including Elastic Kubernetes Service (EKS) container management, Lambda “serverless” functions, Amazon Kinesis, API Gateway manager, DynamoDB, PostgreSQL, AWS Batch, and of course EC2 compute instances and S3 storage. Other parts of our stack include RabbitMQ, Celery, Elasticsearch, FluentBit, and Kibana. We were not the experts in these technologies, and we climbed a steep learning curve to understand how these technologies work and interact with one another. We discovered and overcame many challenges in figuring out how to make this tool scalable, reliable, and efficient for cloud environments.
Figure 1 represents the high-level workflow in running EDA jobs leveraging our internal Orchestration Scheduler tool. We use the AWS Batch service to scale large numbers of Amazon M6g instances to run Mentor Questa in container images with high throughput.
Figure 1: High-level workflow for designing Cortex-M55 on AWS Graviton2
Figures 2 and 3 respectively show 20% better job run times and 36% better cost savings while using Mentor QuestaSim on the M6g instances compared to M5. Our initial testing of Mentor QuestaSim showed we could reduce our job run time by 20% and achieve a 36% cost saving with EC2 M6g instances compared to M5 Xeon-based instances.
Figure 2: Comparison of Mentor QuestaSim run time between M6G and M5 instances (less is better).
Figure 3: Comparison of Mentor QuestaSim run time cost between M6g and M5 instances (cost is based on AWS on-demand pricing).
The following figure shows the result of us switching our Mentor QuestaSim production validation runs on Cortex-M55 from M5 to M6g. The x-axis shows time, and what you can see is that jobs submitted to M6g-based instances complete in a shorter amount of time than those submitted to M5-based instances, matching our prior expectations. The y-axis shows the number of concurrent EDA jobs running, and in moving to M6g we have found we are able to launch more concurrent jobs than we could on M5.
Figure 4: Production validation of Arm Cortex-M55 – comparison between M6g and M5. X-axis: Time for job completion. Y-axis: Number of concurrent jobs
From an EDA engineer’s point of view, this is the one thing they truly care about – getting the capacity needed to run their jobs in the quickest time possible. With AWS Graviton2, we are getting more EDA work done, quicker, and for a lower cost. And we have gained greater operational flexibility while retaining a high level of reliability and access control.
Arm’s move to EDA in the cloud on Arm has just begun. The promising results from the Cortex-M55 CPU project allowed us to move to the cloud in a production setting. And we have set a goal to move majority of our on-premise EDA compute to the cloud in the next few years. We are actively engaged with all the leading EDA partners to make the broader set of EDA tools available on Arm. We are seeing significant benefits from our move to EDA on Arm in the cloud and we invite the rich ecosystem of Arm silicon partners to join us.
We are well underway in our journey from getting the tools and infrastructure for EDA on Arm enabled and look forward to engaging with the community in sharing our best practices and lessons learned. For any direct queries, please reach out to us here.
[CTAToken URL = "https://www.arm.com/company/news/2020/12/arm-moves-production-level-electronic-design-automation-to-the-cloud-with-the-help-of-aws" target="_blank" text="Learn More about Arm EDA in the Cloud" class ="green"]