Arm Community
Arm Community
  • Site
  • User
  • Site
  • Search
  • User
  • Groups
    • Arm Research
    • DesignStart
    • Education Hub
    • Graphics and Gaming
    • High Performance Computing
    • Innovation
    • Multimedia
    • Open Source Software and Platforms
    • Physical
    • Processors
    • Security
    • System
    • Software Tools
    • TrustZone for Armv8-M
    • 中文社区
  • Blog
    • Announcements
    • Artificial Intelligence
    • Automotive
    • Healthcare
    • HPC
    • Infrastructure
    • Innovation
    • Internet of Things
    • Machine Learning
    • Mobile
    • Smart Homes
    • Wearables
  • Forums
    • All developer forums
    • IP Product forums
    • Tool & Software forums
  • Support
    • Open a support case
    • Documentation
    • Downloads
    • Training
    • Arm Approved program
    • Arm Design Reviews
  • Community Help
  • More
  • Cancel
System
  • Developer Community
  • IP Products
  • System
  • Jump...
  • Cancel
System
SoC Design blog PC meets Arm: Integrating PCIExpress into the Arm Server Architecture
  • Blogs
  • Forums
  • Videos & Files
  • Mentions
  • Sub-Groups
  • Tags
  • Jump...
  • Cancel
  • New
More blogs in System
  • Embedded blog

  • SoC Design blog

Tags
  • xilinx
  • Cadence Design Systems
  • GICv3/v4
  • SMMUv3
  • Arm TechCon
  • tsmc
Actions
  • RSS
  • More
  • Cancel
Related blog posts
Related forum threads

PC meets Arm: Integrating PCIExpress into the Arm Server Architecture

Nick
Nick
September 25, 2017

Earlier this month you may have noticed some press coverage regarding a collaboration between Xilinx, Arm, Cadence and TSMC to deliver 7nm test chip. 

There are some significant challenges assembling server SoCs for the infrastructure market with the latest PCIExpress gen4 capabilities, this blog provides a brief overview of the challenges which will be covered in more detail at the Arm Techcon event in California in October. 

A typical Arm Server SoC would look something like the block diagram below.

Typical Arm Server SoC infastructure

Notice the two key external connections via PCIExpress gen4 and CCIX (pronounced “cee-six”). CCIX is a new protocol announced in 2016 by the CCIX Consortium to address the challenge of connecting multiple integrated circuits together coherently. Notice how CCIX uses the PCIe PHY, this is because the CCIX protocol provides a new protocol layer which operates over the top of the PCIe protocol. Adopting this architecture has enabled the CCIX protocol to be defined and implemented in an incredibly short period of time. Take a look at this blog from Jeff Defillipi which explains the rationale for CCIX 'How do AMBA, CCIX and GenZ address the needs of the data center?'.

The key challenges with verifying the integration of gen4 PCIExpress IP into an Arm SoC come about because gen4 has new system-level aspects which impinge on multiple other complex system IP such as System Memory Management Unit (SMMU) and Generic Interrupt Controller (GIC). 

One such example of complex system interaction is the gen4 capability for supporting Address Translation Services (ATS). ATS provides the opportunity to build intelligent End Point devices which have their own Memory Management Unit which is sync’d with the SMMU on the infrastructure host. Translation Lookaside Buffers (TLBs) which provide cached lookup for address translation tables are loaded across the PCIe interface to enhance performance. Traffic can then be sent using “ready translated” addresses which obviate the need for further translation by the SMMU. The diagram below shows the kind of testbench needed to verify ATS.

Example of a typical testbench to verify ATS

The packets initiated from the EP are tagged in such a way that the SMMU just passes them straight through. SW on the host side manages page tables and hence Distributed Virtual Memory (DVM) messages are also passed from the host back to the ATS EP to maintain the TLBs. As you can probably imagine, in order to verify this capability is functioning correctly there needs to be considerable system setup done, a Verification IP is needed which supports ATS, and then SW needs to drive traffic to verify that ATS is working in all it’s various modes. 

One of the most significant challenges for ATS as well as dozens of other integration aspects is the need for complex use-cases. At Arm Techcon we will be introducing a new Portable Stimulus Library (PSS) for PCIExpress which provides significant out-of-the-box capabilities for creating complex PCIExpress use-cases. 

I will be joined by Sujil Kottekkat from Arm to present how we addressed the ATS and other challenges on Tuesday 24th October at Arm Techcon.

Anonymous
SoC Design blog
  • A painless upgrade to 22nm – the right cost with the available IP

    Claudia Pike
    Claudia Pike
    In this blog, read about upgrading to 22nm in your ASIC design feature.
    • August 27, 2020
  • Simplifying workload modeling with AMBA ATP Engine

    Francisco Socal
    Francisco Socal
    Following the release of the AMBA Adaptive Traffic Profiles (ATP) Specification, we are pleased to announce the AMBA ATP Engine, to further facilitate ATP’s adoption into a variety of platforms.
    • May 20, 2020
  • Docker enables Arm Cycle Model Studio on Ubuntu

    Jason Andrews
    Jason Andrews
    Arm Cycle Model Studio (CMS) is a great tool to create SystemC simulation models from Verilog RTL source code. This articles shows how to use Docker to run CMS and create models on an Ubuntu machine.
    • October 23, 2019