PyTorch 2.0 Optimized for Graviton3 and a Graviton-focused virtual event you can’t miss
Originally posted on the AWS Graviton Weekly website: https://awsgravitonweekly.com/posts/aws-graviton-weekly-28-pytorch-2-0-optimized-for-graviton3-and-a-graviton-focused-virtual-event-you-can-t-miss
Issue # 28: March 10th, 2023 to March 17th, 2023
Welcome to Issue # 28 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Silicon: from March 10th, 2023 to March 17th, 2023.
Enjoy the content of this week.
The recommended readings for this week? 2 things
- PyTorch 2.0 was highly optimized to take advantage of Graviton3
- The Graviton-focused virtual event you can’t miss
Amazon AWS optimizes the PyTorch CPU inference on AWS Graviton3 based C7g instances. PyTorch 2.0 improves inference performance on Graviton compared to the previous releases, including improvements for Resnet50 and Bert.
The optimizations focused on three key areas: GEMM kernels, bfloat16 support, primitive caching and the memory allocator. For aarch64 platforms, PyTorch supports Arm Compute Library (ACL) GEMM kernels via Mkldnn(OneDNN) backend. The ACL library provides Neon/SVE GEMM kernels for fp32 and bfloat16 formats. The bfloat16 support on c7g allows efficient deployment of bfloat16 trained, AMP (Automatic Mixed Precision) trained, or even the standard fp32 trained models. The standard fp32 models leverage bfloat16 kernels via OneDNN fast math mode, without any model quantization. Next we implemented primitive caching for conv, matmul and inner product operators. More information on the updated PyTorch user guide with the upcoming 2.0 release improvements and TorchBench benchmark details can be found
Learn more here:
by Team PyTorch We are excited to announce the release of PyTorch® 2.0 which we highlighted during the PyTorch…
Today, we are announcing the general availability of Amazon Linux 2023(AL2023), our new Linux-based operating system for AWS that is designed to provide a secure, stable, high-performance environment to develop and run your cloud applications.
AL2023 provides seamless integration with various AWS services and development tools and offers optimized performance for Amazon Elastic Compute Cloud (EC2) Graviton-based instances and AWS Support at no additional licensing cost.
Starting with AL2023, a new Amazon Linux major release will be available every 2 years. This cadence provides you with a more predictable release cycle and up to 5 years of support, making it easier for you to plan your upgrades.
Learn more here:
ARM64 has become a very mainstream server architecture for cloud vendors. Various cloud vendors have begun rolling out servers based on Arm architecture like AWS Graviton and GCP Tau T2AA.
Apache APISIX has done a comprehensive CI regression test on ARM64 to ensure smoothness when users run Apache APISIX under the Arm architecture. Users care a lot about this.
Learn more here:
Articles and Tutorials
Tehama adopted Graviton technology from a strategic level to greatly improve business core competency. In about 6 months, Tehama has successfully migrated its whole technology stack to Graviton. The end result is that Tehama’s solution can run on both Intel and Graviton platforms, benefiting from the latest advances in silicon technologies.
Learn more here:
By Alexander Zaitsev (CTO at Altinity.Cloud)
Last month, AWS introduced new instance type families, powered by Graviton3 ARM processors: m7g and r7g. Since our go-to instance types for Altinity.Cloud are m5, m6i and m6g, I could not resist testing the performance of m7g. TLDR — it is outstanding!
Learn more here:
Ultra-Fast AWS Graviton Instances in Altinity.Cloud
AWS introduced new instance type families, powered by Graviton3 ARM processors: m7g and r7g. We tested m7g's…
By Stathis Peioglou (Senior Solution Architect at AWS)
AWS Lambda allows you to configure new and existing functions to run on Arm-based AWS Graviton2 processors in addition to x86-based functions.
On top, with this choice you can save money in two ways. First, your functions run more efficiently due to the Graviton2 architecture. Second, you pay less for the time that they run. In fact, Lambda functions powered by Graviton2 are designed to deliver up to 19 percent better performance at 20 percent lower cost.
Learn more here:
Slides, Videos, and Audio
With increasingly constrained budgets, understanding how to optimize spending without degrading performance is vital. SoftServe’s Cost Optimization Accelerator can identify cloud savings opportunities in seconds.
Watch the re-play of SoftServe’s Cloud Cost Optimization Accelerator Powered by AWS Graviton and learn more about SoftServe’s no-cost tool that runs on AMI, is easy to use, and identifies immediate savings wins by workload.
Learn about our AWS Funded Assessment that empowers your engineers with a roadmap to achieve cost savings and performance enhancements through modernization.
SoftServe's Cloud Cost Optimization Accelerator Powered by AWS Graviton
With increasingly constrained budgets, understanding how to optimize spending without degrading performance is vital…
AWS ECS is a powerful service that allows you to run containerized applications at scale. It’s suitable for a variety of use cases, including web applications, microservices, and background processing. In this episode, we’ll provide an introduction to the main concepts of ECS and then dive into cost-optimization strategies.
We’ll explore the different options for running containers on ECS, including EC2, Fargate, and ECS Anywhere. We’ll discuss various opportunities for saving money, such as using Arm (Graviton) instances, Spot instances, Compute Savings Plans, and RIs or EC2 Saving Plans. Finally, we’ll cover how to set up ECS to use Spot instances, including how to create capacity providers and specify a capacity provider strategy. We’ll also discuss whether it’s always best to use EC2 instead of Fargate for cost optimization and recommend some tools that can help you find other opportunities to save on container costs.
In this AWS CFM Talks webinar, our experts answer four common customer questions related to managing and optimizing cloud costs, so you can kickstart your optimization efforts with confidence.
By Komal Pal
At re:Invent 2022, Amazon RDS customers gained access to a number of exciting new features and capabilities. Two of these were Amazon RDS Optimized Writes and Amazon RDS Optimized Reads. Optimized Writes, built on top of the new AWS Nitro System Torn Write Prevention feature, allow you to improve your database’s write transaction throughput by up to 2x in RDS for MySQL at no additional cost. For customers looking for faster query processing capabilities can use Optimized Reads to achieve up to 2x faster query processing in Amazon RDS for MySQL and Amazon RDS for MariaDB at no additional cost.
- Senior Software Engineer, Data Platform at Vantage
- System Dev/Fleet Solutions Eng, Graviton Fleet Operational team at Amazon. Chat with Avishai Mostowicz
- Staff HPC Engineer at Arm. Chat with Troy Gasawa
If you are looking for amazing candidates ready for interviews, we encourage your company to join the Interesting Data Gigs Talent Collective here.
Thursday, March 23th 9:00 AM — 9:50 AM GMT -05 A Demonstration of AI and HPC Applications for NVIDIA Grace CPU [S51880]
It’s often said that “Porting to Arm is boring,” but how easy is it, really? We’ll demonstrate top machine learning frameworks, HPC applications, and tools for data science on the NVIDIA Arm HPC DevKit. The DevKit is an on-ramp platform for NVIDIA Grace CPU that incorporates dual NVIDIA A100 GPUs, an NVIDIA BlueField DPU, and an 80-core Ampere Altra Arm CPU, in a standards-compliant Arm server. We’ll walk through the complete installation process for key applications and their dependencies including codes like TensorFlow, OpenRADIOSS, WRF, GROMACS, BWA-MEM2, and Jupyter Notebook. We’ll also show how codes incorporating x86 AVX and SSE SIMD instructions can be trivially ported to Arm with freely available tools. We’ll conclude with a general guide to porting to NVIDIA Grace and links to downloadable resources and tutorials that fully replicate our demonstrations. This session is a strong starting point for anyone targeting NVIDIA Grace Hopper or the NVIDIA Grace CPU Superchips.
[Virtual Event] Accelerate SaaS revenue and improve price-performance with AWS Graviton Thursday, March30 2023 | 9:00AM — 11:00AM PDT
AWS Container Services, including Amazon Elastic Kubernetes Service (EKS), Amazon Elastic Container Service (ECS), and AWS Fargate, streamline the deployment and management of software applications on AWS. This leads to a significant reduction in operational workload during the development, testing, and large-scale deployment of enterprise software products.
Additionally, these services fully support the use of AWS Graviton-based EC2 instances, which enables customers to benefit from the cost savings associated with Graviton instances. Graviton offers the best price performance of all EC2 instances, so you can accelerate growth while you improve operating margins.
Join us to learn how customers have realized price-performance improvements by switching to AWS Graviton for their container workloads. We will share best practices and considerations when moving workloads to AWS Graviton
- Ravi Yadav, Senior Containers Specialist
- Matheen Raza, Senior Containers Specialist
- Hakha Momeni, Senior Graviton Specialist
RAI Amsterdam, Europaplein 24 1078 GZ Amsterdam Nederland
April 18th — April 21st, 2023, 9am — 6pm CEST
Future proof your Kubernetes cluster for cost optimization
Cost optimization is a common priority. In this session, learn about the many factors that can increase costs in your Kubernetes usage beyond compute, such as compute efficiencies at the node and pod level, scaling parameters, networking cost, multi-architecture image creation, security posture, and more. Discover how to change your application components and migrate from x86-based instances to AWS Graviton to achieve cost efficiency at higher performance. Also, learn how to use open-source tools and AWS services to optimize these costs and make your Kubernetes cluster more resilient to economic instability.