I have been playing with AI for some time, In fact, the featured image of this very blog post was generated by AI Art. With the rise of cloud-native ecosystems like Kubernetes and serverless, the world of software engineering has been completely revolutionized. As AI continues to influence every aspect of our lives, as a cloud-native enthusiast, I’m curious about what it will bring to the cloud-native world as well.
In this blog post, we’ll take a closer look at how AI is impacting the cloud-native system and what it means for the future of modern application development.
Cloud-Native and AI Synergy
If you’ve been using ChatGPT and the new Bing from Microsoft, you may know they’re both powered by generative AI! Generative AI is a type of artificial intelligence that uses neural networks and deep learning algorithms to create unique content such as text, images, videos, music, or even coding on your behalf. It doesn’t just recognize patterns in existing data but goes one step further to create something entirely new when presented with natural language prompts.
Generative AI is one of the most exciting and rapidly evolving areas of the tech industry today, and OpenAI is at the forefront of this innovation. With a total of $11.3B in funding across 7 rounds, according to data from Crunchbase, OpenAI has made significant strides in advancing the field of AI, particularly through their flagship project ChatGPT.
In 2018, at the 2-year mark after OpenAI started to use Kubernetes for deep learning model training, at a time OpenAI pushed the cluster to scale to over 2,500 nodes on Azure on D15v2 and NC24 VMs. 3-years later, this quickly got quickly pushed over again to scaling to over 7500 nodes reported in Jan 2021.
Scaling a single Kubernetes cluster to this size is actually not recommended based on CNCF’s official Kubernetes documentation where even the latest Kubernetes 1.27 supports only up to 5000 nodes (this limit hasn’t been changed and has been part of the release process for quite some time). Sidenote that the official recommendation was only 1000 nodes back in 2016 and bumped up to 5,000 from Kubernetes 1.6 in 2017 (Check out this AWS office hours Youtube video to know the whole story). Notably, AI has always been revolutionizing Kubernetes, as demonstrated by Alibaba’s need for a 10,000-node Kubernetes cluster for a major shopping festival in 2019.
The Rise of New AI Chips
Running AI is incredibly demanding, especially for open AI, it runs large machine learning jobs spanning so many Kubernetes nodes which need full GPU power. To achieve this, it relies on GPUDirect for direct communication with the NIC or NVLink for cross-communication with the GPU.
On March 21st, 2023, NVIDIA announced the availability of the NVIDIA H100 Tensor Core GPU which is introduced as the world’s most powerful option ( and pricey ) for generative AI training and machine learning inference, which is crucial for advancements like GPT4 (you may get this if you have ChatGPT plus membership and OpenAI limits GPT-4 to 25 messages every 3 hours). This official white paper gives an overview of NVIDIA H100 Tensor Core GPU architecture.
This aligns with Microsoft Azure introduced the ND H100 v5 VM on March 13th which is the most powerful and massively scalable AI virtual machine series in Azure by far. Amazon Web Services announced EC2 UltraClusters of Amazon EC2 P5 instances is coming soon. And Oracle Cloud Infrastructure (OCI) announced the new OCI Compute bare-metal GPU instances featuring H100 GPUs in limited availability. I am intrigued to see what these announcements will bring to the table in terms of advancements in the cloud-native and AI synergy.
Challenges and Opportunities in Cloud-Native with AI
Although computational power is not the only factor, there also comes the challenges with networking, service reliability (particularly from high-demanding incoming requests for API servers), observability (using tools like Prometheus and Grafana ), and of course, security also play critical roles in all scenarios.
Over the past years, Cloud-native ecosystems such as Kubernetes, and serverless have revolutionized software design, development, and deployment. As the importance of AI capabilities continues to grow, cloud-native apps are increasingly being infused with AI to enable new use cases and improve business efficiencies.
At the core of your next software innovation lies a modular and extensible Kubernetes-based service architecture that can accelerate the construction of AI platforms while improving resource utilization and delivery efficiency. This approach allows for greater flexibility to integrate AI components while capitalizing on the benefits of cloud-native and AI technologies.
Conclusion
As we have seen, cloud-native and AI technologies can be leveraged to create powerful software solutions that are both efficient and reliable. By combining the strengths of these two cutting-edge technologies, businesses can capitalize on their benefits while driving innovation and business success. With more advancements in this space being made every day, it’s an exciting time for those interested in leveraging the power of cloud-native technology with AI capabilities.
If you enjoy reading blog posts like this one, don’t forget to subscribe to my private distribution list from https://bit.ly/cloudnativeinnovators so you don’t miss them out!
Related Resources
Kubernetes Up & Running
Kubernetes radically changes the way applications are built and deployed in the cloud. The updated edition of the ‘Kubernetes up and running’ ebook shows developers and ops personnel how to use Kubernetes and container technology to achieve new levels of velocity, agility, reliability, and efficiency. It’s available to download for free from here or purchase it from Amazon here if you wish to get a physical copy.
Certified Kubernetes Administrator (CKA) Exam Guide
This book helps individuals learn Kubernetes and obtain certification, opening doors to new career paths as Kubernetes administrators and adding value to their organizations. You can purchase it in either paperback or Kindle format using the code or direct link: https://amzn.to/3nfiGAu with a 20% discount code by May 13th 2023.
Generative Deep Learning
Generative AI is the hottest topic in tech. This practical book teaches machine learning engineers and data scientists how to create impressive generative deep learning models from scratch. purchase it from Amazon here if you wish to get a physical copy from here.
