Vertex AI on Google Cloud Platform is already a powerful partner for developers, offering a range of advanced tools to build, train, and deploy machine learning models. Now, with the combination of Deep and Ray, the AI ecosystem is becoming more complete. Ray is a flexible and powerful decentralized application framework that can cleverly break down complex tasks into many small tasks and execute them simultaneously on multiple machines or cores, significantly improving computing efficiency. In the field of AI, Ray’s uniqueness makes it an ideal tool for accelerating the training of machine learning models. This combination allows developers to easily implement the entire process from build to deployment faster and with less effort, opening up a new era of AI development. The following will guide you through the AI/ML process, from getting started with AI/ML to suggesting your next steps in learning. Whether you are just getting started with AI/ML or are already looking for a breakthrough in the field, this article will provide you with valuable insights and guidance.
Developers and engineers face several major challenges when scaling AI/ML workloads:
- challenge 1：Difficulties of acquiring the required AI infrastructure. AI/ML workloads require a lot of computing resources, such as CPU and GPU. Developers need to have enough resources to run their workloads.
- challenge 2：Handle the multiple modes and programming interfaces required for effective AI/ML workload scaling. Developers may need to adapt their code to operate efficiently on the specific infrastructure available to them. This can be a time-consuming and complex task.
Ray provides a comprehensive and easy-to-use distributed framework for Python which helps solving these problems. With Ray, you can configure scalable clusters of computing resources and leverage a range of domain-specific libraries to efficiently assign common AI/ML tasks such as training, servicing, and tuning.
Google Cloud announced this year that seamless integration of the powerful decentralized Python framework Ray with Vertex AI is now fully available. This integrated approach enables AI developers to easily scale their AI workloads on Vertex AI’s versatile infrastructure, unlocking the potential of machine learning, data processing, and distributed computing.
Table of Contents
Table of Contents
Why Ray on Vertex AI?
- Scalable and accelerated AI development: Scalable and Accelerated AI Development: Ray’s distributed computing framework provides a unified experience for generative and predictive AI and integrates seamlessly with Vertex AI’s infrastructure services. Scaling Python-based machine learning, deep learning, reinforcement learning, data processing, and scientific computing workloads from a single machine to large clusters addresses the most demanding AI challenges without managing the complexity of the underlying infrastructure.
- Unified development experience: By integrating Ray’s API with the Vertex AI SDK for Python, AI developers can now, with minimal code changes, build interactive prototypes from local development environments or from Vertex AI Colab Enterprise. Seamless transition to production deployments on Vertex AI-hosted infrastructure.
- Enterprise-Level Security: Vertex AI’s powerful security features, including VPC service control, dedicated service connectivity, and customer-managed encryption keys (CMEK), help protect your sensitive data and models while leveraging Ray’s distributed computing capabilities. Vertex AI’s comprehensive security framework can help ensure that your Ray applications meet stringent enterprise security requirements.
Vertex AI and Ray 101
Suppose you want to tune a small language model (SML), such as Gemma or Llama. To fine-tune Gemma using Ray on Vertex AI, first you need a Ray cluster on Vertex AI. With the help from Ray on Vertex AI, using the console or the Vertex AI SDK for Python, this step takes just a few minutes . You can monitor clusters by leveraging integration with Google Cloud Logging or using Ray Dashboard.
(1) Create a cluster using Ray on Vertex AI
At present, Ray on Vertex AI supports version 2.9.3. Additionally, you have the option to create a custom image, which allows for greater flexibility in managing the dependencies within your Ray cluster.
Once the Ray cluster is running, it is very simple to use Ray on Vertex AI to develop AI/ML applications. The process may vary depending on your development environment. You can use the Vertex AI SDK for Python in Colab Enterprise or any IDE you like to establish a connection to the Ray cluster and run applications interactively. Alternatively, you can choose to create a Python script and programmatically submit it to a Ray cluster on Vertex AI using the Ray Jobs API, as shown below.
(2) Tuning Gemma on Ray on Vertex AI
Utilizing Ray on Vertex AI for developing AI/ML applications provides several advantages. In this context, you can take advantage of Vertex AI TensorBoard to validate your tuning jobs. Vertex AI TensorBoard offers a managed TensorBoard service that lets you monitor, visualize, and compare your tuning jobs while facilitating team collaboration. Additionally, Cloud Storage can be used to easily store model checkpoints, metrics, and other data. This setup allows for efficient use of the model in downstream AI/ML tasks, such as generating batch predictions with Ray Data, as demonstrated in the code below:
# Libraries
import datasets
import ray
input_data = datasets.load_dataset(dataset_id)
ray_input_data = ray.data.from_huggingface(input_data)
predictions_data = ray_input_data.map_batches(
Summarizer,
concurrency=config[“num_gpus”],
num_gpus=1,
batch_size=config[‘batch_size’])
# Store resulting predictions
predictions_data.write_json(‘your-bucker-uri/preds.json’, try_create_dir=True)
Case Study: chain grocery stores
For large operations, particularly in grocery stores, accurately forecasting demand is crucial for business profitability. While forecasting for a single item is challenging, managing forecasts for millions of items across hundreds of stores adds significant difficulty. Scaling the forecasting model is a difficult endeavor. H-E-B, a privately owned American supermarket chain with over 435 locations in Texas and Mexico, utilizes Ray on Vertex AI to ensure speed, reliability, and scalability in their forecasting processes.
“Ray has enabled transformative efficiencies in distributed machine learning for H-E-B, which have been crucial for large-scale demand forecasting” said Carey Chou, VP of Data AI at H-E-B. “We are excited about the co-innovation journey with Google Cloud and the enhanced accessibility to Vertex AI’s infrastructure through the Ray on Vertex integration. This integration plays a critical role in HEB’s distributed machine learning landscape.”
Microfusion Technology is committed to helping customers create a superior digital experience by conducting data and public opinion analysis projects for chain catering industries.Through in-depth analysis of consumer reviews, we can help customers to gain insight into customer needs, immediately send Google Map business information and consumer reviews back to the head office, greatly reduce the communication distance with consumers, identify service shortcomings, improve the problems, and further enhance the brand image.Detailed customer case sharing will be online soon, so stay tuned.
Check successful customer cases supported by Microfusion!
What’s Next?
Are you finding it challenging to scale your AI/ML applications? Begin by setting up a Ray cluster on Vertex AI via the Google Cloud console. New users receive $300 in free credits upon registration.
Ray on Vertex AI will enable you to build innovative and scalable applications. With Ray on Vertex AI, scaling Gen AI and predictive workloads on Vertex AI has never been easier, unlocking new possibilities for your organization!
If you want to learn more about Ray on Vertex AI, join the vibrant Ray community and Vertex AI Google Cloud community to share your experiences, ask questions, and collaborate on new projects.
Useful guidelines for Vertix AI
- Documentation
- Github samples:
This article is translated and adapted from the official Google Cloud blog.
Microfusion have being using AI technology to achieve business benefits for enterprise data. If you are interested, please contact Microfusion for consulting. We look forward to continuing to bring the most popular and front-line topics to you.
