Azure Confidential VMs with NVIDIA H100 GPUs for secure retrieval-augmented generation

Watch Azure CTO and Technical Fellow Mark Russinovich discuss the benefits of confidential AI and give a demonstration of Azure confidential VMs with NVIDIA H100 Tensor Core GPUs for protecting retrieval-augmented generation (RAG). These VMs let you run your most sensitive GPU enhanced machine learning (ML) workloads on Azure in a hardware based and attested trusted execution environment which helps protect data in memory from unauthorized access, even by Azure operators. These VMs can help protect three types of sensitive data while in use: the ML model itself, pre-training data, and user prompts and responses.

To learn more, read the blog: https://aka.ms/CVM-H100-preview.

#microsoft #microsoftazure