Using NVIDIA Run:ai GPU Fractioning on Amazon EKS | Amazon Web Services
GPU sharing is not new: Multi-Instance GPU (MIG), time-slicing, and Multi-Process Service (MPS) address different isolation and utilization needs. NVIDIA Run:ai GPU Fractioning builds on MPS with Kubernetes-integrated scheduling, per-Pod memory contracts, enforcement, and metrics.
Validated result: Two Pods shared one NVIDIA A10G Tensor Core GPU. Each Pod successfully allocated 6 GiB within an 8 GiB limit, while an allocation above the limit failed as expected.
NVIDIA Run:ai is AI workload orchestration software for Kubernetes. Its GPU Fractioning Operator can help several teams share one or two GPUs on Amazon Elastic Kubernetes Service (Amazon EKS). Notebooks, model evaluations, and small inference services can leave teams waiting even when GPU memory remains available. In this walkthroughâs baseline, the NVIDIA device plugin exposes the single GPU as one nvidia.com/gpu resource, with GPU sharing disabled. Two Pods each requesting nvidia.com/gpu: 1 cannot run concurrently on that node, even when their combined GPU-memory...
Copyright of this story solely belongs to aws.amazon.com. To see the full text click HERE