Skip to content
AI ConnectPowered by VELENTIS
AI-generated1 min

Amazon SageMaker Inference Adds NVIDIA Multi-Instance GPU Partitioning Support

Amazon SageMaker Inference now supports NVIDIA Multi-Instance GPU partitioning, letting teams divide a single GPU into hardware-isolated segments.

This article was AI-generated and published automatically. Context, labelling and all sources at the end of the article.

(KI-generiertes Symbolbild: Gemini / AI Connect)

Amazon SageMaker Inference has introduced support for GPU partitioning utilizing NVIDIA Multi-Instance GPU technology. The capability functions by dividing an individual GPU into hardware-isolated partitions. As a result of this isolation, users can run a distinct model in each separate partition or take advantage of a right-sized fraction of a newer accelerator.

The deployment workflow utilizes two optional API fields configured to place one model per partition. In addition to outlining the endpoint configuration, the release details comparative evaluations against legacy hardware. Specifically, the metrics demonstrate how performance on partitioned hardware compares against a whole older GPU.

What this means for you

For cloud infrastructure teams, partitioning hardware allows organizations to allocate right-sized fractions of modern accelerators to individual workloads instead of provisioning an entire chip. The support for two optional API fields simplifies running distinct models across isolated segments of the same physical GPU. This enables teams to benchmark resource utilization and workload placement directly against complete older GPUs.

Evidence

Solidly sourced
46/100

  • Amazon SageMaker Inference provides support for GPU partitioning through NVIDIA Multi-Instance GPU technology.

    single source
    Quote

    „Amazon SageMaker Inference now supports GPU partitioning with NVIDIA Multi-Instance GPU (MIG)“

    AWS, Machine Learning

  • The partitioning approach splits a GPU into hardware-isolated sections to support separate models.

    single source
    Quote

    „divides one GPU into hardware-isolated partitions so you can run a different model in each“

    AWS, Machine Learning

  • Users can allocate a right-sized portion of a modern accelerator.

    single source
    Quote

    „take a right-sized fraction of a newer accelerator“

    AWS, Machine Learning

  • Workload placement across partitions is handled via two optional API fields.

    single source
    Quote

    „two optional API fields place one model per partition“

    AWS, Machine Learning

  • Performance was evaluated by comparing the configuration against an entire older GPU.

    single source
    Quote

    „what we measured against a whole older GPU“

    AWS, Machine Learning

Sources

The evidence score is computed, not hand-set: from confidence, the number of sources and the share of verified statements.

Source & transparency

As of: November 06, 2026

AI-generatedAI-generated: produced automatically from vetted sources with technical quality checks (source, quote and figure verification); no human sign-off of each item before publication

Sources
1
Verified statements
0 / 5
Evidence score
46Solidly sourced
Use Cases

Want to put this into practice?

We connect you with suitable AI providers from the DACH region, free of charge and without obligation.

What's next?

What does AI really cost?

Honest price ranges for your project, independent, with nothing to sell you.

See prices

Get your team ready (AI Act)

The EU AI Act requires providers and deployers to take measures supporting AI literacy in the team — not a specific level. Baseline training for your team is in development.

Find the right training

Find the right provider

Describe your project. We connect you with suitable providers at a fair price.

Start the analysis
Back to AI Radar