Amazon SageMaker AI Async Inference now supports inline request payloads | Amazon Web Services
Today, we’re announcing inline payload support for Amazon SageMaker AI Async Inference. Customers can now send inference payloads directly in the request body of the InvokeEndpointAsync API, removing the need to upload input data to Amazon Simple Storage Service (Amazon S3) before each invocation.
For payloads up to 128,000 bytes, this removes an entire network round-trip, simplifies client-side code, and reduces the operational surface area of asynchronous inference workloads.
In this post, we explain the motivation behind this feature, walk through the customer experience before and after, and show you how to start using inline payloads today.
Background: How async inference worked before
You can use Amazon SageMaker AI Async Inference to queue inference requests and process them asynchronously. It’s a good fit for workloads with large payloads, variable traffic, or tolerance for seconds-to-minutes latency. It supports automatic scaling to zero, making it cost-efficient for bursty or batch-style workloads.
Until...
Copyright of this story solely belongs to amazon.com. To see the full text click HERE
