Announcing OpenAI-compatible API support for Amazon SageMaker AI endpoints | Amazon Web Services
Today, Amazon SageMaker AI introduces OpenAI-compatible API support for real-time inference endpoints. If you use the OpenAI SDK, LangChain, or Strands Agents, you can now invoke models on SageMaker AI by changing only your endpoint URL. You don’t need a custom client, a SigV4 wrapper, or code rewrites.
Overview
With this launch, SageMaker AI endpoints expose an /openai/v1 path that accepts Chat Completions requests and returns responses as is from the container, including streaming. OpenAI endpoints are turned on for all endpoints and inference components using standard SageMaker AI APIs and SDK.
SageMaker AI routes based on the endpoint name in the URL, so any OpenAI-compatible client works out of the box. You can now create time-limited bearer tokens for your endpoints and use them with your OpenAI clients.
For a working example that includes deployment and invocation, see the accompanying notebook on GitHub.
“We run AI coding agents...
Copyright of this story solely belongs to amazon.com. To see the full text click HERE
