AIServiceBackend
aigateway.envoyproxy.io / v1alpha1
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: AIServiceBackend
metadata:
name: example
apiVersion
string
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind
string
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata
object
spec object
Spec defines the details of AIServiceBackend.
backendRef object required
BackendRef is the reference to the Backend resource that this AIServiceBackend corresponds to.
A backend must be a Backend resource of Envoy Gateway. Note that k8s Service will be supported
as a backend in the future. See https://github.com/envoyproxy/ai-gateway/issues/902 for more details.
This is required to be set.
group
string
Group is the group of the referent. For example, "gateway.networking.k8s.io".
When unspecified or empty string, core API group is inferred.
pattern:
^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$maxLength:
253
kind
string
Kind is the Kubernetes resource kind of the referent. For example
"Service".
Defaults to "Service" when not specified.
ExternalName services can refer to CNAME DNS records that may live
outside of the cluster and as such are difficult to reason about in
terms of conformance. They also may not be safe to forward to (see
CVE-2021-25740 for more information). Implementations SHOULD NOT
support ExternalName Services.
Support: Core (Services with a type other than ExternalName)
Support: Implementation-specific (Services with type ExternalName)
pattern:
^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$minLength:
1maxLength:
63
name
string required
Name is the name of the referent.
minLength:
1maxLength:
253
namespace
string
Namespace is the namespace of the backend. When unspecified, the local
namespace is inferred.
Note that when a namespace different than the local namespace is specified,
a ReferenceGrant object is required in the referent namespace to allow that
namespace's owner to accept the reference. See the ReferenceGrant
documentation for details.
Support: Core
pattern:
^[a-z0-9]([-a-z0-9]*[a-z0-9])?$minLength:
1maxLength:
63
port
integer
Port specifies the destination port number to use for this resource.
Port is required when the referent is a Kubernetes Service. In this
case, the port number is the service port number, not the target port.
For other resources, destination port might be derived from the referent
resource or this field.
format:
int32minimum:
1maximum:
65535bodyMutation object
BodyMutation defines the mutation of HTTP request body JSON fields that will be applied to the request
before sending it to the backend.
remove
[]string
Remove the given JSON field(s) from the HTTP request body before sending to the backend.
The value of Remove is a list of top-level field names to remove.
Input:
{
"model": "gpt-4",
"service_tier": "default",
"internal_flag": true
}
Config:
remove: ["service_tier", "internal_flag"]
Output:
{
"model": "gpt-4"
}
maxItems:
16set []object
Set overwrites/adds the request body with the given JSON field (name, value)
before sending to the backend. Only top-level fields are currently supported.
Input:
{
"model": "gpt-4",
"service_tier": "default"
}
Config:
set:
- path: "service_tier"
value: "scale"
Output:
{
"model": "gpt-4",
"service_tier": "scale"
}
maxItems:
16
path
string required
Path is the top-level field name to set in the request body.
Examples: "service_tier", "max_tokens", "temperature"
minLength:
1
value
string required
Value is the JSON value to set at the specified field. This can be any valid JSON value:
string, number, boolean, object, array, or null.
The value will be parsed as JSON and inserted at the specified field.
Examples:
- "\"scale\"" (string)
- "42" (number)
- "true" (boolean)
- "{\"key\": \"value\"}" (object)
- "[1, 2, 3]" (array)
- "null" (null)
headerMutation object
HeaderMutation defines the mutation of HTTP headers that will be applied to the request
before sending it to the backend.
remove
[]string
Remove the given header(s) from the HTTP request before the action. The
value of Remove is a list of HTTP header names. Note that the header
names are case-insensitive (see
https://datatracker.ietf.org/doc/html/rfc2616#section-4.2).
Input:
GET /foo HTTP/1.1
my-header1: foo
my-header2: bar
my-header3: baz
Config:
remove: ["my-header1", "my-header3"]
Output:
GET /foo HTTP/1.1
my-header2: bar
maxItems:
16set []object
Set overwrites/adds the request with the given header (name, value)
before the action.
Input:
GET /foo HTTP/1.1
my-header: foo
Config:
set:
- name: "my-header"
value: "bar"
Output:
GET /foo HTTP/1.1
my-header: bar
maxItems:
16
name
string required
Name is the name of the HTTP Header to be matched. Name matching MUST be
case-insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2).
If multiple entries specify equivalent header names, the first entry with
an equivalent name MUST be considered for a match. Subsequent entries
with an equivalent header name MUST be ignored. Due to the
case-insensitivity of header names, "foo" and "Foo" are considered
equivalent.
pattern:
^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$minLength:
1maxLength:
256
value
string required
Value is the value of HTTP Header to be matched.
<gateway:experimental:description>
Must consist of printable US-ASCII characters, optionally separated
by single tabs or spaces. See: https://tools.ietf.org/html/rfc7230#section-3.2
</gateway:experimental:description>
<gateway:experimental:validation:Pattern=`^[!-~]+([\t ]?[!-~]+)*$`>
minLength:
1maxLength:
4096schema object required
APISchema specifies the API schema of the output format of requests from
Envoy that this AIServiceBackend can accept as incoming requests.
Based on this schema, the ai-gateway will perform the necessary transformation for
the pair of AIGatewayRouteSpec.APISchema and AIServiceBackendSpec.APISchema.
This is required to be set.
name
string required
Name is the name of the API schema of the AIGatewayRoute or AIServiceBackend.
enum:
OpenAI, Cohere, AWSBedrock, AzureOpenAI, GCPVertexAI, GCPAnthropic, Anthropic, AWSAnthropic
prefix
string
Prefix is the prefix for the API.
When the name is set to "OpenAI", "chat completions" API endpoint will be "${this_field}/chat/completions".
When the name is set to "Anthropic", the "messages" API endpoint will be "${this_field}/messages".
It can be with or without a leading slash ("/").
This field is ignored for AWSAnthropic and GCPAnthropic.
This is especially useful when routing to a backend that has an OpenAI or Anthropic compatible API but has a different
prefix. For example, Gemini OpenAI compatible API (https://ai.google.dev/gemini-api/docs/openai) uses
"/v1beta/openai" prefix. Another example is that Cohere AI (https://docs.cohere.com/v2/docs/compatibility-api)
uses "/compatibility/v1" prefix. On the other hand, DeepSeek (https://api-docs.deepseek.com/) doesn't
use prefix, so you can leave this field unset.
See https://aigateway.envoyproxy.io/docs/capabilities/llm-integrations/supported-providers for details.
version
string
Version is the version of the API schema.
When the name is set to AzureOpenAI, this version maps to "API Version" in the
Azure OpenAI API documentation (https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#rest-api-versioning).
This field is ignored for OpenAI, AWSBedrock, GCPVertexAI, and Anthropic.
For OpenAI and Anthropic, use prefix to configure custom request paths.
See https://aigateway.envoyproxy.io/docs/capabilities/llm-integrations/supported-providers for details.
status object
Status defines the status details of the AIServiceBackend.
conditions []object
Conditions is the list of conditions by the reconciliation result.
Currently, at most one condition is set.
Known .status.conditions.type are: "Accepted", "NotAccepted".
lastTransitionTime
string required
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format:
date-time
message
string required
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength:
32768
observedGeneration
integer
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format:
int64minimum:
0
reason
string required
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
pattern:
^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$minLength:
1maxLength:
1024
status
string required
status of the condition, one of True, False, Unknown.
enum:
True, False, Unknown
type
string required
type of condition in CamelCase or in foo.example.com/CamelCase.
pattern:
^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$maxLength:
316No matches. Try .spec.backendRef for an exact path