QuotaPolicy
aigateway.envoyproxy.io / v1alpha1
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: QuotaPolicy
metadata:
name: example
apiVersion
string
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind
string
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata
object
spec object
QuotaPolicySpec specifies rules for computing token based costs of requests.
perModelQuotas []object
PerModelQuotas specifies quota for different models served by the AIServiceBackend(s) where this
policy is attached.
When multiple QuotaPolicies define the same Model for the same AIServiceBackend,
the policy whose namespace/name is alphabetically first takes precedence.
Keys are sorted to ensure deterministic snapshot generation.
maxItems:
128
modelName
string required
Model name for which the quota is specified.
This must match the ModelNameOverride set in the AIGatewayRoute's BackendRef
for the quota policy to apply to that backend.
minLength:
1quota object required
Expression for computing request cost and rules for matching requests to quota buckets.
bucketRules []object
BucketRules are a list of client selectors and quotas. If a request
matches multiple rules, each of their associated quotas get applied, so a
single request might burn down the quota for multiple rules.
Client selectors that match under the same model / service backend will be
combined with the first limit taking precedence.
clientSelectors []object
ClientSelectors holds the list of conditions to select
specific clients using attributes from the traffic flow.
All individual select conditions must hold True for this rule
and its limit to be applied.
Currently, only header types are honored.
If no client selectors are specified, the rule applies to all traffic of
the targeted AIServiceBackend.
maxItems:
8headers []object
Headers is a list of request headers to match. Multiple header values are ANDed together,
meaning, a request MUST match all the specified headers.
maxItems:
64
invert
boolean
Invert specifies whether the value match result will be inverted.
Do not set this field when Type="Distinct", implying matching on any/all unique
values within the header.
name
string required
Name of the HTTP header.
The header name is case-insensitive unless PreserveHeaderCase is set to true.
For example, "Foo" and "foo" are considered the same header.
minLength:
1maxLength:
256
type
string
Type specifies how to match against the value of the header.
enum:
Exact, RegularExpression, Distinct
value
string
Value within the HTTP header.
Do not set this field when Type="Distinct", implying matching on any/all unique
values within the header.
maxLength:
1024methods []object
Methods is a list of request methods to match. Multiple method values are ORed together,
meaning, a request can match any one of the specified methods. If not specified, it matches all methods.
invert
boolean
Invert specifies whether the value match result will be inverted.
value
string required
Value specifies the HTTP method.
enum:
GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCHpath object
Path is the request path to match.
Support Exact, PathPrefix and RegularExpression match types.
invert
boolean
Invert specifies whether the value match result will be inverted.
type
string
Type specifies how to match against the value of the path.
enum:
Exact, PathPrefix, RegularExpression
value
string required
Value specifies the HTTP path.
maxLength:
1024queryParams []object
QueryParams is a list of query parameters to match. Multiple query parameter values are ANDed together,
meaning, a request MUST match all the specified query parameters.
maxItems:
16
invert
boolean
Invert specifies whether the value match result will be inverted.
Do not set this field when Type="Distinct", implying matching on any/all unique
values within the query parameter.
name
string required
Name of the query parameter.
minLength:
1maxLength:
256
type
string
Type specifies how to match against the value of the query parameter.
enum:
Exact, RegularExpression, Distinct
value
string
Value of the query parameter.
Do not set this field when Type="Distinct", implying matching on any/all unique
values within the query parameter.
maxLength:
1024sourceCIDR object
SourceCIDR is the client IP Address range to match on.
invert
boolean
Invert specifies whether the source range match result will be inverted.
When true, the rule matches when the client IP is not in the specified range(s).
type
string
enum:
Exact, Distinct
value
string required
Value is the IP CIDR that represents the range of Source IP Addresses of the client.
These could also be the intermediate addresses through which the request has flown through and is part of the `X-Forwarded-For` header.
For example, `192.168.0.1/32`, `192.168.0.0/24`, `001:db8::/64`.
minLength:
1maxLength:
256quota object required
Quota value for given client selectors.
This quota is applied for traffic flows when the selectors
compute to True, causing the request to be counted towards the limit.
A response with 429 HTTP status code is sent back to the client when
the selected requests have exceeded the quota.
duration
string required
Time window. Must be exactly one of: "1s" (1 second), "1m" (1 minute), "1h" (1 hour), or "1d" (1 day).
enum:
1s, 1m, 1h, 1d
limit
integer required
The limit alloted for a specified time window.
shadowMode
boolean
ShadowMode indicates whether this quota rule runs in shadow mode.
When enabled, all quota checks are performed (cache lookups,
counter updates, telemetry generation), but the outcome is never enforced.
The request always succeeds, even if the configured quota is exceeded.
costExpression
string
CostExpression specifies a CEL expression for computing the quota burndown of the LLM-related request.
If no expression is specified the "total_tokens" value is used.
For example:
"input_tokens + cached_input_tokens + output_tokens"
defaultBucket object
Quota applicable to all traffic. This value can be overridden for specific classes of requests
using the "BucketRules" configuration.
duration
string required
Time window. Must be exactly one of: "1s" (1 second), "1m" (1 minute), "1h" (1 hour), or "1d" (1 day).
enum:
1s, 1m, 1h, 1d
limit
integer required
The limit alloted for a specified time window.
mode
string
The "Mode" determines how quota is charged to the "DefaultBucket" and matching "BucketRules".
In the "shared" mode the quota is charged to all matching "BucketRules" AND the "DefaultBucket"
and request is allowed only if the quota is available in at least one matching buckets.
Defaults to "Shared".
enum:
SharedserviceQuota object
Quota for all models served by AIServiceBackend(s). This value can be overridden for specific models using the "PerModelQuotas"
configuration.
Currently, the rate limit configuration is properly set up, but the descriptor set is not being set in the Envoy Configuration
costExpression
string
CostExpression specifies a CEL expression for computing the quota burndown of the LLM-related request.
If no expression is specified the "total_tokens" value is used.
For example:
"input_tokens + cached_input_tokens + output_tokens"
quota object required
Quota value applicable to all requests.
A response with 429 HTTP status code is sent back to the client when
the selected requests have exceeded the quota.
duration
string required
Time window. Must be exactly one of: "1s" (1 second), "1m" (1 minute), "1h" (1 hour), or "1d" (1 day).
enum:
1s, 1m, 1h, 1d
limit
integer required
The limit alloted for a specified time window.
targetRefs []object
TargetRefs are the names of the AIServiceBackend resources this QuotaPolicy is being attached to.
maxItems:
16
group
string required
Group is the group of the target resource.
pattern:
^$|^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$maxLength:
253
kind
string required
Kind is kind of the target resource.
pattern:
^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$minLength:
1maxLength:
63
name
string required
Name is the name of the target resource.
minLength:
1maxLength:
253status object
Status defines the status details of the QuotaPolicy.
conditions []object
Conditions is the list of conditions by the reconciliation result.
Currently, at most one condition is set.
Known .status.conditions.type are: "Accepted", "NotAccepted".
lastTransitionTime
string required
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format:
date-time
message
string required
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength:
32768
observedGeneration
integer
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format:
int64minimum:
0
reason
string required
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
pattern:
^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$minLength:
1maxLength:
1024
status
string required
status of the condition, one of True, False, Unknown.
enum:
True, False, Unknown
type
string required
type of condition in CamelCase or in foo.example.com/CamelCase.
pattern:
^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$maxLength:
316No matches. Try .spec.perModelQuotas for an exact path