Policy targeting
Which targetRef kinds each policy accepts at each position on 2.14 and 3.0, labels-only selection, policy roles, and control-plane-owned labels.
A targetRef policy selects proxies in spec.targetRef (top level), destinations in
spec.to[].targetRef, and clients in spec.from[].targetRef or spec.rules[]. Each position
accepts a different set of kinds, and 3.0 narrows all three. It also changes how a real resource
is selected (labels only) and which labels you may set on the policy itself.
What changed upstream
| Version | Change |
|---|---|
| 2.7 | from[].targetRef.kind: MeshService deprecated on MeshTrafficPermission/MeshFaultInjection (UPGRADE) |
| 2.10 / 2.11 | Top-level MeshSubset, MeshService, MeshServiceSubset and MeshHTTPRoute deprecated (validators.TopLevelTargetRefDeprecations) |
| 2.10 | rules[] API for inbound policies (MeshTimeout, MeshCircuitBreaker, MeshRateLimit, MeshAccessLog, MeshTLS) |
| 3.0 | Top-level MeshSubset, MeshService, MeshServiceSubset rejected. Use Mesh or Dataplane + labels. Applies to KDS sync too: a policy with a legacy kind can fail to sync into an upgraded zone (UPGRADE) |
| 3.0 | MeshGateway is no longer a valid targetRef.kind (UPGRADE) |
| 3.0 | from[] removed from MeshTimeout, MeshCircuitBreaker, MeshAccessLog, MeshRateLimit, MeshTLS, MeshTrafficPermission (UPGRADE) |
| 3.0 | Dataplane, MeshService, MeshExternalService, MeshMultiZoneService and MeshHTTPRoute are selected by labels only in spec.targetRef and spec.to[].targetRef; sectionName stays (UPGRADE) |
| 3.0 | Fields not in a policy's schema are rejected with 400 (spec.from: unknown field) by the API server and kumactl apply; kubectl apply prunes them with a warning (UPGRADE) |
| 3.0 | spec.targetRef.proxyTypes removed (no field in targetref.go). Kubernetes prunes it and the API server rejects it as an unknown field. A stored policy selects every proxy. The 2.x default mesh-gateways-timeout-all-<mesh> (proxyTypes: [Gateway], streamIdleTimeout: 5s) then fails every sidecar response that is idle for 5s with 504 (kumahq/kuma#18857) |
| 3.0 | Control-plane-owned labels rejected when they differ from the computed value (UPGRADE) |
| 3.0 | kuma.io/ and k8s.kuma.io/ are reserved label prefixes; unknown labels under them are rejected (UPGRADE) |
Kinds accepted per policy
Extracted from every pkg/plugins/policies/*/api/v1alpha1/valid*.go validator on Kuma master
(3.0) and, for the last column, at v2.14.4. Kong Mesh adds MeshOPA.
| Policy | top-level targetRef (3.0) |
to[] (3.0) |
additional top-level kinds in 2.14.4 |
|---|---|---|---|
| MeshAccessLog | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService, MeshHTTPRoute | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshCircuitBreaker | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService | MeshSubset, MeshService, MeshGateway, MeshServiceSubset |
| MeshFaultInjection | Mesh, Dataplane | Mesh only | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshHealthCheck | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshHTTPRoute | Mesh, Dataplane | MeshService, MeshExternalService, MeshMultiZoneService (no Mesh) | MeshGateway, MeshSubset, MeshService, MeshServiceSubset |
| MeshLoadBalancingStrategy | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService, MeshHTTPRoute | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshMetric | Mesh, Dataplane | — | MeshSubset, MeshService, MeshServiceSubset, MeshGateway |
| MeshPassthrough | Mesh, Dataplane | — | MeshSubset |
| MeshProxyPatch | Mesh, Dataplane | — | MeshSubset, MeshService, MeshServiceSubset, MeshGateway |
| MeshRateLimit | Mesh, Dataplane | Mesh only | MeshSubset, MeshGateway, MeshService, MeshServiceSubset, MeshHTTPRoute |
| MeshRetry | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService, MeshHTTPRoute | MeshSubset, MeshGateway, MeshService, MeshServiceSubset, MeshHTTPRoute |
| MeshTCPRoute | Mesh, Dataplane | MeshService, MeshExternalService, MeshMultiZoneService (no Mesh) | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshTimeout | Mesh, Dataplane | Mesh, MeshService, MeshExternalService, MeshMultiZoneService, MeshHTTPRoute | MeshSubset, MeshGateway, MeshService, MeshServiceSubset, MeshHTTPRoute |
| MeshTLS | Mesh, Dataplane | — | MeshSubset |
| MeshTrace | Mesh, Dataplane | — | MeshSubset, MeshGateway, MeshService, MeshServiceSubset |
| MeshTrafficPermission | Mesh, Dataplane | — (rules[]) |
MeshSubset, MeshService, MeshServiceSubset |
| MeshOPA (Kong Mesh) | Mesh, Dataplane | — | MeshService |
The top-level rule on 3.0 is uniform: every policy accepts Mesh and Dataplane, with no
per-policy and no system-role exception. to[] has four shapes:
| Shape | Kinds | Policies |
|---|---|---|
| Outbound and route anchor | Mesh, the three service kinds, MeshHTTPRoute | MeshAccessLog, MeshLoadBalancingStrategy, MeshRetry, MeshTimeout |
| Outbound | Mesh, the three service kinds | MeshCircuitBreaker, MeshHealthCheck |
| Mesh only | Mesh | MeshFaultInjection, MeshRateLimit (inbound policies) |
| Route backends | The three service kinds, no Mesh | MeshHTTPRoute, MeshTCPRoute |
No policy accepts MeshSubset, MeshServiceSubset or MeshGateway in to[] on 3.0. Route
backendRefs accept MeshService, MeshExternalService and MeshMultiZoneService
(UPGRADE).
MeshTimeout and MeshRetry still contain a validator branch for a top-level MeshHTTPRoute
that no input can reach on 3.0.
Labels only
On 3.0 a reference to a real resource carries kind, labels and optionally sectionName:
| 2.x | 3.0 |
|---|---|
name: backend |
labels: {kuma.io/display-name: backend} |
namespace: demo |
labels: {k8s.kuma.io/namespace: demo} |
mesh: default |
labels: {kuma.io/mesh: default} |
sectionName: http |
sectionName: http (unchanged) |
A label selector matches every resource that carries the labels. On 2.14 a selector with only
kuma.io/display-name and k8s.kuma.io/namespace also matches same-named services synced from
other zones; add kuma.io/zone to pin one.
Policy roles
The control plane sets kuma.io/policy-role on every policy. Precedence, low to high:
system → producer → consumer → workload-owner
(docs).
| Role | When (2.x docs) |
|---|---|
system |
Policy in the system namespace (kuma-system, kong-mesh-system) |
workload-owner |
No spec.to (only spec.from or rules[], or neither) |
producer |
spec.to[].targetRef names services in the policy's own namespace |
consumer |
spec.to[] selects services by labels or in another namespace |
A producer policy applies to every client of the service and is synced to other zones; a consumer policy applies to clients in its own namespace and wins over a producer.
v3: On 3.0 the role is computed from labels. A
to[]item counts as producer only when it selects aMeshServiceorMeshHTTPRoutewith exactly three labels:kuma.io/display-name,k8s.kuma.io/namespaceequal to the policy's namespace, andkuma.io/zoneequal to the local zone. A policy mixing producer and consumer items is rejected (compute.go). The policy's own zone is thekuma.io/zonethe control plane computes for it, so a producer policy only exists on a Zone CP.
--to-latest v3treats a policy as a producer when itskuma.io/policy-rolelabel saysproducer, or, with no role label, when it is in an app namespace and everyto[]item names aMeshServiceorMeshHTTPRoutein that same namespace. For such a policy it addskuma.io/zoneto everyto[]label selector, using the policy's ownkuma.io/zonelabel (policies extracted from a Zone CP carry it). When the policy has nokuma.io/zonelabel the zone is not in the manifest: the warning names eachto[]selector to which you must addkuma.io/zone: <zone-name>, the zone CP you apply the policy to. Without it the policy becomes a consumer policy that only applies to clients in its own namespace. A selector with labels beyond the three is also warned, because it makes the policy a consumer.
Control-plane-owned labels
The control plane computes these labels on every write: kuma.io/origin, kuma.io/zone,
kuma.io/env, kuma.io/mesh, kuma.io/display-name, kuma.io/policy-role,
k8s.kuma.io/namespace, k8s.kuma.io/service-account, kuma.io/listener-zoneingress,
kuma.io/listener-zoneegress. On 3.0 a supplied value that differs from the computed one is
rejected. Extracted manifests carry the stored values, which become wrong as soon as the migration
renames, splits or moves a policy (a split policy changes role, for example).
What kuma-migrator does
| Input | --to-latest v2 |
--to-latest v3 |
|---|---|---|
Top-level MeshSubset with a service tag |
Dataplane with labels: {app: <name>, k8s.kuma.io/namespace: <ns>}, never name (see MeshService) |
Same |
| Tags copied into a label selector | Kuma-reserved tags that are not labels (kuma.io/protocol, kuma.io/instance, k8s.kuma.io/service-port, …) are dropped, with a warning naming them |
Same |
Top-level MeshSubset without a service tag |
Dataplane with the tags as labels |
Same |
Top-level MeshService / MeshServiceSubset |
Warning | Error: written to _manual/. name → labels, kind unchanged |
Top-level MeshHTTPRoute |
Warning: move to to[].targetRef |
Error: written to _manual/ |
Top-level MeshGateway |
Nothing (valid on 2.14 for system-role policies) | Error: written to _manual/ |
| Any other top-level kind | — | Error: written to _manual/ |
to[] kind not accepted by that policy on 3.0 (matrix above) |
— | Error naming the accepted kinds; written to _manual/ |
targetRef, to[].targetRef, route backendRefs with name/namespace/mesh |
Kept | Rewritten to labels. Dataplane{name} cannot be (a Dataplane name is not a label): error, written to _manual/ |
Producer policy (to[] names services in its own namespace) |
Kept | kuma.io/zone added to each to[] selector from the policy's own kuma.io/zone label; warning naming the selectors when that label is absent |
from[] on MeshTimeout/MeshCircuitBreaker/MeshRateLimit/MeshAccessLog/MeshTLS |
rules[] (Rules scenario). A source that is not kind: Mesh is collapsed, with a warning |
Same |
rules[] and to[] in one policy |
Split into <name> and <name>-outbound |
Same |
Control-plane-owned labels on the resource's own metadata.labels |
Kept | Removed (kuma.io/mesh stays). Selectors are not touched |
spec.targetRef.proxyTypes [Sidecar] |
Warning | Removed, with a warning: the policy now also selects the proxies that are gateways on 2.x, which 3.0 runs as ordinary Dataplanes |
spec.targetRef.proxyTypes with both values, an empty list, or on a kind other than Mesh/MeshSubset (2.x ignores it there) |
Warning | Removed; the scope does not change |
spec.targetRef.proxyTypes [Gateway] |
Warning | Error: written to _manual/. Without the field the policy selects every proxy; select the gateway Dataplanes by labels or delete the policy |
Default MeshTimeout mesh-gateways-timeout-all-<mesh> |
Advisory: delete it right before the 3.0 upgrade | Not converted. The original goes to _manual/ and is listed under Delete Before Upgrading to 3.0. Deleting it, not dropping proxyTypes, is the fix |
A document that 3.0 would reject as written is never put in the apply tree: the run reports it as
an error and writes it under _manual/, the same as a conversion that failed.
Code: convert.go
(convertTargetRef, selectorLabelsFromTags),
v3_policy.go
(stripControlPlaneLabels, fixRealResourceTargetRefsForV3, pinProducerZoneForV3,
warnV3ToTargetRefKinds),
rejected.go
(RejectedByTargetError),
deprecation.go
(warnDeprecatedTopLevelTargetRef),
rulesapi.go.
What it cannot do
- Expand a
MeshServiceselector intoDataplanelabels. A top-levelMeshServicenames a service; the equivalentDataplaneselector is the set of pod labels behind it, which is not in the policy. Only Kuma-internal_svc_names are expanded (toapp: <name>, warned). - Re-target
MeshGatewaypolicies. The delegated gateway's labels depend on how you deploy it. See Gateways. - Replace
proxyTypes: [Gateway]. 3.0 has no gateway proxy type. The gatewayDataplanes are selected by their own labels, which are not in the policy. - Prove that a label selector selects one resource. Labels match every resource that carries them. Each rewrite is warned.
Manual steps
- Run
migrate --dry-run --to-latest v3and list every top-leveltargetRefwarning. - On 2.14, rewrite top-level
MeshService/MeshServiceSubset/MeshSubsettoMeshorDataplane+labels, and top-levelMeshHTTPRoutetoto[].targetRef. Verify coverage in every zone before upgrading zone control planes. - Fix
to[]entries the matrix rejects. - For producer policies without a
kuma.io/zonelabel, addkuma.io/zone: <zone-name>to theto[]label selectors the warning names. - Delete
mesh-gateways-timeout-all-<mesh>in every mesh before the upgrade. On Kubernetes, first addMeshTimeout(or*) to the Mesh'sspec.skipCreatingInitialPolicies: the 2.14 Mesh controller recreates the default on the next Mesh reconcile. 3.0 ignores that field. Rewrite any otherproxyTypes: [Gateway]policy tokind: Dataplane+ the gateway's labels. - Apply the
v3output, which strips control-plane-owned labels.
Example
Input — two files:
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
name: backend-timeouts
namespace: demo
labels:
kuma.io/mesh: default
kuma.io/origin: zone
kuma.io/zone: zone-1
kuma.io/policy-role: producer
k8s.kuma.io/namespace: demo
spec:
targetRef:
kind: MeshService
name: backend
to:
- targetRef:
kind: MeshService
name: backend
sectionName: http
default:
http:
requestTimeout: 10s
---
apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
name: cb
namespace: kong-mesh-system
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: MeshGateway
name: edge-gateway
to:
- targetRef:
kind: MeshHTTPRoute
name: some-route
default:
connectionLimits:
maxConnections: 100
--to-latest v2 copies both unchanged; the MeshTimeout gets one warning:
⚠ MeshTimeout "backend-timeouts": spec.targetRef.kind MeshService is deprecated as a top-level
target in Kuma 2.10+ — use kind: Dataplane with labels instead.
--to-latest v3 converts both, but 3.0 rejects both as converted (the top-level kinds), so the run
reports each as an error and writes it under _manual/ instead of the apply tree:
# _manual/resiliency/MeshTimeout-demo-backend-timeouts.yaml
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
labels:
kuma.io/mesh: default
name: backend-timeouts
namespace: demo
spec:
targetRef:
kind: MeshService
labels:
k8s.kuma.io/namespace: demo
kuma.io/display-name: backend
kuma.io/zone: zone-1
to:
- default:
http:
requestTimeout: 10s
targetRef:
kind: MeshService
labels:
k8s.kuma.io/namespace: demo
kuma.io/display-name: backend
kuma.io/zone: zone-1
sectionName: http
# _manual/resiliency/MeshCircuitBreaker-kong-mesh-system-cb.yaml
apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
labels:
kuma.io/mesh: default
name: cb
namespace: kong-mesh-system
spec:
targetRef:
kind: MeshGateway
name: edge-gateway
to:
- default:
connectionLimits:
maxConnections: 100
targetRef:
kind: MeshHTTPRoute
labels:
k8s.kuma.io/namespace: kong-mesh-system
kuma.io/display-name: some-route
The control-plane-owned labels are gone and selectors use labels. The MeshTimeout producer
selector keeps kuma.io/zone: zone-1 from the policy's own label, so it stays a producer policy.
The warnings say why each policy is still invalid on 3.0 (shortened):
⚠ MeshTimeout "backend-timeouts": spec.targetRef.kind MeshService is deprecated as a top-level
target … Kuma 3.0 rejects it outright: Mesh and Dataplane are the only top-level targetRef kinds
any policy accepts.
⚠ MeshTimeout "backend-timeouts": removed control-plane-owned labels kuma.io/origin, kuma.io/zone,
kuma.io/policy-role, k8s.kuma.io/namespace. …
⚠ MeshCircuitBreaker "cb": spec.targetRef.kind MeshGateway is removed in Kuma 3.0 … Re-target the
policy at the delegated gateway's own proxies (kind: Dataplane with labels).
⚠ MeshCircuitBreaker "cb": spec.targetRef (kind MeshGateway) sets name/namespace/mesh, which Kuma
3.0 removes from targetRef. Remove them.
⚠ MeshCircuitBreaker "cb": spec.to[0].targetRef.kind MeshHTTPRoute is rejected by Kuma 3.0.
MeshCircuitBreaker accepts Mesh, MeshExternalService, MeshMultiZoneService, MeshService in to[] …
Related
- MeshService —
kuma.io/serviceselectors - Gateways, Gateway API and GAMMA
- Legacy policies —
sources/destinations→targetRef - Access control —
from[]vsrules[]onMeshTrafficPermission