Dataplanes
Universal Dataplane fields and Kubernetes Pod annotations that change or disappear between 2.x and 3.0, and what kuma-migrator rewrites.
On Kubernetes the control plane generates every Dataplane from the Pod and
its Service, so the manifests to migrate are the Pod annotations. On Universal the Dataplane
is hand-written and is the manifest. Kuma 3.0 removes most of the 2.x tag-based Dataplane
model, and many removals are silent: the field is dropped on read, the resource still applies,
and traffic changes.
What changed upstream
| Field or annotation | 3.0 change | Silent? |
|---|---|---|
networking.inbound[].tags |
Removed. Use Dataplane labels (UPGRADE) |
Yes: anything selecting on the tags stops matching |
kuma.io/protocol inbound tag |
No longer sets the protocol; set inbound[].protocol (UPGRADE) |
Yes: the inbound becomes plain TCP and loses L7 policies |
networking.outbound[].tags |
Removed; outbounds use backendRef to a MeshService, MeshExternalService or MeshMultiZoneService (UPGRADE) |
Stored ones lose their listener; new writes rejected |
transparentProxying.reachableServices, kuma.io/transparent-proxying-reachable-services |
Removed; use reachableBackends (UPGRADE) |
— |
reachableBackends.refs[] name/namespace |
Labels only (UPGRADE) | Yes: refs resolve to nothing, the proxy gets no outbounds |
No reachableBackends |
A transparent-proxy data plane gets no outbounds, unless defaults.restrictOutbound=false (UPGRADE) |
Yes |
networking.advertisedAddress |
Removed; other proxies use networking.address (UPGRADE) |
Yes |
transparentProxying.redirectPortInbound / redirectPortOutbound |
Deprecated; read from kuma-dp --transparent-proxy instead. Removed in 3.1 (UPGRADE) |
— |
transparentProxying.ipFamilyMode |
Removed | Yes |
transparentProxying.directAccessServices, kuma.io/direct-access-services |
Only * is honoured; named services are ignored (direct_access_proxy_generator.go) |
Yes: the proxy loses direct access to those services |
inbound[].protocol: kafka |
Rejected: Kafka is no longer a protocol (kumahq/kuma#17831) | — |
probes, kuma.io/virtual-probes, kuma.io/virtual-probes-port |
Virtual probes removed; Application Probe Proxy only (UPGRADE) | Yes on Universal |
networking.gateway (any type), kuma.io/gateway |
Removed. See Gateways (UPGRADE) | Yes |
kuma.io/ingress, kuma.io/egress Pod annotations |
Removed (UPGRADE) | — |
kuma.io/mesh annotation |
No longer honoured; use the label (UPGRADE) | Yes: the resource lands in the default mesh |
kuma.io/transparent-proxying-ebpf* |
eBPF transparent proxy removed (UPGRADE) | — |
kuma.io/ and k8s.kuma.io/ labels |
Reserved; an unknown one on a Universal Dataplane fails registration (UPGRADE) |
— |
Sidecar probes on admin port 9901 |
Kubernetes probes on the sidecar use the readiness port (UPGRADE) | — |
Earlier 2.x deprecations still worth checking:
| Annotation or field | Status |
|---|---|
transparentProxying.redirectPortInboundV6 |
Removed in 2.9 |
kuma.io/* annotation values yes/no |
Use true/false (2.9) |
kuma.io/builtindns, kuma.io/builtindnsport |
Not supported and ignored; use kuma.io/builtin-dns, kuma.io/builtin-dns-port |
kuma.io/sidecar-injection as an annotation |
No effect; it must be a label |
prometheus.metrics.kuma.io/port, /path |
Deprecated; use MeshMetric. The aggregate-<name>-* annotations are not deprecated |
What kuma-migrator does
Dataplane is skipped on Kubernetes extractions (the control plane owns it) and extracted on
Universal. Pod annotations are scanned wherever they appear in the input: metadata.annotations,
pod templates (Deployment, StatefulSet, DaemonSet, ReplicaSet, Job) and CronJob job
templates. extract does not export workloads; add their manifests to the input directory to have
them scanned.
| Input | --to-latest v2 |
--to-latest v3 |
|---|---|---|
inbound[].tags |
Nothing (inbound tags are required on 2.x) | Non-kuma.io/ tags moved to labels; kuma.io/service/kuma.io/protocol dropped (reserved prefix); warned |
kuma.io/protocol inbound tag |
— | Copied to inbound[].protocol |
inbound[].protocol: kafka (field or kuma.io/protocol tag) |
Warning | Set to tcp; only the Kafka stats are lost |
Named directAccessServices entries, on the Dataplane or in the kuma.io/direct-access-services annotation |
Warning | With * in the list: reduced to * (the names were redundant). Without *: warning, not rewritten. Use * or drop direct access |
outbound[].tags |
— | Converted to backendRef when the kuma.io/service value decodes to name, namespace and port. The labels include kuma.io/zone: the outbound's own zone tag, else the Dataplane's zone; warned when neither is in the input. Otherwise 3.0 rejects the Dataplane (networking.outbound[0].backendRef: must be defined), so it is written to _manual/ with the backendRef to fill in |
reachableServices |
Warning | Converted to reachableBackends refs with labels where the value decodes; warned otherwise |
reachableBackends refs with name/namespace, on the Dataplane or in the kuma.io/reachable-backends annotation |
Warning | Rewritten to labels (v3 only: on 2.14 a label selector also matches same-named services from other zones) |
Zone of a rewritten MeshService ref |
— | kuma.io/zone set to the Dataplane's own zone. 2.x resolved a service name in the proxy's zone; on 3.0 a labels selector without kuma.io/zone also matches same-named MeshServices from other zones, and an outbound binds the oldest one. MeshExternalService and MeshMultiZoneService refs are left without it. When the zone is not in the input (no kuma.io/zone label, or a pod annotation), the warning says to add it |
Transparent proxy without reachableBackends |
— | Warning: no outbounds on 3.0 |
advertisedAddress, redirect ports, ipFamilyMode, probes |
— | Warning |
networking.gateway |
— | Block removed; non-kuma.io/ gateway tags moved to labels |
redirectPortInboundV6 |
Warning | Warning |
Removed Pod annotations (kuma.io/ingress, kuma.io/egress, kuma.io/mesh, kuma.io/transparent-proxying-reachable-services, kuma.io/transparent-proxying-ebpf*, kuma.io/virtual-probes*) |
Deprecated ones warned | All warned |
| Deprecated Pod annotations (table above) | Warning | Warning |
ContainerPatch |
Copied unchanged | Copied; warning for probes on admin port 9901 (use 9902) |
Code: v3_dataplane.go,
v3_scan.go,
deprecation.go.
What it cannot do
- Find a MeshService port. A Universal outbound
kuma.io/service: backenddoes not say whichMeshServiceport it maps to. The warning gives thebackendRefto fill in. - Choose
reachableBackends. Which services a workload calls is not in itsDataplane. On 3.0 a missing list means no outbounds. - Keep
kuma.io/tags.kuma.io/serviceand other reserved-prefix tags cannot become labels. Policies orMeshServiceselectors that matched on them must select on another label or on theMeshService. - Restart workloads. Pods injected before 3.0 keep stale annotations and sidecar configuration until redeployed.
Manual steps
- Universal: on 2.14, set
inbound[].protocolon every inbound and move non-reserved inbound tags tolabels. - Universal: replace outbound
tagswithbackendRef, and listreachableBackendsrefs by labels for every transparent-proxy data plane. - Make every Universal proxy reachable on
networking.address. - Start
kuma-dpwith--transparent-proxy(or--transparent-proxy-config) and drop the redirect port fields before 3.1. - Kubernetes: replace removed and deprecated Pod annotations; switch
kuma.io/meshandkuma.io/sidecar-injectionto labels. - Upgrade the control plane and all data planes together (CoreDNS is removed from the data plane, UPGRADE), then re-inject every pod.
Example
Input — a Universal Dataplane:
type: Dataplane
mesh: default
name: web-01
networking:
address: 10.0.0.12
advertisedAddress: 192.168.1.12
inbound:
- port: 8080
servicePort: 8081
tags:
kuma.io/service: web
kuma.io/protocol: http
version: v1
outbound:
- port: 33033
tags:
kuma.io/service: backend
transparentProxying:
redirectPortInbound: 15006
redirectPortOutbound: 15001
reachableServices:
- backend
With --to-latest v2 the document is copied unchanged, with one warning about
reachableServices.
With --to-latest v3:
# other/Dataplane-web-01.yaml
labels:
version: v1
mesh: default
name: web-01
networking:
address: 10.0.0.12
advertisedAddress: 192.168.1.12
inbound:
- port: 8080
protocol: http
servicePort: 8081
outbound:
- port: 33033
tags:
kuma.io/service: backend
transparentProxying:
reachableServices:
- backend
redirectPortInbound: 15006
redirectPortOutbound: 15001
type: Dataplane
The inbound is converted; the outbound and reachableServices are not, because backend is a
Universal name with no port. The outbound alone makes 3.0 reject the Dataplane, so it is
written to _manual/ instead of the apply tree. Warnings (shortened):
⚠ 3.0 no longer reads the inbound protocol from the kuma.io/protocol tag … Auto-set
inbound[].protocol from the tag: inbound port 8080=http.
⚠ networking.inbound[].tags is removed in 3.0 and dropped silently on apply … Auto-moved to
Dataplane labels: version=v1. Dropped, because kuma.io/ and k8s.kuma.io/ are reserved label
prefixes: kuma.io/protocol=http, kuma.io/service=web. …
⚠ networking.outbound[].tags is removed in 3.0 … Not converted, because the MeshService port is
not in the manifest: outbound[0] (port 33033): replace tags with
`backendRef: {kind: MeshService, name: backend, port: <MeshService port>}`.
⚠ transparentProxying.reachableServices is removed in 3.0 … these entries could not be
converted: [backend]. …
⚠ networking.advertisedAddress (192.168.1.12) is removed in 3.0 and dropped silently. …
⚠ transparentProxying.redirectPortInbound / redirectPortOutbound still works on 3.0 but is
removed in 3.1. …
⚠ transparent proxying without reachableBackends. On 3.0 such a proxy gets no generated
outbounds and cannot reach any service …
Related
- MeshService —
reachableBackendsand display names - Gateways, Gateway API and GAMMA —
networking.gatewayand delegated gateways - Multizone — zone proxies as
Dataplanes