Skip to content

Commit 88891a9

Browse files
Merge pull request #2907 from tsorya/igal/dpu-host-cleanup
NVIDIA-554: DPU-host mode: use ConfigMap for OVN feature enablement instead of per-node script gating
2 parents 0341016 + dfa1262 commit 88891a9

10 files changed

Lines changed: 144 additions & 560 deletions

File tree

README.md

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -157,18 +157,6 @@ OVNKubernetes supports the following configuration options, all of which are opt
157157
* `egressIPConfig`: holds the configuration for EgressIP options.
158158
* `reachabilityTotalTimeoutSeconds`: Set EgressIP node reachability total timeout in seconds, 0 means disable reachability check and the default is 1 second.
159159

160-
#### DPU Host Mode Support
161-
162-
OVN-Kubernetes supports specialized hardware deployments such as DPU (Data Processing Unit) hosts through the `OVN_NODE_MODE` environment variable. In `dpu-host` mode, certain features are automatically disabled on those nodes regardless of cluster-wide configuration:
163-
164-
- Egress IP and related features (egress firewall, egress QoS, egress service)
165-
- Multicast support
166-
- Multi-external gateway support
167-
- Multi-network policies and admin network policies
168-
- Network segmentation features
169-
170-
This per-node feature enforcement is implemented through conditional logic in the startup scripts, allowing the same cluster configuration to work across heterogeneous node types. For detailed information about node modes and the technical implementation, see `docs/ovn_node_mode.md`.
171-
172160
These configuration flags are only in the Operator configuration object.
173161

174162
Example from the `manifests/cluster-network-03-config.yml` file:

bindata/network/ovn-kubernetes/common/008-script-lib.yaml

Lines changed: 2 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -537,16 +537,8 @@ data:
537537

538538
echo "I$(date "+%m%d %H:%M:%S.%N") - starting ovnkube-node"
539539

540-
# enable egress ip, egress firewall, egress qos, egress service
541-
egress_features_enable_flag="--enable-egress-ip=true --enable-egress-firewall=true --enable-egress-qos=true --enable-egress-service=true"
542540
init_ovnkube_controller="--init-ovnkube-controller ${K8S_NODE}"
543-
multi_external_gateway_enable_flag="--enable-multi-external-gateway=true"
544-
gateway_interface=br-ex
545-
546-
# enable multicast
547-
enable_multicast_flag="--enable-multicast"
548-
549-
# Use OVN_NODE_MODE environment variable, default to "full" if not set
541+
gateway_interface="br-ex"
550542
OVN_NODE_MODE=${OVN_NODE_MODE:-full}
551543
# We check only dpu-host mode and not smart-nic mode here as currently we do not support it yet
552544
# Once we support it, we will need to check for it here and add relevant code.
@@ -555,17 +547,10 @@ data:
555547
# https://github.com/ovn-kubernetes/ovn-kubernetes/pull/5327/files
556548
gateway_interface="derive-from-mgmt-port"
557549
ovnkube_node_mode="--ovnkube-node-mode dpu-host"
558-
# disable egress ip for dpu-host mode as it is not supported
559-
egress_features_enable_flag=""
560-
561-
# disable multicast for dpu-host mode as it is not supported
562-
enable_multicast_flag=""
563550

564551
# disable init-ovnkube-controller for dpu-host mode as it is not supported
565552
init_ovnkube_controller=""
566553

567-
# disable multi-external-gateway for dpu-host mode as it is not supported
568-
multi_external_gateway_enable_flag=""
569554
fi
570555

571556
if [ "{{.OVN_GATEWAY_MODE}}" == "shared" ]; then
@@ -610,17 +595,6 @@ data:
610595
node_mgmt_port_netdev_flags="$node_mgmt_port_netdev_flags --ovnkube-node-mgmt-port-dp-resource-name ${OVNKUBE_NODE_MGMT_PORT_DP_RESOURCE_NAME}"
611596
fi
612597

613-
multi_network_enabled_flag=
614-
if [[ "{{.OVN_MULTI_NETWORK_ENABLE}}" == "true" && "${OVN_NODE_MODE}" != "dpu-host" ]]; then
615-
multi_network_enabled_flag="--enable-multi-network"
616-
fi
617-
618-
network_segmentation_enabled_flag=
619-
if [[ "${OVN_NODE_MODE}" != "dpu-host" ]]; then
620-
multi_network_enabled_flag="--enable-multi-network"
621-
network_segmentation_enabled_flag="--enable-network-segmentation"
622-
fi
623-
624598
route_advertisements_enable_flag=
625599
if [[ "{{.OVN_ROUTE_ADVERTISEMENTS_ENABLE}}" == "true" ]]; then
626600
route_advertisements_enable_flag="--enable-route-advertisements"
@@ -630,29 +604,12 @@ data:
630604
if [[ "{{.OVN_EVPN_ENABLE}}" == "true" ]]; then
631605
evpn_enable_flag="--enable-evpn"
632606
fi
633-
634-
preconfigured_udn_addresses_enable_flag="--enable-preconfigured-udn-addresses"
635607

636608
network_observability_enabled_flag=
637609
if [[ "{{.OVN_OBSERVABILITY_ENABLE}}" == "true" ]]; then
638610
network_observability_enabled_flag="--enable-observability"
639611
fi
640612

641-
multi_network_policy_enabled_flag=
642-
if [[ "{{.OVN_MULTI_NETWORK_POLICY_ENABLE}}" == "true"&& "${OVN_NODE_MODE}" != "dpu-host" ]]; then
643-
multi_network_policy_enabled_flag="--enable-multi-networkpolicy"
644-
fi
645-
646-
admin_network_policy_enabled_flag=
647-
if [[ "${OVN_NODE_MODE}" != "dpu-host" ]]; then
648-
admin_network_policy_enabled_flag="--enable-admin-network-policy"
649-
fi
650-
651-
dns_name_resolver_enabled_flag=
652-
if [[ "{{.DNS_NAME_RESOLVER_ENABLE}}" == "true" ]]; then
653-
dns_name_resolver_enabled_flag="--enable-dns-name-resolver"
654-
fi
655-
656613
# If IP Forwarding mode is global set it in the host here. IPv6 IP Forwarding shuld be
657614
# enabled for all interfaces at all times if cluster is configured as single stack IPv6
658615
# or dual stack. This will be taken care by ovn-kubernetes(ovn-org/ovn-kubernetes#4376).
@@ -729,16 +686,9 @@ data:
729686
--export-ovs-metrics \
730687
--disable-snat-multiple-gws \
731688
${export_network_flows_flags} \
732-
${multi_network_enabled_flag} \
733-
${network_segmentation_enabled_flag} \
734689
${route_advertisements_enable_flag} \
735690
${evpn_enable_flag} \
736-
${preconfigured_udn_addresses_enable_flag} \
737-
${multi_network_policy_enabled_flag} \
738-
${admin_network_policy_enabled_flag} \
739-
${dns_name_resolver_enabled_flag} \
740691
${network_observability_enabled_flag} \
741-
${enable_multicast_flag} \
742692
--zone ${K8S_NODE} \
743693
--enable-interconnect \
744694
--acl-logging-rate-limit "{{.OVNPolicyAuditRateLimit}}" \
@@ -752,7 +702,5 @@ data:
752702
${ovn_v4_masquerade_subnet_opt} \
753703
${ovn_v6_masquerade_subnet_opt} \
754704
${ovn_v4_transit_switch_subnet_opt} \
755-
${ovn_v6_transit_switch_subnet_opt} \
756-
${egress_features_enable_flag} \
757-
${multi_external_gateway_enable_flag}
705+
${ovn_v6_transit_switch_subnet_opt}
758706
}

bindata/network/ovn-kubernetes/managed/004-config.yaml

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -33,14 +33,23 @@ data:
3333
dns-service-name="dns-default"
3434

3535
[ovnkubernetesfeature]
36+
enable-egress-ip=true
37+
enable-egress-firewall=true
38+
enable-egress-qos=true
39+
enable-egress-service=true
3640
{{- if .ReachabilityNodePort }}
3741
egressip-node-healthcheck-port={{.ReachabilityNodePort}}
3842
{{- end }}
39-
{{- if not .OVN_MULTI_NETWORK_ENABLE }}
4043
enable-multi-network=true
41-
{{- end }}
4244
enable-network-segmentation=true
4345
enable-preconfigured-udn-addresses=true
46+
47+
{{- if .OVN_MULTI_NETWORK_POLICY_ENABLE }}
48+
enable-multi-networkpolicy=true
49+
{{- end }}
50+
enable-admin-network-policy=true
51+
enable-multi-external-gateway=true
52+
enable-multicast=true
4453
{{- if .DNS_NAME_RESOLVER_ENABLE }}
4554
enable-dns-name-resolver=true
4655
{{- end }}
@@ -117,14 +126,15 @@ data:
117126
{{- if .ReachabilityNodePort }}
118127
egressip-node-healthcheck-port={{.ReachabilityNodePort}}
119128
{{- end }}
120-
{{- if .OVN_MULTI_NETWORK_ENABLE }}
121-
enable-multi-network=true
122-
{{- end }}
123-
{{- if not .OVN_MULTI_NETWORK_ENABLE }}
124129
enable-multi-network=true
125-
{{- end }}
126130
enable-network-segmentation=true
127131
enable-preconfigured-udn-addresses=true
132+
{{- if .OVN_MULTI_NETWORK_POLICY_ENABLE }}
133+
enable-multi-networkpolicy=true
134+
{{- end }}
135+
enable-admin-network-policy=true
136+
enable-multi-external-gateway=true
137+
enable-multicast=true
128138
{{- if .DNS_NAME_RESOLVER_ENABLE }}
129139
enable-dns-name-resolver=true
130140
{{- end }}

bindata/network/ovn-kubernetes/managed/ovnkube-control-plane.yaml

Lines changed: 1 addition & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -183,19 +183,8 @@ spec:
183183
ovn_v6_masquerade_subnet_opt="--gateway-v6-masquerade-subnet {{.V6MasqueradeSubnet}}"
184184
fi
185185
186-
dns_name_resolver_enabled_flag=
187-
if [[ "{{.DNS_NAME_RESOLVER_ENABLE}}" == "true" ]]; then
188-
dns_name_resolver_enabled_flag="--enable-dns-name-resolver"
189-
fi
190-
191186
persistent_ips_enabled_flag="--enable-persistent-ips"
192187
193-
# This is needed so that converting clusters from GA to TP
194-
# will rollout control plane pods as well
195-
network_segmentation_enabled_flag=
196-
multi_network_enabled_flag="--enable-multi-network"
197-
network_segmentation_enabled_flag="--enable-network-segmentation"
198-
199188
route_advertisements_enable_flag=
200189
if [[ "{{.OVN_ROUTE_ADVERTISEMENTS_ENABLE}}" == "true" ]]; then
201190
route_advertisements_enable_flag="--enable-route-advertisements"
@@ -206,17 +195,6 @@ spec:
206195
evpn_enable_flag="--enable-evpn"
207196
fi
208197
209-
preconfigured_udn_addresses_enable_flag="--enable-preconfigured-udn-addresses"
210-
211-
# Enable multi-network policy if configured (control-plane always full mode)
212-
multi_network_policy_enabled_flag=
213-
if [[ "{{.OVN_MULTI_NETWORK_POLICY_ENABLE}}" == "true" ]]; then
214-
multi_network_policy_enabled_flag="--enable-multi-networkpolicy"
215-
fi
216-
217-
# Enable admin network policy if configured (control-plane always full mode)
218-
admin_network_policy_enabled_flag="--enable-admin-network-policy"
219-
220198
echo "I$(date "+%m%d %H:%M:%S.%N") - ovnkube-control-plane - start ovnkube --init-cluster-manager ${K8S_NODE}"
221199
exec /usr/bin/ovnkube \
222200
--enable-interconnect \
@@ -235,21 +213,9 @@ spec:
235213
${ovn_v6_transit_switch_subnet_opt} \
236214
${ovn_v4_masquerade_subnet_opt} \
237215
${ovn_v6_masquerade_subnet_opt} \
238-
${dns_name_resolver_enabled_flag} \
239216
${persistent_ips_enabled_flag} \
240-
${multi_network_enabled_flag} \
241-
${network_segmentation_enabled_flag} \
242217
${route_advertisements_enable_flag} \
243-
${evpn_enable_flag} \
244-
${preconfigured_udn_addresses_enable_flag} \
245-
--enable-egress-ip=true \
246-
--enable-egress-firewall=true \
247-
--enable-egress-qos=true \
248-
--enable-egress-service=true \
249-
--enable-multicast \
250-
--enable-multi-external-gateway=true \
251-
${multi_network_policy_enabled_flag} \
252-
${admin_network_policy_enabled_flag}
218+
${evpn_enable_flag}
253219
volumeMounts:
254220
- mountPath: /run/ovnkube-config/
255221
name: ovnkube-config

bindata/network/ovn-kubernetes/self-hosted/004-config.yaml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,21 +33,25 @@ data:
3333
dns-service-name="dns-default"
3434

3535
[ovnkubernetesfeature]
36-
36+
enable-egress-ip=true
37+
enable-egress-firewall=true
38+
enable-egress-qos=true
39+
enable-egress-service=true
3740
{{- if .ReachabilityTotalTimeoutSeconds }}
3841
egressip-reachability-total-timeout={{.ReachabilityTotalTimeoutSeconds}}
3942
{{- end }}
4043
{{- if .ReachabilityNodePort }}
4144
egressip-node-healthcheck-port={{.ReachabilityNodePort}}
4245
{{- end }}
43-
{{- if not .OVN_MULTI_NETWORK_ENABLE }}
4446
enable-multi-network=true
45-
{{- end }}
4647
enable-network-segmentation=true
4748
enable-preconfigured-udn-addresses=true
4849
{{- if .OVN_MULTI_NETWORK_POLICY_ENABLE }}
4950
enable-multi-networkpolicy=true
5051
{{- end }}
52+
enable-admin-network-policy=true
53+
enable-multi-external-gateway=true
54+
enable-multicast=true
5155
{{- if .DNS_NAME_RESOLVER_ENABLE }}
5256
enable-dns-name-resolver=true
5357
{{- end }}

bindata/network/ovn-kubernetes/self-hosted/ovnkube-control-plane.yaml

Lines changed: 1 addition & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -129,19 +129,8 @@ spec:
129129
ovn_v6_masquerade_subnet_opt="--gateway-v6-masquerade-subnet {{.V6MasqueradeSubnet}}"
130130
fi
131131
132-
dns_name_resolver_enabled_flag=
133-
if [[ "{{.DNS_NAME_RESOLVER_ENABLE}}" == "true" ]]; then
134-
dns_name_resolver_enabled_flag="--enable-dns-name-resolver"
135-
fi
136-
137132
persistent_ips_enabled_flag="--enable-persistent-ips"
138133
139-
# This is needed so that converting clusters from GA to TP
140-
# will rollout control plane pods as well
141-
network_segmentation_enabled_flag=
142-
multi_network_enabled_flag="--enable-multi-network"
143-
network_segmentation_enabled_flag="--enable-network-segmentation"
144-
145134
route_advertisements_enable_flag=
146135
if [[ "{{.OVN_ROUTE_ADVERTISEMENTS_ENABLE}}" == "true" ]]; then
147136
route_advertisements_enable_flag="--enable-route-advertisements"
@@ -151,18 +140,7 @@ spec:
151140
if [[ "{{.OVN_EVPN_ENABLE}}" == "true" ]]; then
152141
evpn_enable_flag="--enable-evpn"
153142
fi
154-
155-
preconfigured_udn_addresses_enable_flag="--enable-preconfigured-udn-addresses"
156143
157-
# Enable multi-network policy if configured (control-plane always full mode)
158-
multi_network_policy_enabled_flag=
159-
if [[ "{{.OVN_MULTI_NETWORK_POLICY_ENABLE}}" == "true" ]]; then
160-
multi_network_policy_enabled_flag="--enable-multi-networkpolicy"
161-
fi
162-
163-
# Enable admin network policy if configured (control-plane always full mode)
164-
admin_network_policy_enabled_flag="--enable-admin-network-policy"
165-
166144
if [ "{{.OVN_GATEWAY_MODE}}" == "shared" ]; then
167145
gateway_mode_flags="--gateway-mode shared"
168146
elif [ "{{.OVN_GATEWAY_MODE}}" == "local" ]; then
@@ -187,22 +165,10 @@ spec:
187165
${ovn_v6_transit_switch_subnet_opt} \
188166
${ovn_v4_masquerade_subnet_opt} \
189167
${ovn_v6_masquerade_subnet_opt} \
190-
${dns_name_resolver_enabled_flag} \
191168
${persistent_ips_enabled_flag} \
192-
${multi_network_enabled_flag} \
193-
${network_segmentation_enabled_flag} \
194169
${gateway_mode_flags} \
195170
${route_advertisements_enable_flag} \
196-
${evpn_enable_flag} \
197-
${preconfigured_udn_addresses_enable_flag} \
198-
--enable-egress-ip=true \
199-
--enable-egress-firewall=true \
200-
--enable-egress-qos=true \
201-
--enable-egress-service=true \
202-
--enable-multicast \
203-
--enable-multi-external-gateway=true \
204-
${multi_network_policy_enabled_flag} \
205-
${admin_network_policy_enabled_flag}
171+
${evpn_enable_flag}
206172
volumeMounts:
207173
- mountPath: /run/ovnkube-config/
208174
name: ovnkube-config

docs/architecture.md

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -141,12 +141,6 @@ The Network operator needs to make sure that the input configuration doesn't cha
141141

142142
The persisted configuration must **make all defaults explicit**. This protects against inadvertent code changes that could destabilize an existing cluster.
143143

144-
### Per-Node Configuration
145-
146-
For certain specialized deployments (e.g., DPU host nodes), some features need to be disabled on a per-node basis even when enabled cluster-wide. Since ConfigMap values cannot be reliably overridden per-node, the CNO implements per-node feature enforcement through conditional logic in the startup scripts.
147-
148-
The `OVN_NODE_MODE` environment variable is injected into `ovnkube-node` pods and consumed by the startup script (`008-script-lib.yaml`) to conditionally enable or disable features based on the node's operational mode. This ensures that unsupported features are deterministically disabled on specialized hardware regardless of cluster-wide configuration.
149-
150144
## Egress Router
151145

152146
**Input:** `EgressRouter.network.operator.openshift.io`

docs/operands.md

Lines changed: 0 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -91,26 +91,6 @@ configuration object (which in turn is copied there from the
9191
configuration) is "`OVNKubernetes`". If the specified network type is
9292
not "`OVNKubernetes`", the CNO will not render any network plugin.
9393

94-
### OVN-Kubernetes Node Modes
95-
96-
OVN-Kubernetes supports different node operational modes through the `OVN_NODE_MODE`
97-
environment variable. This allows per-node feature enforcement, particularly for
98-
specialized hardware like DPU (Data Processing Unit) hosts where certain features
99-
must be disabled.
100-
101-
The startup script (`008-script-lib.yaml`) contains conditional logic that adjusts
102-
feature enablement based on the node mode:
103-
104-
- **`full` mode (default)**: All features enabled as configured
105-
- **`dpu-host` mode**: Certain features like egress IP, multicast, multi-network
106-
policies, and admin network policies are automatically disabled regardless of
107-
cluster-wide configuration
108-
109-
This approach was necessary because ConfigMap values (`004-config.yaml`) cannot be
110-
reliably overridden on a per-node basis, but startup script logic can be conditional.
111-
112-
For detailed information, see `docs/ovn_node_mode.md`.
113-
11494
## Multus
11595

11696
Multus is deployed as long as `.spec.disableMultiNetwork` is not set.

0 commit comments

Comments
 (0)