Skip to content

CORENET-6097: Add OVN BGP NoOverlay workflow for baremetal clusters - #73335

Closed
zhaozhanqi wants to merge 3 commits into
openshift:mainfrom
zhaozhanqi:add-ovn-bgp-no-overlay-workflow
Closed

CORENET-6097: Add OVN BGP NoOverlay workflow for baremetal clusters#73335
zhaozhanqi wants to merge 3 commits into
openshift:mainfrom
zhaozhanqi:add-ovn-bgp-no-overlay-workflow

Conversation

@zhaozhanqi

@zhaozhanqi zhaozhanqi commented Jan 12, 2026

Copy link
Copy Markdown
Contributor

Add OVN BGP NoOverlay workflow for baremetal clusters

This PR adds two workflows

  • 'baremetalds-e2e-ovn-bgp-dualstack-local-gw-no-overlay'
  • 'baremetalds-e2e-ovn-bgp-dualstack-no-overlay'

and two lanes for no-overlay with routing mode: Unmanaged
also with outboundSNAT is Disabled and Enables

  • e2e-metal-ipi-ovn-dualstack-no-overlay-managed-lgw
  • e2e-metal-ipi-ovn-dualstack-no-overlay-unmanaged-sgw

deploy clsuter steps:

  1. Sets up FRR route reflector container on the packet server before cluster installation
  2. Creates network operator manifest with NoOverlay transport configuration
  3. Configures FRRConfiguration to peer with external route reflector
  4. Adds RouteAdvertisements manifest for PodNetwork
  5. Set nftables rules on the packet server for outboundSNAT is Disabled to make bootstrap node can reply to pod
  6. Reorders pre-steps to run baremetalds-e2e-ovn-bgp-pre before baremetalds-ofcir-pre

The baremetalds-e2e-ovn-bgp-nftables-pre ref step to configure
nftables rules required for NoOverlay with outboundSNAT is Disabled and BGP routing on the
bare metal host. The step uses a systemd timer to ensure firewall rules
persist and self-heal throughout cluster lifecycle.

Key changes:

  • Configure ct state invalid handling for overlay-free networking
  • Fix FORWARD chain rules for dualstack (IPv4/IPv6) traffic
  • Adjust MASQUERADE rules for proper SNAT behavior
  • Run fix script every 2 minutes via systemd timer for resilience

This step is placed after no-overlay manifest creation and before
cluster provisioning to ensure proper network configuration

Signed-off-by: zhaozhanqi zzhao@redhat.com

@openshift-ci
openshift-ci Bot requested review from arkadeepsen and tssurya January 12, 2026 01:50
@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jan 12, 2026
@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch 3 times, most recently from bf16951 to 93ee080 Compare January 12, 2026 04:08
@openshift-ci-robot openshift-ci-robot removed the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jan 12, 2026
@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch 2 times, most recently from 1a93f8f to 9ea69da Compare January 13, 2026 07:41
@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/payload-job-with-prs pull-ci-openshift-cluster-network-operator-master-e2e-metal-ipi-ovn-dualstack-bgp-lgw-no-overlay-techpreview openshift/api#2537,openshift/cluster-network-operator#2844,openshift/ovn-kubernetes#2878,openshift/client-go#349

@openshift-ci

openshift-ci Bot commented Jan 13, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: it appears that you have attempted to use some version of the payload command, but your comment was incorrectly formatted and cannot be acted upon. See the docs for usage info.

@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch from 9ea69da to 8c04bc5 Compare January 13, 2026 08:26
@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/payload-job-with-prs periodic-ci-openshift-cluster-network-operator-master-e2e-metal-ipi-ovn-dualstack-bgp-lgw-no-overlay-test openshift/cluster-network-operator#2844 openshift/ovn-kubernetes#2878 openshift/client-go#349

@openshift-ci

openshift-ci Bot commented Jan 13, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: the repo openshift/release does not contribute to the OpenShift official images, or the base branch is not currently having images promoted

@openshift-ci

openshift-ci Bot commented Jan 13, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: This PR was included in a payload test run from openshift/cluster-network-operator#2844
trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

@openshift-ci

openshift-ci Bot commented Jan 13, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: This PR was included in a payload test run from openshift/ovn-kubernetes#2920
trigger 0 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch from 8c04bc5 to e9ff88f Compare January 14, 2026 09:18
@zhaozhanqi
zhaozhanqi marked this pull request as draft January 28, 2026 09:35
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 28, 2026
@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch from e9ff88f to 2e14735 Compare January 30, 2026 09:20
@openshift-ci-robot openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jan 30, 2026
@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch from 2e14735 to 230a4a8 Compare February 5, 2026 03:32
@zhaozhanqi

zhaozhanqi commented Feb 5, 2026

Copy link
Copy Markdown
Contributor Author

/retitle CORENET-6097: Add OVN BGP NoOverlay workflow for baremetal clusters

@openshift-ci openshift-ci Bot changed the title Add OVN BGP NoOverlay workflow for baremetal clusters CORENET-6097: Add OVN BGP NoOverlay workflow for baremetal clusters Feb 5, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 5, 2026
@openshift-ci-robot

openshift-ci-robot commented Feb 5, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: This pull request references CORENET-6097 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

This PR adds a new workflow 'baremetalds-e2e-ovn-bgp-dualstack-local-gw-no-overlay'
with the following steps:

  1. Sets up FRR route reflector container on the packet server before cluster installation
  2. Creates network operator manifest with NoOverlay transport configuration
  3. Configures FRRConfiguration to peer with external route reflector
  4. Adds RouteAdvertisements manifest for PodNetwork
  5. Reorders pre-steps to run baremetalds-e2e-ovn-bgp-pre before baremetalds-ofcir-pre

New components:

  • baremetalds-e2e-ovn-bgp-conf-no-overlay ref step
  • baremetalds-e2e-ovn-bgp-dualstack-local-gw-no-overlay workflow

Which for:
Network operator configuration:

  • defaultNetworkTransport: NoOverlay
  • routing: Unmanaged/Managed
  • outboundSNAT: Enabled/Disabled
  • additionalRoutingCapabilities: FRR
  • routeAdvertisements: Enabled

and also Introduce baremetalds-e2e-ovn-bgp-nftables-pre ref step to configure
nftables rules required for NoOverlay with outboundSNAT is Disabled and BGP routing on the
bare metal host. The step uses a systemd timer to ensure firewall rules
persist and self-heal throughout cluster lifecycle.

Key changes:

  • Configure ct state invalid handling for overlay-free networking
  • Fix FORWARD chain rules for dualstack (IPv4/IPv6) traffic
  • Adjust MASQUERADE rules for proper SNAT behavior
  • Run fix script every 2 minutes via systemd timer for resilience

This step is placed after no-overlay manifest creation and before
cluster provisioning to ensure proper network configuration

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@zhaozhanqi
zhaozhanqi force-pushed the add-ovn-bgp-no-overlay-workflow branch from 230a4a8 to 666c7d0 Compare February 6, 2026 11:30
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-snat-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-snat-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-snat-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-snat-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-snat-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@jluhrsen

jluhrsen commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

@CodeRabbit review

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

@CodeRabbit review

Sound like this repo don't enable codeRabbit.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you want to remove this line?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, this should be mistake update by 'make update', now it's clean

@MahnoorAsghar

Copy link
Copy Markdown
Contributor

/pj-rehearse pull-ci-openshift-ovn-kubernetes-master-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@MahnoorAsghar: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

This PR adds two workflows:
- 'baremetalds-e2e-ovn-bgp-dualstack-local-gw-no-overlay'
- 'baremetalds-e2e-ovn-bgp-dualstack-no-overlay'

and two test lanes for no-overlay with routing mode: Unmanaged
with outboundSNAT Disabled and Enabled:
- e2e-metal-ovn-dualstack-no-overlay-managed-lgw-techpreview
- e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview

Deploy cluster steps:

1. Sets up FRR route reflector container on the packet server before cluster installation
2. Creates network operator manifest with NoOverlay transport configuration
3. Configures FRRConfiguration to peer with external route reflector
4. Adds RouteAdvertisements manifest for PodNetwork
5. Configures nftables rules on the packet server when outboundSNAT is Disabled

The baremetalds-e2e-ovn-bgp-nftables-pre ref step configures
nftables rules required for NoOverlay with outboundSNAT Disabled and BGP routing
on the bare metal host. The step uses a one-shot systemd service that waits for
master VMs to appear before applying firewall rules.

Key changes:
- Stop and disable firewalld to prevent rule conflicts
- Enable IP forwarding for NoOverlay routing
- Remove ct state invalid drop rule from NETAVARK_FORWARD for overlay-free networking
- Add FORWARD chain rules for dualstack (IPv4/IPv6) cluster network traffic
- Add MASQUERADE rules for proper SNAT behavior on egress traffic
- Add NAT RETURN rule to skip masquerading for cluster-to-pod traffic from bootstrap host

This step is placed after no-overlay manifest creation and before
cluster provisioning to ensure proper network configuration.

Signed-off-by: zhaozhanqi <zzhao@redhat.com>
…s these resource already created in day0 for no-overlay
@openshift-ci

openshift-ci Bot commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: zhaozhanqi
Once this PR has been reviewed and has the lgtm label, please assign andfasano, knobunc for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@zhaozhanqi

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@zhaozhanqi: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@zhaozhanqi: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-managed-lgw-techpreview openshift/cluster-network-operator presubmit Presubmit changed
pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview openshift/cluster-network-operator presubmit Presubmit changed
pull-ci-openshift-cluster-network-operator-release-4.22-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview openshift/cluster-network-operator presubmit Presubmit changed
pull-ci-openshift-cluster-network-operator-master-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-5.0-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.23-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.22-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.21-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.20-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.19-e2e-metal-ipi-ovn-dualstack-bgp openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-master-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-master-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-5.0-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-5.0-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.23-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.23-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.22-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.22-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.21-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.20-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-cluster-network-operator-release-4.19-e2e-metal-ipi-ovn-dualstack-bgp-local-gw openshift/cluster-network-operator presubmit Registry content changed
pull-ci-openshift-ovn-kubernetes-master-e2e-metal-ovn-dualstack-no-overlay-managed-lgw-techpreview openshift/ovn-kubernetes presubmit Presubmit changed
pull-ci-openshift-ovn-kubernetes-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview openshift/ovn-kubernetes presubmit Presubmit changed
pull-ci-openshift-ovn-kubernetes-release-4.22-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview openshift/ovn-kubernetes presubmit Presubmit changed
pull-ci-openshift-ovn-kubernetes-release-4.22-e2e-metal-ovn-fdp-qe-ipv6 openshift/ovn-kubernetes presubmit Presubmit changed

A total of 126 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci

openshift-ci Bot commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

@zhaozhanqi: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/openshift/origin/release-4.22/e2e-metal-ipi-ovn-dualstack-bgp 0aa75ac link unknown /pj-rehearse pull-ci-openshift-origin-release-4.22-e2e-metal-ipi-ovn-dualstack-bgp
ci/rehearse/openshift/cluster-network-operator/master/e2e-metal-ovn-dualstack-no-overlay-sgw-snat-techpreview a78cda6 link unknown /pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-sgw-snat-techpreview
ci/rehearse/openshift/cluster-network-operator/release-4.22/e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview 9a50122 link unknown /pj-rehearse pull-ci-openshift-cluster-network-operator-release-4.22-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview
ci/rehearse/openshift/ovn-kubernetes/master/e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview 04aa0ca link unknown /pj-rehearse pull-ci-openshift-ovn-kubernetes-master-e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview
ci/prow/step-registry-metadata 001b32c link true /test step-registry-metadata
ci/rehearse/openshift/cluster-network-operator/master/e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview 001b32c link unknown /pj-rehearse pull-ci-openshift-cluster-network-operator-master-e2e-metal-ovn-dualstack-no-overlay-unmanaged-sgw-techpreview

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-bot

Copy link
Copy Markdown
Contributor

Closing, please open a separate request if this work is still required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants