Skip to content

SDN-5330: Keep IPsec in OVN enabled during the upgrade - #2621

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:masterfrom
pperiyasamy:ovn-ipsec-disable
Jan 24, 2025
Merged

SDN-5330: Keep IPsec in OVN enabled during the upgrade#2621
openshift-merge-bot[bot] merged 1 commit into
openshift:masterfrom
pperiyasamy:ovn-ipsec-disable

Conversation

@pperiyasamy

@pperiyasamy pperiyasamy commented Jan 20, 2025

Copy link
Copy Markdown
Member

The following change on the machine pool:

status.MachineCount == status.UpdatedMachineCount && hasSourceInMachineConfigStatus(status, machineConfigs)

is introduced with PR #2349 which ensures IPsec machine config is always installed on all the nodes in the cluster,
So this is deleting the IPsec daemonset as per the CNO state machine for IPsec when the condition is not met. But this is also accidentally disabling IPsec in OVN which is not an expected behavior. This causes ovs-monitor-ipsec to refresh existing ipsec connections unnecessarily when IPsec pod comes up as it is not able to find remote_name from the tunnel. This may also trigger deleting IPsec connection entries from openshift.conf file if ovs-monitor-ipsec is not killed timely when ipsec daemonset is removed. So this PR enables ovn ipsec option as long as the API is set with Full mode.

@openshift-ci
openshift-ci Bot requested review from dougbtv and tssurya January 20, 2025 10:19
@pperiyasamy

pperiyasamy commented Jan 20, 2025

Copy link
Copy Markdown
Member Author

/assign @jcaamano @igsilya @trozet

Thanks to @igsilya for catching this issue while troubleshooting IPsec problems with CI.

@pperiyasamy pperiyasamy changed the title Keep IPsec in OVN enabled during the upgrade SDN-5330: Keep IPsec in OVN enabled during the upgrade Jan 20, 2025
@openshift-ci-robot

openshift-ci-robot commented Jan 20, 2025

Copy link
Copy Markdown
Contributor

@pperiyasamy: This pull request references SDN-5330 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

Details

In response to this:

Due to known reasons as introduced in the PR #2349, The IPsec daemonset is removed at the time of OCP upgrade or MCP in progressing state. But it also accidently disabling IPsec in OVN. This causes ovs-monitor-ipsec to refresh existing ipsec connections unnecessarily (when pod comes up) as it is not able to find remote_name from tunnel. This may also trigger deleting IPsec connection entries from openshift.conf file if ovs-monitor-ipsec is not killed timely when ipsec daemonset is removed. So this commit enables ovn ipsec option as long as the API is set with Full mode.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jan 20, 2025
@pperiyasamy

Copy link
Copy Markdown
Member Author

/test e2e-aws-ovn-ipsec-upgrade
/test e2e-aws-ovn-ipsec-serial

@pperiyasamy

Copy link
Copy Markdown
Member Author

/label acknowledge-critical-fixes-only

@openshift-ci openshift-ci Bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Jan 22, 2025

@jcaamano jcaamano left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to known reasons as introduced in the PR #2349, The IPsec daemonset is removed at the time of OCP upgrade or MCP in progressing state.

Can you please elaborate?

Comment thread pkg/network/ovn_kubernetes.go Outdated
Comment on lines +671 to +672
// We render OVN IPsec as long as East-West IPsec is enabled.
renderIPsecOVN = mode == operv1.IPsecModeFull

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not

renderIPsecOVN = (renderIPsecHostDaemonSet || renderIPsecContainerizedDaemonSet || isOVNIPsecActive) && mode == operv1.IPsecModeFull

Why don't we need to wait anymore for the NS IPSec to be deployed before activating OVN IPSec?

Also, it's best if the comment explains why we are doing what we are doing, not just what we are doing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we need to wait anymore for the NS IPSec to be deployed before activating OVN IPSec?

activating OVN IPSec at this case won't trigger anything because ovs-monitor-ipsec script is not started yet. but still your suggestion on checking isOVNIPsecActive makes more sense as it enables IPsec OVN only when needed. changed it. added some info about explaining 'why', hope that's ok now.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jcaamano for your time and patience, it was a great long discussion to update IPsec state machine with up to date information. Updated the PR as we discussed.

@openshift-ci-robot

openshift-ci-robot commented Jan 22, 2025

Copy link
Copy Markdown
Contributor

@pperiyasamy: This pull request references SDN-5330 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

Details

In response to this:

The following change on the machine pool:

status.MachineCount == status.UpdatedMachineCount && hasSourceInMachineConfigStatus(status, machineConfigs)

is introduced with PR #2349 which ensures IPsec machine config is always installed on all the nodes in the cluster,
So this is deleting the IPsec daemonset as per the CNO state machine for IPsec when the condition is not met. But this is also accidentally disabling IPsec in OVN which is not an expected behavior. This causes ovs-monitor-ipsec to refresh existing ipsec connections unnecessarily when IPsec pod comes up as it is not able to find remote_name from the tunnel. This may also trigger deleting IPsec connection entries from openshift.conf file if ovs-monitor-ipsec is not killed timely when ipsec daemonset is removed. So this PR enables ovn ipsec option as long as the API is set with Full mode.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@pperiyasamy

Copy link
Copy Markdown
Member Author

/retest-required

   
The following change on the machine pool
`status.MachineCount == status.UpdatedMachineCount && hasSourceInMachineConfigStatus(status, machineConfigs)`
is introduced with PR openshift#2349
which ensures IPsec machine config is always installed on all the nodes in the cluster,
So this is deleting the IPsec daemonset as per the CNO state machine for IPsec when
the condition is not met. But this is also accidentally disabling IPsec in OVN which
is not an expected behavior. This causes ovs-monitor-ipsec to refresh existing ipsec
connections unnecessarily when IPsec pod comes up as it is not able to find remote_name
from the tunnel. This may also trigger deleting IPsec connection entries from openshift.conf
file if ovs-monitor-ipsec is not killed timely when ipsec daemonset is removed.
So this commit enables ovn ipsec option as long as the API is set with Full mode.

Signed-off-by: Periyasamy Palanisamy <pepalani@redhat.com>
@jcaamano

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jan 23, 2025
@openshift-ci

openshift-ci Bot commented Jan 23, 2025

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jcaamano, pperiyasamy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 23, 2025
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 0dfccc1 and 2 for PR HEAD 4e57dcd in total

1 similar comment
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 0dfccc1 and 2 for PR HEAD 4e57dcd in total

@openshift-ci

openshift-ci Bot commented Jan 24, 2025

Copy link
Copy Markdown
Contributor

@pperiyasamy: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-ipsec-upgrade e1059c0 link false /test e2e-aws-ovn-ipsec-upgrade
ci/prow/e2e-aws-ovn-ipsec-serial e1059c0 link false /test e2e-aws-ovn-ipsec-serial
ci/prow/security 4e57dcd link false /test security
ci/prow/e2e-openstack-ovn 4e57dcd link false /test e2e-openstack-ovn
ci/prow/e2e-vsphere-ovn-dualstack-primaryv6 4e57dcd link false /test e2e-vsphere-ovn-dualstack-primaryv6
ci/prow/e2e-aws-hypershift-ovn-kubevirt 4e57dcd link false /test e2e-aws-hypershift-ovn-kubevirt
ci/prow/okd-scos-e2e-aws-ovn 4e57dcd link false /test okd-scos-e2e-aws-ovn
ci/prow/e2e-ovn-step-registry 4e57dcd link false /test e2e-ovn-step-registry

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 51a6fc7 into openshift:master Jan 24, 2025
@openshift-bot

Copy link
Copy Markdown
Contributor

[ART PR BUILD NOTIFIER]

Distgit: cluster-network-operator
This PR has been included in build cluster-network-operator-container-v4.19.0-202501240608.p0.g51a6fc7.assembly.stream.el9.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants