Skip to content

feat: optimize the indexes for session/continuity cleanup, SPLAT-930 - #128

Open
abador wants to merge 2 commits into
fandom_masterfrom
SPLAT-930-add-index-for-session-cleanup
Open

feat: optimize the indexes for session/continuity cleanup, SPLAT-930#128
abador wants to merge 2 commits into
fandom_masterfrom
SPLAT-930-add-index-for-session-cleanup

Conversation

@abador

@abador abador commented Aug 7, 2026

Copy link
Copy Markdown

Related issue(s)

Checklist

  • I have read the contributing guidelines.
  • I have referenced an issue containing the design document if my change
    introduces a new feature.
  • I am following the
    contributing code guidelines.
  • I have read the security policy.
  • I confirm that this pull request does not address a security
    vulnerability. If this pull request addresses a security vulnerability, I
    confirm that I got the approval (please contact
    security@ory.sh) from the maintainers to push
    the changes.
  • I have added tests that prove my fix is effective or that my feature
    works.
  • I have added or changed the documentation.

Further Comments

Comment on lines +1 to +2
CREATE INDEX continuity_containers_expires_at_idx
ON continuity_containers (expires_at);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Do we need this index?

Comment on lines +1 to +2
CREATE INDEX sessions_expires_at_idx
ON sessions (expires_at);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Do we need this index?

@mmeller-wikia mmeller-wikia left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: I am not sure if these changes will actually speed up cleanup process, as we are removing up to 5000 rows from each table each 5min.
Could we purge all stale rows instead in one run?
If yes, we might run it once per hour and reclaim some k8s resources

func (p *Persister) CleanupDatabase(ctx context.Context, wait time.Duration, older time.Duration, batchSize int) error {
currentTime := time.Now().Add(-older)
p.r.Logger().Printf("Cleaning up records older than %s\n", currentTime)
p.r.Logger().Println("Cleaning up expired sessions")
if err := p.DeleteExpiredSessions(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired continuity containers")
if err := p.DeleteExpiredContinuitySessions(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired login flows")
if err := p.DeleteExpiredLoginFlows(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired recovery flows")
if err := p.DeleteExpiredRecoveryFlows(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired registation flows")
if err := p.DeleteExpiredRegistrationFlows(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired settings flows")
if err := p.DeleteExpiredSettingsFlows(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired verification flows")
if err := p.DeleteExpiredVerificationFlows(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning up expired session token exchangers")
if err := p.DeleteExpiredExchangers(ctx, currentTime, batchSize); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Cleaning seen selfservice errors")
if err := p.ClearErrorContainers(ctx, older, false); err != nil {
return err
}
time.Sleep(wait)
p.r.Logger().Println("Successfully cleaned up the latest batch of the SQL database! " +
"This should be re-run periodically, to be sure that all expired data is purged.")
return nil
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants