Stream module lockfiles directly to disk - #30428
Closed
dzbarsky wants to merge 5 commits into
Closed
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
A real hidden lockfile reached 456,397,262 bytes (435.254 MiB). Serializing it as Gson.toJson(updatedLockfile) + "\n" builds two full strings, adding about 870 MiB of transient heap and causing java.lang.OutOfMemoryError: Java heap space in updateLockfile. Stream Gson through a buffered UTF-8 writer instead, while preserving the trailing newline and the existing I/O failure behavior. Add a regression test for the exact persisted JSON and newline. (cherry picked from commit 715f49f)
dzbarsky
force-pushed
the
codex/stream-module-lockfiles-dzbarsky
branch
from
July 22, 2026 22:41
72ee047 to
1cc1957
Compare
dzbarsky
marked this pull request as ready for review
July 22, 2026 22:45
github-actions
Bot
added
team-ExternalDeps
Collaborator
|
@bazel-io fork 9.3.0 |
fmeum
reviewed
Jul 23, 2026
fmeum
reviewed
Jul 23, 2026
fmeum
reviewed
Jul 23, 2026
Contributor
Author
|
@fmeum I addressed all four review comments, including preserving an existing -zbarskybot |
fmeum
approved these changes
Jul 23, 2026
github-actions
Bot
added
the
community-reviewed
Wyverald
approved these changes
Jul 27, 2026
Wyverald
added
awaiting-PR-merge
github-actions
Bot
removed
the
awaiting-PR-merge
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Stream
MODULE.bazel.lockserialization through a buffered UTF-8 writer instead of building the complete JSON as aString. The same helper writes both workspace and hidden lockfiles. Preserve the existing trailing newline and I/O failure logging, and add a regression test for the exact persisted JSON and newline.Motivation
A real hidden lockfile reached 456,397,262 bytes (435.254 MiB).
Gson.toJson(updatedLockfile) + "\n"creates two full strings, about 870 MiB of transient heap, and can fail inBazelLockFileModule.updateLockfilewithjava.lang.OutOfMemoryError: Java heap space.Streaming Gson directly to disk removes those full-size intermediate strings.
Build API Changes
No
Checklist
Release Notes
RELNOTES: None