chore(comps): annotate backport overlays with category metadata by liunan-ms · Pull Request #17727 · microsoft/azurelinux

liunan-ms · 2026-06-16T00:03:56Z

This PR annotates the overlays surfaced by the overlay classifier with structured [metadata] (a category plus, for genuine backports, the verified upstream commits), so future Fedora rebases can mechanically identify and retire overlays that have landed upstream. It also pins azldev to a version that understands these annotations and the per-file overlay-files layout, and regenerates the authoritative JSON schema to match.

The change is organized as two commits:

chore(azldev): pin to 35d8fae and regenerate schema — bumps .azldev-version and regenerates external/schemas/azldev.schema.json.
chore(comps): annotate and migrate backport overlays to per-file layout — the annotation + per-file migration.

azldev version change (`0256227` → `35d8fae`)

Changelog

Features

Overlay metadata & per-file overlay format (#249) — Annotate overlays with structured provenance and split them into per-logical-change files:
- Inline [metadata] block on component overlays (category, commits, upstreamable, pr/bugs) so backports can be mechanically identified and retired once they land upstream.
- New per-file overlay layout: declare overlay-files = ["overlays/*.overlay.toml"] on a component and drop one *.overlay.toml file per logical change, each with a single file-level [metadata] block.
Inherited overlay-files resolved after config merge (#256) — overlay-files can now be declared once in [default-component-config] (project/distro/component-group) and is inherited by every applicable component, with relative globs resolved per concrete component (or matched spec directory). A non-empty component list overrides the inherited one; an empty list (overlay-files = []) disables inherited patterns. No-match globs are a no-op rather than an error.
azldev repo query (#213) — New command to inspect and manage the RPM repositories defined under [resources].
Deterministic archive extract/repack utility (#223) — Reproducible tarball extraction and repacking.

Fixes & behavior changes

Surface close errors in shell completions (#233) — Completion generation no longer silently drops I/O close errors, and an unused writable handle was removed.

Excluded as non-user-facing: dev/test tooling (#244 gremlins mutation testing), CI config (#232 CodeQL), docs refresh (#222), and dependency bumps (#237, #240, #248, go-toml v2.4.0→v2.4.2).

Project config

base/comps/components.toml gains a project-wide [default-component-config] with overlay-files = ["overlays/*.overlay.toml"], so every component discovers its own overlays/*.overlay.toml without redeclaring the glob.

Per-file overlay migration (24 components)

ant, apache-ivy, atlas, cloud-init, cpio, glade, gnulib, grub2, gt, intel-qpl, javapackages-tools, kdump-utils, maven, maven4, rust-podman-sequoia, sos, sssd, vamp-plugin-sdk, xbean, xclock, xmvn, xmvn5, xsimd, zbar — inline overlays were moved into per-logical-change overlays/*.overlay.toml files, each carrying one file-level [metadata] block. Multi-commit backports (e.g. javapackages-tools openjdk21 drop) consolidate related upstream commits into a single file via commits = [url1, url2, …]. Patch source paths are rewritten to ../<patch> to point back at the component root.

Category spread

Most overlays are backport-dist-git (with verified upstream commits). A few non-backport overlays surfaced alongside them are labelled accordingly: azl-pruning (e.g. grub2 xen-module disable), azl-security-compliance (grub2 os-prober reverts — restoring upstream GRUB's secure default Fedora diverged from), azl-compatibility, and azl-branding-policy.

Validation

Pure annotation/restructuring: the rendered spec is byte-identical before and after (verified via before/after prepare-sources diffs for every migrated component).
azldev comp update reports no lock drift (metadata is excluded from fingerprints; the overlay-files restructuring keeps the same input set).

dmcilvaney · 2026-06-16T19:59:38Z

-replacement = ""
+# Overlays live as one-logical-change-per-file under ./overlays/.
+# See https://github.com/microsoft/azure-linux-dev-tools/blob/main/docs/user/reference/config/overlays.md#per-file-overlay-format-overlaytoml
+overlay-dir = "overlays"


It would be great to update the AI instructions to explain when to use this pattern vs. not, and what sort of data to add to each overlay.

Yes, I'll add a SKILL.md and instruction in a separate PR for using this pattern and suggesting annotations for new overlays.

Copilot

Pull request overview

This PR annotates the Azure Linux overlays previously flagged by the overlay classifier with structured metadata (a category plus upstream commits/pr/bugs), so future Fedora rebases can mechanically identify and retire backported overlays once they land upstream. It also bumps the pinned azldev version to one that understands these annotations and a new per-file overlay layout.

Changes:

Inline annotations (17 components): add a [components.<name>.overlays.metadata] block to existing single-/few-overlay .comp.toml files. Note these attach per-overlay (to the immediately preceding overlay), which is used deliberately in some files (e.g. sos, sssd) but is inconsistent in xbean.
Per-file migration (7 components): cpio, gnulib, javapackages-tools, qemu, zbar, apache-ivy, cloud-init move inline overlays into overlays/*.overlay.toml files (top-level [metadata] + [[overlays]], with source paths rewritten to ../ to point back at the component root). qemu also consolidates per-section subpackage removals into spec-remove-subpackage (verified output-equivalent against the rendered spec).
.azldev-version bump to enable the new overlay-files/metadata features; the committed authoritative schema was not regenerated to match.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`.azldev-version`	Bumps pinned azldev to a version supporting `overlay-files`/`metadata`; committed schema not regenerated (flagged).
`base/comps/{ant,atlas,glade,gt,intel-qpl,maven,maven4,xclock,xmvn,xmvn5}/*.comp.toml`	Single-overlay inline `metadata` annotations (correct attachment).
`base/comps/{sos,sssd,kdump-utils,rust-podman-sequoia,vamp-plugin-sdk}/*.comp.toml`	Multi-overlay inline `metadata` with distinct per-overlay commits (correct).
`base/comps/xbean/xbean.comp.toml`	Inline `metadata`; remove/add openjdk retarget pair annotated inconsistently (flagged).
`base/comps/cpio/*`	Migrated to per-file overlay; `rmt` BR drop annotated.
`base/comps/gnulib/*`	Migrated; temporary Source-URL backport annotated.
`base/comps/javapackages-tools/*`	Migrated; openjdk21-drop multi-commit backport consolidated.
`base/comps/qemu/*`	Migrated to 4 overlay files; subpackage removals consolidated to `spec-remove-subpackage` (output-equivalent).
`base/comps/zbar/*`	Migrated; GIR-drop (backport) and video-disable (pruning) split into two files.
`base/comps/apache-ivy/*`	Migrated; `0001` overlays lack required `description` fields (flagged nit).
`base/comps/cloud-init/*`	Migrated to 3 overlay files (cpe-fix, azl4 patch series, branding); `../` source paths point at component-root patches.

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 60 out of 60 changed files in this pull request and generated 4 comments.

- apache-ivy: add per-overlay descriptions to the pack200 backport file (hygiene rule #1 — every overlay must explain why the change is needed). - gnulib: drop the redundant per-component overlay-files declaration; the project-wide [default-component-config] already discovers overlays/*.overlay.toml. - grub2: reconcile the os-prober revert overlays (0002 + 0003) to a single category. Re-disabling os-prober restores upstream GRUB's secure default that Fedora diverged from (it will not retire on a Fedora rebase), so both halves are azl-security-compliance and the misleading backport commits are dropped. - external/schemas/azldev.schema.json: regenerate against the pinned azldev (4b5f72c) so the authoritative schema knows the new metadata / overlay-files / BugRef fields used throughout this PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 60 out of 60 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 61 out of 61 changed files in this pull request and generated 1 comment.

Pin azldev to 35d8fae562980db7bda057d79322cc51ddad8d62, the merged form of the overlay-files-after-resolution change (microsoft/azure-linux-dev-tools#256). This version supports inheriting overlay-files from a project-wide [default-component-config] and allows an empty overlay-files list to disable inherited patterns. Regenerate external/schemas/azldev.schema.json against the pinned tool so the authoritative schema knows the new metadata / overlay-files / BugRef fields. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Annotate every overlay surfaced by the overlay classifier with structured [metadata] (a category plus, for genuine backports, the verified upstream commits) so future Fedora rebases can mechanically identify and retire overlays that have landed upstream. - Add a project-wide [default-component-config] in base/comps/components.toml with overlay-files = ["overlays/*.overlay.toml"], so each component discovers its own per-file overlays without redeclaring the glob. - Migrate all 24 annotated components to the per-file overlay-files layout: inline overlays move into per-logical-change overlays/*.overlay.toml files, each carrying one file-level [metadata] block. Multi-commit backports consolidate related upstream commits via commits = [url1, url2, ...]. Patch source paths are rewritten to ../<patch> to point back at the component root. - Categories: most overlays are backport-dist-git (with verified commits); a few non-backport overlays are labelled accordingly (azl-pruning, azl-security-compliance, azl-compatibility, azl-branding-policy). Pure annotation/restructuring: the rendered spec is byte-identical before and after (verified via prepare-sources), and azldev comp update reports no lock drift (metadata is excluded from fingerprints). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

liunan-ms mentioned this pull request Jun 16, 2026

feat(component): add overlay metadata and 'component metadata' command microsoft/azure-linux-dev-tools#242

Closed

dmcilvaney reviewed Jun 16, 2026

View reviewed changes

liunan-ms changed the title ~~chore(comps): annotate backport-fedora overlays with category metadata~~ chore(comps): annotate backport overlays with category metadata Jun 18, 2026

liunan-ms force-pushed the liunan/overlay_metadata branch from 93c3e9b to e29cdb5 Compare June 18, 2026 19:11

Copilot AI review requested due to automatic review settings June 25, 2026 20:37

Copilot started reviewing on behalf of liunan-ms June 25, 2026 20:38 View session

liunan-ms force-pushed the liunan/overlay_metadata branch from 23ed86c to 4107a39 Compare June 25, 2026 20:44

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread .azldev-version Outdated

Comment thread base/comps/xbean/xbean.comp.toml Outdated

Comment thread base/comps/apache-ivy/overlays/0001-pack200-commons-compress.overlay.toml

Copilot AI review requested due to automatic review settings June 25, 2026 20:49

Copilot started reviewing on behalf of liunan-ms June 25, 2026 20:50 View session

liunan-ms force-pushed the liunan/overlay_metadata branch from 4107a39 to 13381f7 Compare June 25, 2026 20:58

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread base/comps/xbean/xbean.comp.toml Outdated

Comment thread .azldev-version Outdated

Copilot AI review requested due to automatic review settings June 25, 2026 21:05

Copilot started reviewing on behalf of liunan-ms June 25, 2026 21:05 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread base/comps/xbean/xbean.comp.toml Outdated

Comment thread base/comps/apache-ivy/overlays/0001-pack200-commons-compress.overlay.toml

Copilot AI review requested due to automatic review settings June 29, 2026 21:44

Copilot started reviewing on behalf of liunan-ms June 29, 2026 21:44 View session

Copilot AI reviewed Jun 29, 2026

View reviewed changes

Comment thread base/comps/gnulib/gnulib.comp.toml Outdated

Comment thread base/comps/apache-ivy/overlays/0001-pack200-commons-compress.overlay.toml

Comment thread base/comps/grub2/overlays/0003-append-patch0383-patch0384-patch0385.overlay.toml Outdated

Comment thread base/comps/components.toml

Copilot AI review requested due to automatic review settings June 30, 2026 17:05

Copilot started reviewing on behalf of liunan-ms June 30, 2026 17:05 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings June 30, 2026 20:50

Copilot started reviewing on behalf of liunan-ms June 30, 2026 20:51 View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

Comment thread base/comps/gt/overlays/0001-add-std-gnu89-cflags.overlay.toml

liunan-ms and others added 2 commits June 30, 2026 20:58

liunan-ms force-pushed the liunan/overlay_metadata branch from f55c1ce to 7db736c Compare June 30, 2026 21:04

microsoft deleted a comment from github-actions Bot Jun 30, 2026

liunan-ms marked this pull request as ready for review June 30, 2026 21:27

liunan-ms requested a review from a team as a code owner June 30, 2026 21:27

Copilot AI review requested due to automatic review settings June 30, 2026 21:27

Copilot started reviewing on behalf of liunan-ms June 30, 2026 21:28 View session

Copilot AI reviewed Jun 30, 2026

Uh oh!

Conversation

liunan-ms commented Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

azldev version change (0256227 → 35d8fae)

Changelog

Features

Fixes & behavior changes

Project config

Per-file overlay migration (24 components)

Category spread

Validation

Uh oh!

dmcilvaney Jun 16, 2026

Choose a reason for hiding this comment

Uh oh!

liunan-ms Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

liunan-ms commented Jun 16, 2026 •

edited

Loading

azldev version change (`0256227` → `35d8fae`)

liunan-ms Jun 16, 2026 •

edited

Loading