Skip to content

Commit f89dcbd

Browse files
author
Lukas Puehringer
committed
Add custom hatch plugin to attest for builds
Create and configure a "custom" hatch BuildHookInterface to generate in-toto link metadata for builds, signed with a key configured via envvar. The commit also replaces the `in-toto-run ... -- python -m build ...` cd and maintainer build instructions. hatch docs: https://ofek.dev/hatch/latest/plugins/build-hook/#custom Signed-off-by: Lukas Puehringer <lukas.puehringer@nyu.edu>
1 parent 017031b commit f89dcbd

5 files changed

Lines changed: 137 additions & 67 deletions

File tree

.github/workflows/cd.yml

Lines changed: 6 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -35,51 +35,13 @@ jobs:
3535

3636
- name: Build binary wheel and source tarball
3737
env:
38-
IN_TOTO_KEY: ${{ secrets.IN_TOTO_KEY }}
39-
IN_TOTO_KEY_PW: ${{ secrets.IN_TOTO_KEY_PW }}
38+
HATCH_BUILD_HOOK_ENABLE_CUSTOM: true
39+
HATCH_IN_TOTO_KEY: ${{ secrets.IN_TOTO_KEY }}
40+
HATCH_IN_TOTO_KEY_PW: ${{ secrets.IN_TOTO_KEY_PW }}
4041
run: |
41-
#######################################################
42-
# Build and generate signed attestions with in-toto CLI
43-
44-
# Make signing key available to in-toto commands
45-
echo -n "$IN_TOTO_KEY" > .in_toto/key
46-
47-
# Define patterns for files that need not be recorded as materials below
48-
exclude=('__pycache__' 'build' 'htmlcov' '.?*' '*~' '*.egg-info' '*.pyc')
49-
50-
# Grab TUF version to construct build artifact names for product recording
51-
version=$(python3 -c 'import tuf; print(tuf.__version__)')
52-
53-
# Build sdist and record all files in CWD as materials and the build artifact
54-
# as product in a signed attestation 'sdist.<signing key id>.link'.
55-
in-toto-run \
56-
--step-name sdist \
57-
--key .in_toto/key \
58-
--key-type ed25519 \
59-
--password "$IN_TOTO_KEY_PW" \
60-
--materials . \
61-
--products dist/tuf-${version}.tar.gz \
62-
--exclude ${exclude[@]} \
63-
--metadata-directory .in_toto \
64-
--verbose \
65-
-- python3 -m build --sdist --outdir dist/ .
66-
67-
# Build wheel and record all files in CWD as materials and the build artifact
68-
# as product in a signed attestation 'wheel.<signing key id>.link'.
69-
in-toto-run \
70-
--step-name wheel \
71-
--key .in_toto/key \
72-
--key-type ed25519 \
73-
--password "$IN_TOTO_KEY_PW" \
74-
--materials . \
75-
--products dist/tuf-${version}-py3-none-any.whl \
76-
--exclude ${exclude[@]} dist/tuf-${version}.tar.gz \
77-
--metadata-directory .in_toto \
78-
--verbose \
79-
-- python3 -m build --wheel --outdir dist/ .
80-
81-
# Remove signing key file
82-
rm .in_toto/key
42+
################################################################
43+
# Build and generate signed attestions with in-toto hatch plugin
44+
python3 -m build --sdist --wheel --outdir dist/ .
8345
8446
- id: gh-release
8547
name: Publish GitHub release candiate

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ tests/htmlcov/
2727
# Ignore in-toto metadata
2828
.in_toto/*
2929
!.in_toto/create_layout.py
30+
!.in_toto/hatch_with_in_toto.py
3031

3132
# Debian generated files
3233
debian/.debhelper/

.in_toto/hatch_with_in_toto.py

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
#!/usr/bin/env python
2+
3+
# Copyright 2022, TUF contributors
4+
# SPDX-License-Identifier: MIT OR Apache-2.0
5+
6+
"""Hatch plugin to generate in-toto link metadata for a build.
7+
8+
Hooks into the hatch build process to record the used sources as materials and the built
9+
artifact(s) as products, signing the resulting link metadata with a configured
10+
functionary key.
11+
12+
13+
export HATCH_BUILD_HOOK_ENABLE_CUSTOM=true
14+
export HATCH_IN_TOTO_GPG_KEYID=<gpg key id>
15+
**OR** export HATCH_IN_TOTO_KEY=<encrypted ed25519 private key data>
16+
export HATCH_IN_TOTO_KEY_PW=<decryption password>
17+
python -m build
18+
19+
"""
20+
import os
21+
22+
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
23+
from in_toto.models.link import FILENAME_FORMAT, Link
24+
from in_toto.models.metadata import Metablock
25+
from in_toto.runlib import record_artifacts_as_dict
26+
from securesystemslib.keys import decrypt_key
27+
28+
IN_TOTO_DIR = ".in_toto"
29+
30+
# TODO: Is there a better way to configure this? KEY makes sense as envvar, but
31+
# non-confidential config would be nice as arg. Can we use `build`s `--config-setting`?
32+
class EnvVars:
33+
KEY = "HATCH_IN_TOTO_KEY"
34+
KEY_PW = "HATCH_IN_TOTO_KEY_PW"
35+
GPG_KEYID = "HATCH_IN_TOTO_GPG_KEYID"
36+
37+
38+
# Define exclude filters for recording materials in the project directory
39+
# NOTE: No filters needed for products, because they are recorded explicitly
40+
# TODO: Consider configuration via pyproject.toml
41+
# see https://ofek.dev/hatch/latest/config/build/#build-hooks
42+
MATERIAL_EXCLUDES = [
43+
"__pycache__",
44+
"build",
45+
"dist",
46+
"htmlcov",
47+
".*",
48+
"*~",
49+
"*.egg-info",
50+
"*.pyc",
51+
]
52+
53+
54+
class InTotoBuildHook(BuildHookInterface):
55+
def __init__(self, *args, **kwargs):
56+
super().__init__(*args, **kwargs)
57+
58+
# Common config for artifact recording
59+
self.lstrip = [f"{self.root}/"]
60+
61+
# Parse config from envvars
62+
key_data = os.environ.get(EnvVars.KEY)
63+
key_pw = os.environ.get(EnvVars.KEY_PW)
64+
self.gpg_keyid = os.environ.get(EnvVars.GPG_KEYID)
65+
66+
if key_data and self.gpg_keyid:
67+
raise ValueError(
68+
f"Set only one of {EnvVars.KEY} or {EnvVars.GPG_KEYID}"
69+
)
70+
71+
if not (key_data or self.gpg_keyid):
72+
raise ValueError(
73+
f"Requires one of {EnvVars.KEY} or {EnvVars.GPG_KEYID}"
74+
)
75+
76+
self.key = None
77+
if key_data:
78+
if not key_pw:
79+
raise ValueError(
80+
f"Requires {EnvVars.KEY_PW} if {EnvVars.KEY} is set"
81+
)
82+
83+
self.key = decrypt_key(key_data, key_pw)
84+
85+
def initialize(self, version, build_data):
86+
self.materials = record_artifacts_as_dict(
87+
[self.root],
88+
exclude_patterns=MATERIAL_EXCLUDES,
89+
lstrip_paths=self.lstrip,
90+
)
91+
92+
def finalize(self, version, build_data, artifact_path):
93+
products = record_artifacts_as_dict(
94+
[artifact_path], lstrip_paths=self.lstrip
95+
)
96+
97+
link = Metablock(
98+
signed=Link(
99+
name=self.target_name,
100+
materials=self.materials,
101+
products=products,
102+
)
103+
)
104+
if self.key:
105+
sig = link.sign(self.key)
106+
elif self.gpg_keyid:
107+
sig = link.sign_gpg(self.gpg_keyid)
108+
else:
109+
raise RuntimeError("No signing key")
110+
111+
link_filename = FILENAME_FORMAT.format(
112+
step_name=self.target_name, keyid=sig["keyid"]
113+
)
114+
115+
link.dump(os.path.join(IN_TOTO_DIR, link_filename))

docs/RELEASE_with_in-toto.md

Lines changed: 8 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -63,32 +63,17 @@ in-toto-run \
6363

6464
## Build
6565

66-
Call `python3 -m build --sdist ...` and `python3 -m build --wheel ...` with `in-toto` as
67-
shown to create two signed attestations, recording the names and hashes of files in cwd
68-
as *materials*, and the name and hash of each respective build artifact as product. The
69-
attestations are written to `.in_toto/sdist.<signing keyid>.link` and
70-
`.in_toto/wheel.<signing keyid>.link`.
66+
Call `python3 -m build -wheel --sdist ...` with in-toto hatch plugin enabled to create
67+
signed attestations, recording the names and hashes of files in cwd as *materials*, and
68+
the name and hash of each respective build artifact as product. The attestations are
69+
written to `.in_toto/sdist.<signing keyid>.link` and `.in_toto/wheel.<signing
70+
keyid>.link`.
7171

7272
```bash
73-
in-toto-run \
74-
--step-name sdist \
75-
--gpg ${signing_key} \
76-
--materials . \
77-
--products dist/tuf-${version}.tar.gz \
78-
--exclude ${exclude[@]} \
79-
--metadata-directory .in_toto \
80-
-- python3 -m build --sdist --outdir dist/ .
81-
```
73+
export HATCH_BUILD_HOOK_ENABLE_CUSTOM=true
74+
export HATCH_IN_TOTO_GPG_KEYID=${signing_key}
8275

83-
```bash
84-
in-toto-run \
85-
--step-name wheel \
86-
--gpg ${signing_key} \
87-
--materials . \
88-
--products dist/tuf-${version}-py3-none-any.whl \
89-
--exclude ${exclude[@]} dist/tuf-${version}.tar.gz \
90-
--metadata-directory .in_toto \
91-
-- python3 -m build --wheel --outdir dist/ .
76+
python3 -m build --sdist --wheel --outdir dist/ .
9277
```
9378

9479
## Verify

pyproject.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,13 @@ Source = "https://github.com/theupdateframework/python-tuf"
5656
[tool.hatch.version]
5757
path = "tuf/__init__.py"
5858

59+
[tool.hatch.build.hooks.custom]
60+
enable-by-default = false
61+
path = ".in_toto/hatch_with_in_toto.py"
62+
dependencies = [
63+
"in-toto[pynacl]"
64+
]
65+
5966
[tool.hatch.build.targets.sdist]
6067
include = [
6168
"/docs",

0 commit comments

Comments
 (0)