Skip to content

Commit 2ed2caf

Browse files
authored
Documentation/792 improve dependency update documentation (#833)
* Add schedule to view and augment text a bit * Restructure documentation to make clearer the general types of workflows we support * Modify PR description
1 parent 240a8c5 commit 2ed2caf

3 files changed

Lines changed: 109 additions & 78 deletions

File tree

doc/changes/unreleased.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,16 +19,15 @@ details, check out the [Workflow Extensions](https://exasol.github.io/python-too
1919

2020
## Features
2121

22-
* #756: Added `dependency-update.yml` to automate resolving vulnerabilities with a generated pull request
23-
24-
## Bugfix
25-
26-
* #563: Fixed merge-gate to prevent auto-merges from happening when integration tests failed
27-
28-
## Feature
2922

3023
* #829: Extended removing a job from a workflow to also remove it from the `needs` of another job
3124
* #825: Created two workflows by splitting up previous ones:
3225
* Moved the periodic jobs in `ci.yml` to its own `periodic-validation.yml`
3326
* Moved the unit tests job in `checks.yml` to its own `fast-tests.yml`
3427
* #730: Added workflow extensions to `fast-tests` and `merge-gate`
28+
* #756: Added `dependency-update.yml` to automate resolving vulnerabilities with a generated pull request
29+
* #792: Improved `dependency-update.yml` documentation
30+
31+
## Bugfix
32+
33+
* #563: Fixed merge-gate to prevent auto-merges from happening when integration tests failed

doc/user_guide/features/github_workflows/index.rst

Lines changed: 96 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,9 @@ workflows from the templates.
2525
2626
.. attention::
2727

28-
In most cases, we recommend using _all_ workflows without change to ensure
29-
consistent interdependencies. For more details, see :ref:`ci_actions`.
28+
In most cases, we recommend using **all** workflows without change to ensure
29+
consistent interdependencies. For more details, see :ref:`manual_activities`
30+
and :ref:`automated_activities`.
3031

3132

3233
Workflows
@@ -136,79 +137,27 @@ and is maintained by the PTB and what is project-specific.
136137
tests. It's encouraged to add to this workflow extension additional approval
137138
requests, similar to ``approve-run-slow-tests``.
138139

139-
.. _ci_actions:
140140

141-
CI Actions
142-
----------
143141

144-
.. _dependency_update:
145-
146-
Dependency Update
147-
^^^^^^^^^^^^^^^^^
148-
149-
The ``dependency-update.yml`` workflow is used to resolve vulnerabilities by updating our project dependencies.
150-
151-
It can be triggered manually and is also scheduled to run weekly.
152-
153-
The workflow first audits dependencies for known vulnerabilities. If vulnerabilities
154-
are detected, it updates the dependencies using Poetry. When the ``poetry.lock`` is changed,
155-
then it creates a pull request with the update.
156-
157-
.. _pr_merge_yml:
142+
.. _manual_activities:
158143

159-
Merge
160-
^^^^^
144+
Manual Activities
145+
-----------------
161146

162-
When a pull request is merged to main, then the ``pr-merge.yml`` workflow is activated.
147+
The following workflows are typically triggered directly by a developer during normal
148+
project maintenance.
163149

164-
.. mermaid::
165-
:name: merge-diagram
166-
167-
graph TD
168-
%% Workflow Triggers (Solid Lines)
169-
pr-merge[pr-merge.yml] --> publish-docs[gh-pages.yml]
170-
171-
.. _periodic_validation_yml:
150+
.. _ci_yml:
172151

173-
Periodic Validation
174-
^^^^^^^^^^^^^^^^^^^
152+
Create a PR
153+
^^^^^^^^^^^
175154

176-
Once a week, this `periodic-validation.yml` is triggered on the default branch. Its main
177-
purpose is to ensure that our critical checks and tests continue to run, but it also
178-
sends the results of the linting tools and test coverage to Sonar for an overall report.
155+
Modifying any pull request triggers the workflow ``ci.yml``.
179156

180-
.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/periodic-validation.yml
157+
.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/ci.yml
181158
:language: yaml
182-
:start-at: schedule:
183-
:end-at: - cron: "0 0 * * 6"
184-
185-
.. mermaid::
186-
187-
graph TD
188-
%% Define Nodes
189-
checks[checks.yml]
190-
periodic_validation[periodic-validation.yml]
191-
fast-tests[fast-tests.yml]
192-
slow_checks[slow-checks.yml]
193-
report[report.yml]
194-
195-
%% Workflow Triggers
196-
periodic_validation --> checks
197-
periodic_validation --> fast-tests
198-
periodic_validation --> slow_checks
199-
200-
%% Dependencies
201-
checks -.->|needs| report
202-
fast-tests -.->|needs| report
203-
slow_checks -.->|needs| report
204-
205-
.. _ci_yml:
206-
207-
Pull Request
208-
^^^^^^^^^^^^
209-
210-
When any pull request is opened, synchronized, or reopened, then the ``ci.yml`` will be
211-
triggered.
159+
:start-at: on:
160+
:end-at: types: [opened, synchronize, reopened]
212161

213162
When configured as described on :ref:`github_project_configuration`, the
214163
``run-slow-tests`` job requires a developer to manually approve executing the slower
@@ -264,10 +213,24 @@ then the subsequent jobs will not be started.
264213
style approver fill:#fff,stroke:#333,stroke-dasharray: 5 5
265214
style slow_run fill:#fff,stroke:#333,stroke-dasharray: 5 5
266215

216+
.. _pr_merge_yml:
217+
218+
Merge a PR
219+
^^^^^^^^^^
220+
221+
Merging a pull request to the default branch, activates the ``pr-merge.yml`` workflow.
222+
223+
.. mermaid::
224+
:name: merge-diagram
225+
226+
graph TD
227+
%% Workflow Triggers (Solid Lines)
228+
pr-merge[pr-merge.yml] --> publish-docs[gh-pages.yml]
229+
267230
.. _cd_yml:
268231

269-
Release
270-
^^^^^^^
232+
Create a Release
233+
^^^^^^^^^^^^^^^^
271234

272235
When the nox session ``release:trigger`` is used, a new tag is created & pushed
273236
to main. This starts the release process by activating the ``cd.yml`` workflow.
@@ -282,3 +245,68 @@ to main. This starts the release process by activating the ``cd.yml`` workflow.
282245
%% Dependencies / Waiting (Dotted Lines)
283246
check-release-tag -.->|needs| build-and-publish[build-and-publish.yml]
284247
build-and-publish -.->|needs| gh-pages[gh-pages.yml]
248+
249+
.. _automated_activities:
250+
251+
Automated Activities
252+
--------------------
253+
254+
The following workflows are triggered automatically by schedules.
255+
256+
.. _dependency_update:
257+
258+
Update Dependencies
259+
^^^^^^^^^^^^^^^^^^^
260+
261+
The ``dependency-update.yml`` workflow runs on the default branch. It
262+
checks for known vulnerabilities and tries to fix them by updating dependencies.
263+
264+
.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/dependency-update.yml
265+
:language: yaml
266+
:start-at: on:
267+
:end-at: workflow_dispatch:
268+
269+
The workflow first audits dependencies for known vulnerabilities:
270+
271+
* If no vulnerabilities are detected, then no update is needed.
272+
* If vulnerabilities are detected, it updates the dependencies using ``poetry update``.
273+
274+
* If the ``poetry.lock`` is unchanged, then no further action is taken.
275+
* If the ``poetry.lock`` is changed, then it creates a branch, stages the commit,
276+
creates a pull request, and sends a Slack notification.
277+
278+
Afterwards, users need to perform some manual steps which are described in the PR description.
279+
280+
.. _periodic_validation_yml:
281+
282+
Validate Periodically
283+
^^^^^^^^^^^^^^^^^^^^^
284+
285+
The ``periodic-validation.yml`` runs on the default branch. It ensures that critical
286+
checks and tests continue to run and sends the results of the linting tools and test
287+
coverage to Sonar for an overall report.
288+
289+
.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/periodic-validation.yml
290+
:language: yaml
291+
:start-at: schedule:
292+
:end-at: - cron: "0 0 * * 6"
293+
294+
.. mermaid::
295+
296+
graph TD
297+
%% Define Nodes
298+
checks[checks.yml]
299+
periodic_validation[periodic-validation.yml]
300+
fast-tests[fast-tests.yml]
301+
slow_checks[slow-checks.yml]
302+
report[report.yml]
303+
304+
%% Workflow Triggers
305+
periodic_validation --> checks
306+
periodic_validation --> fast-tests
307+
periodic_validation --> slow_checks
308+
309+
%% Dependencies
310+
checks -.->|needs| report
311+
fast-tests -.->|needs| report
312+
slow_checks -.->|needs| report

exasol/toolbox/templates/github/workflows/dependency-update.yml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,9 +91,13 @@ jobs:
9191
BASE_BRANCH=$(gh repo view --json defaultBranchRef -q .defaultBranchRef.name)
9292
9393
PR_BODY="Automated dependency update for \`poetry.lock\`.
94-
This PR was created by the dependency update workflow after running:
95-
- \`poetry run -- nox -s dependency:audit\`
96-
- \`poetry update\`"
94+
This PR was created by the workflow \`dependency-update.yml\`
95+
96+
Please perform the following actions on a locally checked out branch:
97+
- [ ] Execute \`poetry run -- nox -s workflow:generate -- --all\`
98+
- [ ] Use \`poetry run -- nox -s dependency:audit\` to check for vulnerabilities requiring manual action
99+
- [ ] Update file \`doc/changes/unreleased.md\`
100+
"
97101
98102
PR_URL=$(gh pr create \
99103
--base "$BASE_BRANCH" \

0 commit comments

Comments
 (0)