Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[run]
branch = True
source =
shiftings
plugins =
django_coverage_plugin

[report]
show_missing = True
skip_covered = False
omit =
*/migrations/*
*/test_*.py
*/tests/*
*/manage.py

[html]
directory = coverage_html

[django_coverage_plugin]
template_extensions = html,htm,txt
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,5 @@
gunicorn.py
/build/
/src/shiftings/templates/local/
ruff.toml
ruff.toml
.coverage
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ See [Django docs](https://docs.djangoproject.com/en/4.1/howto/deployment/) for d
1. Add a local_settings.py. See local_settings.sample.py for inspiration.
1. Add a cron, systemd timer unit or similar for recurring shift creation

#### Testing

Testing quality beyond line coverage is documented in the [Testing Quality Scorecard](docs/testing_quality.md).

## Authors

[lewellien](https://github.com/lewellien) & [Tjeri](https://github.com/tjeri)
Expand Down
2 changes: 2 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Development

For information on development, refer to the project [README](https://github.com/HaDiNet/shiftings/).

Testing quality beyond line coverage is documented in the [Testing Quality Scorecard](testing_quality.md).
119 changes: 119 additions & 0 deletions docs/testing_quality.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Testing Quality Scorecard

This scorecard complements line coverage with behavior-focused indicators.

## Why

Line coverage answers: "Did code run?"

This scorecard answers: "Would tests catch bad behavior?"

## Current Baseline (2026-04-17)

- Full-suite line coverage: 41%
- Critical auth behavior coverage:
- accounts/views/auth.py: 98%
- accounts/views/user.py: 97%
- Core shared view mixin coverage:
- utils/views/base.py: 98%

## Behavior Quality Metrics

Use these together. No single metric is enough.

### 1) Risk-Weighted Coverage

Track coverage only for high-risk modules first (auth, permission, shift lifecycle).

Suggested risk tiers:

- Tier 1 (critical): authentication, authorization, user identity, participation permissions
- Tier 2 (important): shift creation/update/delete and recurring generation
- Tier 3 (supporting): helpers and UI integration

### 2) Scenario Coverage

Maintain a behavior matrix and mark each scenario as:

- Covered: at least one deterministic automated test
- Partial: path exists but edge/error branches missing
- Missing: no automated coverage

#### Scenario Matrix

| Domain | Scenario | Status | Test Reference |
|---|---|---|---|
| Auth | Local login rendering and fallback flows | Covered | `accounts/tests/test_auth_views.py` |
| Auth | SSO login initiation (enabled/disabled) | Covered | `accounts/tests/test_auth_views.py` |
| Auth | OAuth callback success/failure/exception | Covered | `accounts/tests/test_auth_views.py` |
| Auth | Confirm-email valid/invalid/malformed token paths | Covered | `accounts/tests/test_user_views.py` |
| Auth | Logout and relogin cache/session edge behavior | Partial | `accounts/tests/test_auth_views.py` |
| Auth | Password-reset confirmation invalid/expired token behavior | Missing | `n/a` |
| Auth | Brute-force and repeated failed-login throttling behavior | Missing | `n/a` |
| Permissions | Missing-permission behavior (redirect, 403, fail URL) | Covered | `utils/tests/test_base_views.py` |
| Permissions | Shift participation permission update workflow | Covered | `shifts/tests/test_permission_views.py` |
| Permissions | Duplicate organization rejection in permission formset | Covered | `shifts/tests/test_permission_forms.py` |
| Permissions | Cross-organization isolation for permission edits | Missing | `n/a` |
| Permissions | Permission precedence conflicts across global/org scopes | Partial | `shifts/tests/test_permission_forms.py` |
| Shifts | Participant add/remove for self and others | Covered | `shifts/tests/test_participant_views.py` |
| Shifts | Shift CRUD edge cases (date bounds, authz, redirects) | Partial | `shifts/tests/*` |
| Shifts | Summary view filtering and boundary scenarios | Missing | `n/a` |
| Shifts | Concurrent participant slot contention and max-user enforcement | Missing | `n/a` |
| Shifts | Shift copy/template creation with invalid source references | Missing | `n/a` |
| Recurring shifts | Form validation for recurring setup | Partial | `shifts/tests/*` |
| Recurring shifts | Recurring generation command and date edge cases | Missing | `n/a` |
| Recurring shifts | Timezone and DST boundary generation correctness | Missing | `n/a` |
| Recurring shifts | Idempotency of repeated recurring generation runs | Missing | `n/a` |
| Mail | Attachment size and date-range validation | Covered | `mail/tests/test_mail_forms.py` |
| Mail | Mail view send flows and permission boundaries | Partial | `mail/tests/*` |
| Mail | Delivery failure/retry behavior and user-visible error states | Missing | `n/a` |
| Mail | Recipient resolution with mixed member/group filters | Partial | `mail/tests/*` |
| Utils | Protected media header dispatch by backend type | Covered | `utils/tests/test_protected_content.py` |
| Utils | Saved-path breadcrumb behavior with complex query params | Partial | `utils/tests/test_base_views.py` |
| Utils | Http403 middleware integration and custom error page wiring | Missing | `n/a` |

### 3) Mutation Score (Recommended)

Measure if tests fail when implementation is intentionally perturbed.

Target:

- Tier 1 modules: > 70%
- Tier 2 modules: > 55%

Note:

- Start with small scope to keep runtime manageable.
- Prioritize accounts/views/auth.py, utils/views/base.py, shifts/views/shift.py.

### 4) Regression Protection Index

For each bug fix, add a test that fails before the fix.

Track monthly:

- Bugs fixed with regression test / total bugs fixed

Target:

- 100% for Tier 1 bugs

## Scorecard Template

Update this table in each release cycle.

| Metric | Current | Target | Trend |
|---|---:|---:|---|
| Full-suite line coverage | 41% | 50% | up |
| Tier 1 weighted coverage | n/a | 75% | n/a |
| Scenario coverage (Tier 1) | 8/13 | 10/13 | up |
| Mutation score (Tier 1) | n/a | 70% | n/a |
| Regression protection index (Tier 1) | n/a | 100% | n/a |

## Update Procedure

1. Run full tests with coverage.
2. Update Tier 1/Tier 2 module coverage values.
3. Review scenario matrix and mark new covered paths.
4. Record newly fixed bugs and whether regression tests exist.
5. Update scorecard table and share in release notes.
4 changes: 3 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,6 @@ nav:
- Shift Summary: 'organizations/shift_summary.md'
- Events:
- Coming Soon: 'events/index.md'
- Development: 'development.md'
- Development:
- Overview: 'development.md'
- Testing Quality Scorecard: 'testing_quality.md'
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,14 @@ dev = [
'types-python-dateutil==2.9.0.20260124',
'types-requests==2.32.4.20260107',
'mypy==1.19.1',
'django-extensions==4.1',
]

tests = [
'pytest==9.0.2',
'pytest-django==4.12.0',
'pytest-cov==7.0.0',
'django-coverage-plugin==3.2.2',
]

docs = [
Expand Down
Loading