Containerize the District Files pipeline to provide a consistent local development environment and simplify onboarding for future contributors.

[parcheesime]

Overview

Investigate and containerize the District Files pipeline to provide a consistent local development environment and simplify onboarding for future contributors.

The goal is to make it possible for developers to run the project with minimal local setup while preserving the existing workflow and functionality.

Action Items

• Evaluate the current project structure and runtime requirements.
• Create a Dockerfile for the application.
• Add a .dockerignore file.
• Configure environment variables and document required secrets.
• Verify the pipeline executes successfully inside the container.
• Update the README with Docker-based setup and usage instructions.
• Validate that the containerized workflow produces the same outputs as the current local workflow.

Resources / Instructions

Current repository:

• https://github.com/hackforla/district-files

Considerations:

• Preserve the existing GitHub Actions workflow.
• Do not change the behavior of the data collection pipeline.
• Focus on improving developer experience and reproducibility.

[parcheesime]

I'll start by evaluating the current project structure and identifying the minimum changes needed to support a Docker-based local development environment.

The goal is to improve onboarding and reproducibility while preserving the existing pipeline behavior and GitHub Actions workflow. If additional work (such as CI/CD or workflow changes) is needed, I'll open separate follow-up issues to keep this work scoped.
@chinaexpert1

[parcheesime]

Update:
@chinaexpert1

1. Progress: Verified the existing GitHub Actions pipeline runs successfully end-to-end. Added initial Docker configuration using Python 3.10, successfully built the image, and verified the container starts and runs main.py. Credentials are excluded from the image.
2. Blockers: None
3. Availability: I might be later to the meeting tonight, 7:30 or enter meeting via cell.
4. ETA: 1-2 weeks
5. Next: Test the containerized pipeline end-to-end using the Google Drive service account credential mounted at runtime, verify outputs and logs are uploaded successfully, and update the README with Docker setup instructions.
   link to run_log