Skip to content

FileWatcher only looks for .gitignore in watched directory, not project root #591

@NicolasFerec

Description

@NicolasFerec

Taskiq version

0.12.1

Python version

Python 3.14

OS

Linux

What happened?

When using --reload-dir with a subdirectory in a monorepo, the FileWatcher triggers an infinite reload loop because it only looks for .gitignore in the watched directory itself, not in the project root.

Root cause: In taskiq/cli/watcher.py, the FileWatcher constructor does:

gpath = path / ".gitignore"
if use_gitignore and gpath.exists():
    self.gitignore = parse_gitignore(gpath)

This means when you run:

taskiq worker --reload-dir back/domains -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker

It looks for .gitignore at back/domains/.gitignore (which doesn't exist) instead of the project root .gitignore. Since no gitignore is loaded, Python's __pycache__ directories and .pyc files trigger reload events, creating an infinite loop.

Expected behavior: FileWatcher should walk up the directory tree to find .gitignore files, similar to how git itself works. Many tools (git, ripgrep, etc.) search parent directories for config files.

Current workaround: Use watchmedo from the watchdog package instead:

watchmedo auto-restart \
    --directory=back \
    --pattern=*.py \
    --recursive \
    --ignore-patterns='*/__pycache__/*;*.pyc;*.pyo' \
    -- taskiq worker -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker

This works because watchmedo respects the root .gitignore.

Steps to reproduce

  1. Create a monorepo structure with frontend and backend:
project/
├── .gitignore  (contains __pycache__/)
├── front/
└── back/
    └── domains/
        └── tasks.py
  1. Run taskiq with a subdirectory reload:
taskiq worker --reload-dir back/domains -r -tp 'back/domains/**/*tasks.py' -fsd back.broker:broker
  1. Observer infinite reload loop as Python creates __pycache__/ directories

Suggested fix

Modify FileWatcher.__init__ to search for .gitignore in parent directories:

def __init__(
    self,
    callback: Callable[..., None],
    path: Path,
    use_gitignore: bool = True,
    **callback_kwargs: Any,
) -> None:
    self.callback = callback
    self.gitignore = None
    
    if use_gitignore:
        # Walk up the tree to find .gitignore (like git does)
        current = path.resolve()
        while current != current.parent:
            gpath = current / ".gitignore"
            if gpath.exists():
                self.gitignore = parse_gitignore(gpath)
                break
            current = current.parent
    
    self.callback_kwargs = callback_kwargs

Related issues

Relevant log output

[Task] Sending task=...
[FileWatcher] Reloading due to change in back/domains/__pycache__/tasks.cpython-314.pyc
[Task] Sending task=...
[FileWatcher] Reloading due to change in back/domains/__pycache__/tasks.cpython-314.pyc
... (infinite loop)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions