Skip to content

Manual testing: multimodal image pipeline (Slack inbound, outbound, modality gate, TUI) #63

Description

@Aaronontheweb

Context

The multimodal image pipeline has been implemented and has automated test coverage for serialization round-trips, ChatMessageConverter media handling, modality gating, and the attach_file tool. However, several integration paths require manual testing against real infrastructure before we can consider this feature complete.

Branch: claude-wt-multi-modal-models

Manual Test Plan

1. Slack Inbound — File Download + Content Scanning

  • Send a message with an image attachment to a Slack channel where Netclaw is active
  • Verify the image is downloaded (check session media directory under /tmp/netclaw-sessions/)
  • Verify IContentScanner.ScanAsync() is invoked on the downloaded bytes
  • Verify the image reaches the LLM as DataContent in the chat history
  • Send a non-image file (e.g., PDF, ZIP) — verify it is skipped with a debug log

2. Slack Outbound — File Upload

  • Trigger a FileOutput (e.g., via the attach_file tool or an LLM that produces file output)
  • Verify the file is uploaded to the correct Slack thread via files.uploadV2
  • Verify upload failures are handled gracefully (logged, don't crash the thread actor)

3. Modality Gate — Real Model Behavior

  • Configure a text-only model (e.g., qwen3:30b) and send a message with an image attachment
    • Verify the user sees: [Images removed — the current model does not support vision input]
    • Verify the LLM still receives and responds to the text portion
  • Send an image-only message (no text) to a text-only model
    • Verify the user sees the explanation message and no LLM call is made
  • Configure a vision-capable model and send a message with an image
    • Verify the image passes through without stripping and the model responds to the image content

4. TUI Rendering

  • Trigger a FileOutput and verify ChatPage renders the file path in cyan
  • Verify HeadlessChannel prints the file path to console/log

5. SignalR Wire Format

  • Verify SessionOutputMapper.ToDto correctly maps FileOutput to type "file" with FilePath, FileName, MimeType
  • Verify DaemonClient.FromDto correctly reconstructs FileOutput from the DTO

Automated Coverage (already passing)

For reference, these areas are covered by the 20 new automated tests:

  • Protobuf round-trip: SerializableChatMessage and SendUserMessage with MediaReferences (3 tests)
  • ChatMessageConverter: DataContent write/read, full round-trip, missing file handling, empty content, MIME mapping (7 tests)
  • Modality gate: text-only strips images, image-only skips LLM, vision model passes through (3 tests)
  • AttachFileTool: valid path, path traversal, dotdot traversal, missing file, empty path, no session dir, display name (7 tests)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions