Skip to content

feat(connectors): add Meilisearch sink connector#3497

Open
countradooku wants to merge 5 commits into
apache:masterfrom
countradooku:feat/meilisearch-sink-connector
Open

feat(connectors): add Meilisearch sink connector#3497
countradooku wants to merge 5 commits into
apache:masterfrom
countradooku:feat/meilisearch-sink-connector

Conversation

@countradooku

Copy link
Copy Markdown
Contributor

Summary

  • add the Meilisearch sink connector using the official meilisearch-sdk
  • document the sink configuration and payload mapping behavior
  • add Docker-backed integration coverage for indexing JSON messages into Meilisearch

Motivation

This splits the sink portion out of #3404 so the Meilisearch sink can be reviewed and merged independently from the source connector.

Closes #3495.

Validation

  • cargo sort --no-format --workspace
  • cargo test -p iggy_connector_meilisearch_sink
  • cargo clippy -p iggy_connector_meilisearch_sink --all-targets -- -D warnings
  • env CARGO_BIN_EXE_iggy-server=... CARGO_BIN_EXE_iggy-connectors=... cargo test -p integration --test mod -- connectors::meilisearch::meilisearch_sink
  • git diff --check

@countradooku countradooku marked this pull request as ready for review June 15, 2026 18:58
@github-actions github-actions Bot added the S-waiting-on-review PR is waiting on a reviewer label Jun 15, 2026
@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.01527% with 144 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.84%. Comparing base (9a0f799) to head (0ff5885).

Files with missing lines Patch % Lines
core/connectors/sinks/meilisearch_sink/src/lib.rs 78.01% 128 Missing and 16 partials ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             master    #3497       +/-   ##
=============================================
- Coverage     74.70%   45.84%   -28.86%     
  Complexity      937      937               
=============================================
  Files          1257     1255        -2     
  Lines        124910   109622    -15288     
  Branches     100583    85341    -15242     
=============================================
- Hits          93312    50256    -43056     
- Misses        28595    56661    +28066     
+ Partials       3003     2705      -298     
Components Coverage Δ
Rust Core 38.40% <78.01%> (-37.31%) ⬇️
Java SDK 58.57% <ø> (ø)
C# SDK 71.40% <ø> (-0.65%) ⬇️
Python SDK 88.88% <ø> (ø)
PHP SDK 84.29% <ø> (ø)
Node SDK 91.22% <ø> (ø)
Go SDK 40.36% <ø> (ø)
Files with missing lines Coverage Δ
core/connectors/sinks/meilisearch_sink/src/lib.rs 78.01% <78.01%> (ø)

... and 374 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread Cargo.toml
Comment thread Cargo.toml
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs Outdated
@ryerraguntla

ryerraguntla commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

overall correct approach - JSON-object document model with generated PK is sound, wait_for_tasks + partial-error accounting is thoughtful. test coverage is present but there is
coverage gap — only happy-path (2 JSON messages indexed, iggy_id present). No test for: document_action = update semantics, include_metadata = false, raw-payload base64 fallback, or wait_for_tasks = false fire-and-forget path. Not a merge-blocker given this is the initial connector, but tracking it. Along with coverage, please take care of the above mentioned issues.

@ryerraguntla

Copy link
Copy Markdown
Contributor

/author

@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jun 16, 2026
@countradooku

Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jun 16, 2026
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs Outdated
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
Comment thread core/connectors/sinks/meilisearch_sink/src/lib.rs
@ryerraguntla

Copy link
Copy Markdown
Contributor

/author

@github-actions github-actions Bot added S-waiting-on-author PR is waiting on author response and removed S-waiting-on-review PR is waiting on a reviewer labels Jun 16, 2026
@countradooku

Copy link
Copy Markdown
Contributor Author

/ready

@github-actions github-actions Bot added S-waiting-on-review PR is waiting on a reviewer and removed S-waiting-on-author PR is waiting on author response labels Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-review PR is waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Meilisearch sink connector

3 participants