postgresqlPackages.vectorchord: init at 0.4.2 by diogotcorreia · Pull Request #392710 · NixOS/nixpkgs

diogotcorreia · 2025-03-24T13:10:00Z

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

https://github.com/tensorchord/VectorChord

Probably going to be needed for Immich soon.

Things done

Add a 👍 reaction to pull requests you find important.

Titaniumtown

diff LGTM besides the comments made by others. I think we should hold off merging this until an immich release actually uses this. Thanks for the proactive PR @diogotcorreia !

diogotcorreia · 2025-04-11T10:40:55Z

VectorChord 0.3.0 has been released, but there's a major blocker to updating. Many functions related to avx have been marked as safe in rust nightly, but are still unsafe on rust stable (even 1.86): rust-lang/stdarch#1714
Example:

My guess is that this would land on 1.87, which will be released in the 15th of May, and likely take a few weeks to hit master.

For now I think I'll wait until Immich moves to vectorchord, and then if Rust 1.87 is still not available on master at that time I'll make a patch to add unsafe blocks around these calls to unsafe functions.

If anyone has a better idea, I'd appreciate some guidance.

Related upstream commit: supervc-stack/VectorChord@dc12d84

Ma27

Sorry, I had this tab open for too long without paying attention. What's the status here?

diogotcorreia · 2025-05-09T10:57:24Z

@Ma27 I think we're waiting for Immich to migrate before merging. However, considering what I wrote in my previous comment (#392710 (comment)), upgrading this to 0.3.0 would not be easy and would require extensive patching afaik (EDIT: it seems like the aforementioned changes will be in Rust 1.87, so if Immich doesn't upgrade in the next month or two it should be fine).

There have also been talks about handling dependencies between extensions (#392710 (comment)), but I've unfortunately not had much time to look into this

EDIT: I also realized I haven't updated this PR after #394089 has been merged, so I'll do that

diogotcorreia · 2025-05-09T13:47:19Z

Update: Immich is already working on this, so it might be released soon: immich-app/immich#18042
However, they mention that they will keep supporting pgvecto.rs for a while, so that's good news

diogotcorreia · 2025-05-09T14:40:13Z

Rebased on master, refactored to use finalAttrs and added asserts to tests

mertalev · 2025-05-19T22:12:53Z

Immich dev here! Just a heads up that we plan to switch to VectorChord in a release this week, with 0.3.0 as the minimum supported version. We'll remove pgvecto.rs support later in the coming months, so it's recommended to switch when possible. That being said, pgvecto.rs will still work for now without any manual intervention, so it's okay if you can't make the switch right away.

diogotcorreia · 2025-05-20T09:59:07Z

@mertalev Thanks a lot for the heads up! Rustc in nixpkgs has been updated to 1.87.0 in #407444, so we should be able to update this PR to 0.3.0 in the coming weeks, once the rustc update hits master.

We'll remove pgvecto.rs support later in the coming months

Do you have any estimate for how long you're going to support pgvecto.rs? Afaik rustc won't be updated in NixOS 25.05, so we won't be able to get vectorchord (without extensive patching like I mentioned in previous comments) in the NixOS stable channel until the end of November (for NixOS 25.11).

mertalev · 2025-05-20T15:32:21Z

The current plan is to remove pgvecto.rs support around the time of the stable release, which I'd say is unlikely to happen before August.

Would supporting VectorChord 0.2.2 make things easier for you?

dotlambda · 2025-05-20T15:36:33Z

Afaik rustc won't be updated in NixOS 25.05

Doing so wouldn't be a problem.

EDIT: I don't mean updating but adding a rust_1_78 attribute.

diogotcorreia

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 392710
Commit: 564ed3e9f89f768af9b1f6dcb956a1a4b099f1a5

`x86_64-linux`

✅ 7 packages built:

cargo-pgrx
cargo-pgrx_0_12_6
postgresql13Packages.vectorchord
postgresql14Packages.vectorchord
postgresql15Packages.vectorchord
postgresql16Packages.vectorchord
postgresql17Packages.vectorchord

wolfgangwalther · 2025-06-25T06:55:44Z

Please rename the first commit to postgresqlPackages.{pgvectorscale,pgx_ulid}: .... This would allow ofborg to actually build it (not a rebuild, but anyway).

wolfgangwalther · 2025-06-25T07:06:33Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 392710
Commit: 564ed3e9f89f768af9b1f6dcb956a1a4b099f1a5

`x86_64-linux`

✅ 6 packages built:

cargo-pgrx (cargo-pgrx_0_14_1)
postgresql13Packages.vectorchord
postgresql14Packages.vectorchord
postgresql15Packages.vectorchord
postgresql16Packages.vectorchord
postgresqlPackages.vectorchord (postgresql17Packages.vectorchord)

`aarch64-linux`

✅ 6 packages built:

cargo-pgrx (cargo-pgrx_0_14_1)
postgresql13Packages.vectorchord
postgresql14Packages.vectorchord
postgresql15Packages.vectorchord
postgresql16Packages.vectorchord
postgresqlPackages.vectorchord (postgresql17Packages.vectorchord)

`x86_64-darwin`

✅ 1 package built:

cargo-pgrx (cargo-pgrx_0_14_1)

`aarch64-darwin`

✅ 1 package built:

cargo-pgrx (cargo-pgrx_0_14_1)

tecosaur · 2025-07-01T11:35:58Z

Am I correct in thinking that this is the sole factor blocking an Immich update to 1.135?

diogotcorreia · 2025-07-01T12:30:10Z

Am I correct in thinking that this is the sole factor blocking an Immich update to 1.135?

@tecosaur Hasn't Immich been updated a while ago already? #417977 #418220

Immich still supports pgvecto-rs, so this is not blocking anything at the moment (except PostgreSQL 17 support, since pgvecto-rs does not support that). They will deprecate support for it soon tho, so we should migrate sooner than later

happysalada · 2025-07-02T12:02:20Z

@wolfgangwalther are you okay with merging this ?

You suggested an improvement which i think is good. Just thinking we can potentially keep it as a future improvement to integrate this change.

Does anyone else want to delay merging to look at it more ?

wolfgangwalther

No blockers from my side, except for meta.platforms, I think.

diogotcorreia · 2025-07-02T12:18:20Z

@happysalada I'm not sure if the allocation issues have been fully discussed: #392710 (comment)

cc @dotlambda

wolfgangwalther · 2025-07-05T12:23:56Z

I'm not sure if the allocation issues have been fully discussed: #392710 (comment)

Since the linked issue wasn't entirely clear about finding the root cause of this, given that the allocator is changed in the next version anyway and that all passthru tests pass on both linux platforms (see below), I'd say we're good here.

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 392710 --package postgresql13Packages.vectorchord.tests --package postgresql14Packages.vectorchord.tests --package postgresql17Packages.vectorchord.tests --package postgresql16Packages.vectorchord.tests --package postgresql15Packages.vectorchord.tests
Commit: ab82f5a51158d05d8d713bfc6f4cbec0456f85eb

`x86_64-linux`

✅ 5 packages built:

postgresql13Packages.vectorchord.tests.extension
postgresql14Packages.vectorchord.tests.extension
postgresql15Packages.vectorchord.tests.extension
postgresql16Packages.vectorchord.tests.extension
postgresql17Packages.vectorchord.tests.extension

`aarch64-linux`

✅ 5 packages built:

postgresql13Packages.vectorchord.tests.extension
postgresql14Packages.vectorchord.tests.extension
postgresql15Packages.vectorchord.tests.extension
postgresql16Packages.vectorchord.tests.extension
postgresql17Packages.vectorchord.tests.extension

`x86_64-darwin`

⏩ 5 packages marked as broken and skipped:

postgresql13Packages.vectorchord.tests.extension
postgresql14Packages.vectorchord.tests.extension
postgresql15Packages.vectorchord.tests.extension
postgresql16Packages.vectorchord.tests.extension
postgresql17Packages.vectorchord.tests.extension

`aarch64-darwin`

⏩ 5 packages marked as broken and skipped:

postgresql13Packages.vectorchord.tests.extension
postgresql14Packages.vectorchord.tests.extension
postgresql15Packages.vectorchord.tests.extension
postgresql16Packages.vectorchord.tests.extension
postgresql17Packages.vectorchord.tests.extension

dotlambda · 2025-07-22T03:36:26Z

Am I overlooking something or is there no PR yet moving the Immich module to vectorchord?

diogotcorreia · 2025-07-22T09:09:26Z

@dotlambda correct, I belive no one has done that PR yet

th3voic3 · 2025-07-22T09:18:21Z

For reference since I needed this to migrate from the docker version I have the following in my module:
services.postgresql = { extensions = ps: with ps; [pgvector vectorchord]; settings.shared_preload_libraries = ["vchord"]; };

Scrumplex · 2025-07-27T09:21:38Z

I have opened a PR for supporting VectorChord in Immich. As it can do most of the migration automatically, this is mostly just some enablement and a test: #428568

diogotcorreia · 2025-07-27T13:04:44Z

I kinda forgot, but it likely makes sense to do a backport of this, so that we can backport #428568 later

Will do that manually soon, since it requires adding rust_1_88 I believe

With compound licenses[1] recently merged, we can now properly represent dual licensed packages without affecting its non-free status. See NixOS#392710 (comment) [1]: NixOS#468378

github-actions Bot added the 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS label Mar 24, 2025

nix-owners Bot requested review from Ma27, thoughtpolice and wolfgangwalther March 24, 2025 13:11

github-actions Bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Mar 24, 2025

dotlambda reviewed Mar 24, 2025

View reviewed changes

wolfgangwalther reviewed Mar 24, 2025

View reviewed changes

Comment thread nixos/tests/postgresql/vectorchord.nix Outdated

wolfgangwalther reviewed Mar 24, 2025

View reviewed changes

Comment thread nixos/tests/postgresql/vectorchord.nix Outdated

Titaniumtown suggested changes Mar 24, 2025

View reviewed changes

diogotcorreia force-pushed the postgres-vectorchord branch from 4a077c6 to 2d149c7 Compare March 24, 2025 22:57

github-actions Bot removed the 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS label Mar 24, 2025

wolfgangwalther mentioned this pull request Mar 27, 2025

postgresqlPackages.pgvectorscale: init at 0.7.0, buildPgrxExtension: support cargo workspaces #393689

Merged

13 tasks

diogotcorreia mentioned this pull request Mar 28, 2025

buildPgrxExtension: restructure with lib.extendMkDerivation #394089

Merged

13 tasks

wolfgangwalther mentioned this pull request Apr 26, 2025

postgresqlPackages: override is not supported #401819

Open

3 tasks

wolfgangwalther mentioned this pull request May 4, 2025

postgresqlPackages.omnigres: init at 0-unstable-2025-05-16 #404027

Merged

13 tasks

Ma27 reviewed May 9, 2025

View reviewed changes

Comment thread pkgs/servers/sql/postgresql/ext/vectorchord/package.nix Outdated

diogotcorreia force-pushed the postgres-vectorchord branch from 2d149c7 to d33602c Compare May 9, 2025 14:39

diogotcorreia commented Jun 24, 2025

View reviewed changes

diogotcorreia added 2 commits June 26, 2025 18:40

postgresqlPackages.{pgvectorscale,pgx_ulid}: pin cargo-pgrx 0.12.6

40c3580

cargo-pgrx_0_14_1: init; cargo-pgrx: 0.12.6 -> 0.14.1

136519b

diogotcorreia force-pushed the postgres-vectorchord branch from 564ed3e to 54f182d Compare June 26, 2025 17:41

carbolymer mentioned this pull request Jun 30, 2025

Update Request: pgvecto-rs 0.3.0 → 0.4.0 #381551

Closed

3 tasks

wolfgangwalther reviewed Jul 2, 2025

View reviewed changes

Comment thread pkgs/servers/sql/postgresql/ext/vectorchord/package.nix Outdated

postgresqlPackages.vectorchord: init at 0.4.2

ab82f5a

diogotcorreia force-pushed the postgres-vectorchord branch from 54f182d to ab82f5a Compare July 2, 2025 12:37

wolfgangwalther merged commit 04b04de into NixOS:master Jul 5, 2025
26 of 27 checks passed

diogotcorreia deleted the postgres-vectorchord branch July 5, 2025 12:25

wolfgangwalther mentioned this pull request Jul 7, 2025

buildPgrxExtension: force pinning cargo-pgrx #423140

Merged

7 tasks

diogotcorreia mentioned this pull request Jul 29, 2025

[25.05] postgresqlPackages.vectorchord: init at 0.4.2 #429374

Merged

13 tasks

diogotcorreia added the 8.has: port to stable This PR already has a backport to the stable release. label Jul 29, 2025

diogotcorreia mentioned this pull request Aug 10, 2025

nixos/immich: add support for VectorChord #428568

Merged

16 tasks

diogotcorreia mentioned this pull request Apr 20, 2026

postgresqlPackages.vectorchord: specify dual license #511722

Merged

13 tasks

Uh oh!

Uh oh!

Conversation

diogotcorreia commented Mar 24, 2025

Things done

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Titaniumtown left a comment

Choose a reason for hiding this comment

Uh oh!

diogotcorreia commented Apr 11, 2025

Uh oh!

Ma27 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

diogotcorreia commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diogotcorreia commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diogotcorreia commented May 9, 2025

Uh oh!

mertalev commented May 19, 2025

Uh oh!

diogotcorreia commented May 20, 2025

Uh oh!

mertalev commented May 20, 2025

Uh oh!

dotlambda commented May 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

diogotcorreia left a comment

Choose a reason for hiding this comment

nixpkgs-review result

x86_64-linux

Uh oh!

wolfgangwalther commented Jun 25, 2025

Uh oh!

wolfgangwalther commented Jun 25, 2025

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

Uh oh!

tecosaur commented Jul 1, 2025

Uh oh!

diogotcorreia commented Jul 1, 2025

Uh oh!

happysalada commented Jul 2, 2025

Uh oh!

wolfgangwalther left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

diogotcorreia commented Jul 2, 2025

Uh oh!

wolfgangwalther commented Jul 5, 2025

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

Uh oh!

Uh oh!

dotlambda commented Jul 22, 2025

Uh oh!

diogotcorreia commented Jul 22, 2025

Uh oh!

th3voic3 commented Jul 22, 2025

Uh oh!

Scrumplex commented Jul 27, 2025

diogotcorreia commented May 9, 2025 •

edited

Loading

diogotcorreia commented May 9, 2025 •

edited

Loading

dotlambda commented May 20, 2025 •

edited

Loading

`nixpkgs-review` result

`x86_64-linux`

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`