Fix for - Minimal and full CI matrix impl #2051 by avifenesh · Pull Request #2500 · valkey-io/valkey-glide

avifenesh · 2024-10-22T19:40:31Z

This pull request includes significant updates to the CI/CD workflow documentation and configurations. The changes enhance clarity, add new testing capabilities, and improve the flexibility of the CI/CD pipeline.

Documentation Updates:

.github/DEVELOPER.md: Updated the CI/CD workflow guide with detailed descriptions of workflow triggers, test coverage levels, and language-specific workflows. Added images for better visualization. [1] [2] [3] [4] [5]

Configuration Enhancements:

.github/json_matrices/build-matrix.json: Added new languages (go, dotnet) and package managers (maven). Updated the structure to include a new languages field for better control over which languages are tested on each configuration.
.github/json_matrices/engine-matrix.json: Modified the version field to support both release tags and branch names for more flexible version control.
.github/json_matrices/supported-languages-versions.json: Added dotnet and go languages with their respective supported versions.

Workflow Updates:

.github/workflows/build-node-wrapper/action.yml: Made engine-version input optional and updated the install commands for better dependency management. [1] [2]
.github/workflows/build-python-wrapper/action.yml: Made engine-version input optional.
.github/workflows/create-test-matrices/action.yml: Enhanced the matrix creation logic to support more flexible and comprehensive test configurations. Added new inputs and outputs for better control over the test matrices. [1] [2] [3]

Workflow Trigger Updates:

.github/workflows/csharp.yml: Expanded the branches and paths that trigger the workflow. Added new inputs for manual triggers and updated the job structure to use the new matrix creation logic.

Issue link

#2524

Fix for #2051

Checklist

Before submitting the PR make sure the following are checked:

[v] This Pull Request is related to one issue.
[v] Commit message has a detailed description of what changed and why.
[v] Tests are added or updated.
[v] CHANGELOG.md and documentation files are updated.
[v] Destination branch is correct - main or release
[v] Commits will be squashed upon merging.

Yury-Fridlyand · 2024-10-25T01:16:50Z

Please add repo check for lint rust job in java, node, python and others' workflow files. It runs on forks by cron now.

Yury-Fridlyand · 2024-10-25T01:22:24Z

You also need to fix install shared deps step, because it ignores engine type and always installs valkey, even when redis requested

avifenesh · 2024-10-25T17:19:39Z

Please add repo check for lint rust job in java, node, python and others' workflow files. It runs on forks by cron now.

Are we running cron in forks? Shouldn't we disable the cron completely on forks?
@Yury-Fridlyand

Yury-Fridlyand · 2024-10-25T17:22:38Z

Are we running cron in forks? Shouldn't we disable the cron completely on forks?

Something went wrong and linter tasks run on forks

avifenesh · 2024-10-25T17:35:58Z

You also need to fix install shared deps step, because it ignores engine type and always installs valkey, even when redis requested

I don't have a problem to fix, but it's redundant. 7.2 < it's the same code. Over, we don't use Redis.

Yury-Fridlyand · 2024-10-25T17:50:52Z

You also need to fix install shared deps step, because it ignores engine type and always installs valkey, even when redis requested

I don't have a problem to fix, but it's redundant. 7.2 < it's the same code. Over, we don't use Redis.

This step goes to valeky repo and tries to checkout 6.2.0 and fails

avifenesh · 2024-10-25T18:17:46Z

You also need to fix install shared deps step, because it ignores engine type and always installs valkey, even when redis requested

I don't have a problem to fix, but it's redundant. 7.2 < it's the same code. Over, we don't use Redis.

This step goes to valeky repo and tries to checkout 6.2.0 and fails

Ill check it, but the Valkey repo has 6.2, that's how i use 6.2.

Yury-Fridlyand · 2024-10-25T19:11:51Z

https://github.com/Bit-Quill/valkey-glide/actions/runs/11510075627/job/32041282743#step:5:422

Yury-Fridlyand · 2024-10-25T19:48:37Z

Currently, GHA install valkey 7+ instead redis 6.2 due to cache misuse. Please drop all caches and rerun CI.

avifenesh

Currently, GHA install valkey 7+ instead redis 6.2 due to cache misuse. Please drop all caches and rerun CI.

so we need to avoid cache in valkey instalation unsless were able to create a seperation

avifenesh · 2024-10-25T21:31:32Z

https://github.com/Bit-Quill/valkey-glide/actions/runs/11510075627/job/32041282743#step:5:422

Its need to be 6.2

avifenesh · 2024-10-27T01:40:57Z

    return responseData;
 }

-/**


This is wrong and a repetitive usage of dump and restore test

Yury-Fridlyand

A couple last comments.
Nice work!

Yury-Fridlyand · 2024-10-30T16:52:02Z

@avifenesh CI is red, please have a look

avifenesh · 2024-10-30T17:09:16Z

@avifenesh CI is red, please have a look

Yep, adding support for py 3.13, new errors, on it

Signed-off-by: avifenesh <aviarchi1994@gmail.com>

…pts and dependencies Signed-off-by: avifenesh <aviarchi1994@gmail.com>

…t versions Signed-off-by: avifenesh <aviarchi1994@gmail.com>

… version checks Signed-off-by: avifenesh <aviarchi1994@gmail.com>

…l to actual reads; update connection handling in utilities and allow unused imports in types Signed-off-by: avifenesh <aviarchi1994@gmail.com>

…mprove type handling Signed-off-by: avifenesh <aviarchi1994@gmail.com>

…e version requirements, and remove obsolete Redis installation workflow Signed-off-by: avifenesh <aviarchi1994@gmail.com>

avifenesh requested a review from a team as a code owner October 22, 2024 19:40

avifenesh changed the title ~~Ci/trriger by label~~ Fix for - Minimal and full CI matrix impl #2051 Oct 22, 2024

avifenesh force-pushed the CI/TrrigerByLabel branch from 042d76f to 96f872f Compare October 22, 2024 19:46

avifenesh had a problem deploying to AWS_ACTIONS October 22, 2024 19:47 — with GitHub Actions Error

avifenesh force-pushed the CI/TrrigerByLabel branch from 96f872f to 569ef04 Compare October 22, 2024 20:10

avifenesh had a problem deploying to AWS_ACTIONS October 22, 2024 20:10 — with GitHub Actions Error

avifenesh force-pushed the CI/TrrigerByLabel branch from 569ef04 to e66fc9f Compare October 22, 2024 20:18

avifenesh temporarily deployed to AWS_ACTIONS October 22, 2024 20:18 — with GitHub Actions Inactive

Yury-Fridlyand mentioned this pull request Oct 22, 2024

Rc 1.2 fix #2503

Merged

Yury-Fridlyand reviewed Oct 22, 2024

View reviewed changes

Comment thread .github/workflows/create-test-matrices/action.yml

Yury-Fridlyand added the CI/CD ⚒️ CI/CD related label Oct 22, 2024

avifenesh force-pushed the CI/TrrigerByLabel branch from e66fc9f to 58c1a7f Compare October 22, 2024 23:06

avifenesh had a problem deploying to AWS_ACTIONS October 22, 2024 23:06 — with GitHub Actions Error

avifenesh force-pushed the CI/TrrigerByLabel branch from 58c1a7f to 7f1a5e6 Compare October 22, 2024 23:14

avifenesh had a problem deploying to AWS_ACTIONS October 22, 2024 23:15 — with GitHub Actions Error

avifenesh temporarily deployed to AWS_ACTIONS October 22, 2024 23:15 — with GitHub Actions Inactive

avifenesh had a problem deploying to AWS_ACTIONS October 22, 2024 23:15 — with GitHub Actions Error

Yury-Fridlyand reviewed Oct 25, 2024

View reviewed changes

Comment thread node/tests/TestUtilities.ts

Yury-Fridlyand reviewed Oct 25, 2024

View reviewed changes

Comment thread node/tests/TestUtilities.ts

Yury-Fridlyand reviewed Oct 25, 2024

View reviewed changes

Comment thread .github/workflows/npm-cd.yml Outdated

Comment thread .github/workflows/python.yml Outdated

Comment thread node/tests/GlideClient.test.ts Outdated

Comment thread node/tests/GlideClusterClient.test.ts

Comment thread node/tests/TestUtilities.ts Outdated

Yury-Fridlyand mentioned this pull request Oct 25, 2024

Fix docs for wait and geo commands #2521

Open

2 tasks

avifenesh commented Oct 25, 2024

View reviewed changes

avifenesh commented Oct 27, 2024

View reviewed changes

Yury-Fridlyand reviewed Oct 28, 2024

View reviewed changes

Comment thread .github/workflows/node.yml

Comment thread .github/workflows/pypi-cd.yml

Comment thread .github/workflows/python.yml Outdated

Comment thread node/package.json Outdated

Comment thread node/tests/AsyncClient.test.ts Outdated

Yury-Fridlyand approved these changes Oct 29, 2024

View reviewed changes

avifenesh added 7 commits October 30, 2024 17:44

Refactor tests to use async cleanup and improve error handling

0a363fe

Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Enhance Jest configuration and add test setup file; update build scri…

6f3ed12

…pts and dependencies Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Update devDependencies in package.json for hybrid-node-tests to lates…

f4b171d

…t versions Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Enhance test utilities and command tests with improved wait logic and…

69ce9e7

… version checks Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Refactor tests to assert expected replica reads are less than or equa…

662fd0d

…l to actual reads; update connection handling in utilities and allow unused imports in types Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Update dependencies and enhance PyO3 bindings; add new features and i…

97fc107

…mprove type handling Signed-off-by: avifenesh <aviarchi1994@gmail.com>

Update GitHub workflows: enhance linting configurations, adjust engin…

296c5f7

…e version requirements, and remove obsolete Redis installation workflow Signed-off-by: avifenesh <aviarchi1994@gmail.com>

This was referenced Oct 30, 2024

Add support for Python 3.13 #2499

Closed

Merge implemented tasks from main to release — CI enhancement and fixes, test fixes and py3.13 from main to release1.2 branch, adding support for 3.13 to docs. #2548

Closed

Uh oh!

Conversation

avifenesh commented Oct 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Updates:

Configuration Enhancements:

Workflow Updates:

Workflow Trigger Updates:

Issue link

Checklist

Uh oh!

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

avifenesh commented Oct 25, 2024

Uh oh!

Uh oh!

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

avifenesh commented Oct 25, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

avifenesh commented Oct 25, 2024

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

Yury-Fridlyand commented Oct 25, 2024

Uh oh!

avifenesh left a comment

Choose a reason for hiding this comment

Uh oh!

avifenesh commented Oct 25, 2024

Uh oh!

avifenesh Oct 27, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yury-Fridlyand left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Yury-Fridlyand commented Oct 30, 2024

Uh oh!

avifenesh commented Oct 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

avifenesh commented Oct 22, 2024 •

edited

Loading