Skip to content

refactor: catalog crate#331

Merged
v0y4g3r merged 31 commits into
developfrom
refactor/catalog-crate
Oct 26, 2022
Merged

refactor: catalog crate#331
v0y4g3r merged 31 commits into
developfrom
refactor/catalog-crate

Conversation

@v0y4g3r

@v0y4g3r v0y4g3r commented Oct 20, 2022

Copy link
Copy Markdown
Contributor
  • Add a standalone common-catalog crate that provides necessary consts and helper functions for integrating catalog.
  • CatalogList/CatalogProvider/SchemaProvider's methods now returns a Result<Option<T>> instead of Option<T>. This is also a follow-up of arrow-datafusion's API change.
  • CreateTableRequest's catalog_name/schema_name is changed from Option<String> to String, as for table engine, it should not be aware of the "default catalog name" stuff, catalog name and schema name must be explicitly given.

@v0y4g3r v0y4g3r closed this Oct 20, 2022
@v0y4g3r v0y4g3r force-pushed the refactor/catalog-crate branch from cf2b97b to fbea07e Compare October 20, 2022 09:18
@v0y4g3r v0y4g3r reopened this Oct 20, 2022
@v0y4g3r v0y4g3r force-pushed the refactor/catalog-crate branch from b5790f7 to c9f39d5 Compare October 20, 2022 10:40
@v0y4g3r v0y4g3r marked this pull request as ready for review October 20, 2022 10:42
@v0y4g3r v0y4g3r requested review from MichaelScofield and evenyag and removed request for evenyag October 20, 2022 11:12
@codecov

codecov Bot commented Oct 20, 2022

Copy link
Copy Markdown

Codecov Report

Merging #331 (2402658) into develop (7fe39e9) will decrease coverage by 0.08%.
The diff coverage is 73.09%.

@@             Coverage Diff             @@
##           develop     #331      +/-   ##
===========================================
- Coverage    84.41%   84.32%   -0.09%     
===========================================
  Files          351      354       +3     
  Lines        32273    32528     +255     
===========================================
+ Hits         27243    27430     +187     
- Misses        5030     5098      +68     
Flag Coverage Δ
rust 84.32% <73.09%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/catalog/src/system.rs 81.67% <ø> (ø)
src/common/catalog/src/error.rs 0.00% <0.00%> (ø)
src/datanode/src/sql/insert.rs 80.32% <0.00%> (-1.34%) ⬇️
src/script/src/table.rs 92.68% <ø> (-0.12%) ⬇️
src/sql/src/statements.rs 85.03% <ø> (ø)
src/table/src/engine.rs 100.00% <ø> (ø)
src/table/src/error.rs 75.36% <0.00%> (-2.25%) ⬇️
src/query/src/datafusion/catalog_adapter.rs 43.70% <18.18%> (-0.66%) ⬇️
src/catalog/src/error.rs 54.94% <20.00%> (-2.53%) ⬇️
src/catalog/src/tables.rs 70.58% <27.27%> (-0.58%) ⬇️
... and 32 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Comment thread src/catalog/src/lib.rs Outdated
Comment thread src/catalog/src/lib.rs Outdated
Comment thread src/catalog/src/tables.rs Outdated
Comment thread src/common/catalog/src/error.rs Outdated
Comment thread src/common/catalog/src/helper.rs Outdated
Comment thread src/common/catalog/src/consts.rs Outdated
Comment thread src/common/catalog/src/helper.rs
Comment thread src/common/catalog/src/helper.rs Outdated
Comment thread src/query/src/datafusion/catalog_adapter.rs Outdated
Comment thread test-util/src/memtable.rs Outdated
Comment thread src/common/catalog/src/error.rs Outdated
Comment thread src/datanode/src/error.rs
Comment thread src/datanode/src/server/grpc/ddl.rs Outdated
Comment thread src/common/catalog/src/helper.rs
Comment thread src/common/catalog/src/helper.rs
Comment thread src/common/catalog/src/helper.rs
Comment thread src/catalog/src/error.rs
Comment thread src/catalog/src/error.rs
@v0y4g3r v0y4g3r requested a review from waynexia October 21, 2022 06:23

@waynexia waynexia left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CreateTableRequest's catalog_name/schema_name is changed from Option to String, as for table engine, it should not be aware of the "default catalog name" stuff, catalog name and schema name must be explicitly given.

Good to see the third point. We should go depth with it 👍

I've skimmed this patch roughly, it looks good to me except the signature of CatalogManager::table()

Comment thread src/catalog/src/lib.rs Outdated
Comment thread src/catalog/src/tables.rs
@v0y4g3r v0y4g3r requested a review from fengjiachun October 25, 2022 12:57

@fengjiachun fengjiachun left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@v0y4g3r v0y4g3r merged commit 932b30d into develop Oct 26, 2022
@v0y4g3r v0y4g3r deleted the refactor/catalog-crate branch October 26, 2022 02:50
@v0y4g3r v0y4g3r mentioned this pull request Oct 27, 2022
14 tasks
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
* chore: refactor dir for local catalog manager

* refactor: CatalogProvider returns Result

* refactor: SchemaProvider returns Result

* feat: add kv operations to remote catalog

* chore: refactor some code

* feat: impl catalog initialization

* feat: add register table and register system table function

* refactor: add table_info method for Table trait

* chore: add some tests

* chore: add register schema test

* chore: fix build issue after rebase onto develop

* refactor: mock to separate file

* build: failed to compile

* fix: use a container struct to bridge KvBackend and Accessor trait

* feat: upgrade opendal to 0.17

* test: add more tests

* chore: add catalog name and schema name to table info

* chore: add catalog name and schema name to table info

* chore: rebase onto develop

* refactor: common-catalog crate

* refactor: remove remote catalog related files

* fix: compilation

* feat: add table version to TableKey

* feat: add node id to TableValue

* fix: some CR comments

* chore: change async fn create_expr_to_request to sync

* fix: add backtrace to errors

* fix: code style

* fix: CatalogManager::table also requires both catalog_name and schema_name

* chore: merge develop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants