using file-based metadata - #102
Draft
pkdash wants to merge 89 commits into
Draft
Conversation
Member
Author
|
@sblack-usu Have a look at this initial implementation to make hsclient to work with file-based metadata. There is still quite a bit of work to do. It would be good to get your feedback on what I have implemented so far before I do more work on this. |
sblack-usu
self-requested a review
June 17, 2026 15:33
…ed on schema org (later these should be part of hsmodels)
…w schema org based pydnatic models and the legacy pydantic models
- Introduced S3 client setup and user credential retrieval for accessing S3 resources. - Updated Aggregation and Resource classes to support file-based metadata operations using S3. - Refactored metadata retrieval and saving methods to utilize JSON-LD format. - Enhanced file handling with size and checksum attributes in the File class.
…data and schema.org based resource metadata convertion
…d legacy aggregation types
…gregation using file_manifest.json
…retrieval when colllecting resource aggregations
…s3 protocol - replacing hsydroshare api endpoint use
…art of aggregation move functionality
…egation delete functionality
…urceMetadataAdapter and Relation classes
…e, and obsolete states
…funding fields to empty lists
… types in legacy resource metadata models
…y spatial coverage
… with schema org based models
…rect match betwen the two metadata formats
…ds for ScientificDataset round-tripping
…ScientificDataset round-tripping
… resource metadata tests
… fields in legacy format
…rip with the same field on the newer resource metadata model
…lumns' in resource adapter
… in extra_columns
… and improve spatial reference handling
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of this WIP PR:
Similar to resource level metadata mapping, metadata mapping needs to be implemented for each content type/aggregation to keep the aggregation level metadata editing interface unchanged. Order of implementation:
Tests are currently failing which need to be fixed and updated to use file-based json metadata (WIP)
New Pydantic models for file-based metadata in schema.org format used in hsclient should be moved to hsmodels so that both hydroshare and hsclient can share the same pydantic schemas/models