What do you want, and why?
Background
ktx currently uses a custom YAML format for semantic sources (semantic-layer/<connection>/<source>.yaml). While this format works well internally, I've been looking at Apache Ossie (incubating) — an Apache Foundation project that defines a standard interchange format for semantic models.
I'd like to open a discussion on whether ktx should consider using (or at least supporting) the Ossie format for storing semantic information.
What is Apache Ossie?
Ossie (formerly OSI — Open Semantic Interchange) is a vendor-neutral YAML/JSON format for exchanging semantic model definitions across analytics, BI, and AI platforms. It's backed by 50+ companies including Snowflake, Databricks, dbt Labs, Cube, Tableau, Qlik, ThoughtSpot, GoodData, AtScale, Alation, Atlan, Collibra, and others.
The project already ships 10 bidirectional converters (dbt, Snowflake, Databricks, Salesforce, GoodData, Polaris, Omni, Honeydew, WisdomAI, OrionBelt).
Why I think this is worth discussing
Interoperability. Today, a semantic model defined in ktx can only be consumed by ktx. With Ossie, the same model could be exported and consumed by dbt, Snowflake Cortex Analyst, Databricks, Tableau, GoodData, etc. — and vice versa. This reduces vendor lock-in for users and lowers migration friction in both directions.
AI-native design. Ossie treats ai_context (instructions, synonyms, examples) as a first-class citizen on every field and metric. This is directly useful for ktx's AI agent use case, and more structured than ktx's current free-text descriptions map.
Multi-dialect SQL. Ossie's expression.dialects[] allows the same metric to carry different SQL for different engines (ANSI_SQL, Snowflake, BigQuery, Databricks, etc.). ktx already has sql_dialect_notes as a separate concept — Ossie integrates this directly into the model.
Ecosystem positioning. Being part of the Ossie converter ecosystem would increase ktx's visibility in the broader data stack community.
The gap
ktx's format has features that Ossie doesn't directly model:
grain (row-level uniqueness definition)
sql sources with CTEs
- Overlay sources (adding measures/joins without a base table)
segments (reusable named filters)
joins[].relationship (many_to_one, one_to_many, etc.)
usage statistics (query frequency, common filters/group-bys)
These could be handled via Ossie's custom_extensions mechanism, or they could be reasons to keep the native format and only add import/export.
Questions for the community
- Is there interest in adopting Ossie as ktx's storage format, or is the native YAML format too different to make this practical?
- If full adoption is too disruptive, would import/export converters (keeping ktx's native format internally) be a reasonable middle ground?
- Has anyone else in the community looked at Ossie? Any experience or concerns to share?
- Given that Ossie is still at
0.2.0.dev0, is it too early to invest in this direction?
Which area?
CLI / setup
Want to contribute this?
What do you want, and why?
Background
ktx currently uses a custom YAML format for semantic sources (
semantic-layer/<connection>/<source>.yaml). While this format works well internally, I've been looking at Apache Ossie (incubating) — an Apache Foundation project that defines a standard interchange format for semantic models.I'd like to open a discussion on whether ktx should consider using (or at least supporting) the Ossie format for storing semantic information.
What is Apache Ossie?
Ossie (formerly OSI — Open Semantic Interchange) is a vendor-neutral YAML/JSON format for exchanging semantic model definitions across analytics, BI, and AI platforms. It's backed by 50+ companies including Snowflake, Databricks, dbt Labs, Cube, Tableau, Qlik, ThoughtSpot, GoodData, AtScale, Alation, Atlan, Collibra, and others.
The project already ships 10 bidirectional converters (dbt, Snowflake, Databricks, Salesforce, GoodData, Polaris, Omni, Honeydew, WisdomAI, OrionBelt).
Why I think this is worth discussing
Interoperability. Today, a semantic model defined in ktx can only be consumed by ktx. With Ossie, the same model could be exported and consumed by dbt, Snowflake Cortex Analyst, Databricks, Tableau, GoodData, etc. — and vice versa. This reduces vendor lock-in for users and lowers migration friction in both directions.
AI-native design. Ossie treats
ai_context(instructions, synonyms, examples) as a first-class citizen on every field and metric. This is directly useful for ktx's AI agent use case, and more structured than ktx's current free-textdescriptionsmap.Multi-dialect SQL. Ossie's
expression.dialects[]allows the same metric to carry different SQL for different engines (ANSI_SQL, Snowflake, BigQuery, Databricks, etc.). ktx already hassql_dialect_notesas a separate concept — Ossie integrates this directly into the model.Ecosystem positioning. Being part of the Ossie converter ecosystem would increase ktx's visibility in the broader data stack community.
The gap
ktx's format has features that Ossie doesn't directly model:
grain(row-level uniqueness definition)sqlsources with CTEssegments(reusable named filters)joins[].relationship(many_to_one, one_to_many, etc.)usagestatistics (query frequency, common filters/group-bys)These could be handled via Ossie's
custom_extensionsmechanism, or they could be reasons to keep the native format and only add import/export.Questions for the community
0.2.0.dev0, is it too early to invest in this direction?Which area?
CLI / setup
Want to contribute this?