Skip to content

Discussion: Should ktx adopt Apache Ossie as the semantic model storage format? #359

Description

@TimothyHologres

What do you want, and why?

Background

ktx currently uses a custom YAML format for semantic sources (semantic-layer/<connection>/<source>.yaml). While this format works well internally, I've been looking at Apache Ossie (incubating) — an Apache Foundation project that defines a standard interchange format for semantic models.

I'd like to open a discussion on whether ktx should consider using (or at least supporting) the Ossie format for storing semantic information.

What is Apache Ossie?

Ossie (formerly OSI — Open Semantic Interchange) is a vendor-neutral YAML/JSON format for exchanging semantic model definitions across analytics, BI, and AI platforms. It's backed by 50+ companies including Snowflake, Databricks, dbt Labs, Cube, Tableau, Qlik, ThoughtSpot, GoodData, AtScale, Alation, Atlan, Collibra, and others.

The project already ships 10 bidirectional converters (dbt, Snowflake, Databricks, Salesforce, GoodData, Polaris, Omni, Honeydew, WisdomAI, OrionBelt).

Why I think this is worth discussing

Interoperability. Today, a semantic model defined in ktx can only be consumed by ktx. With Ossie, the same model could be exported and consumed by dbt, Snowflake Cortex Analyst, Databricks, Tableau, GoodData, etc. — and vice versa. This reduces vendor lock-in for users and lowers migration friction in both directions.

AI-native design. Ossie treats ai_context (instructions, synonyms, examples) as a first-class citizen on every field and metric. This is directly useful for ktx's AI agent use case, and more structured than ktx's current free-text descriptions map.

Multi-dialect SQL. Ossie's expression.dialects[] allows the same metric to carry different SQL for different engines (ANSI_SQL, Snowflake, BigQuery, Databricks, etc.). ktx already has sql_dialect_notes as a separate concept — Ossie integrates this directly into the model.

Ecosystem positioning. Being part of the Ossie converter ecosystem would increase ktx's visibility in the broader data stack community.

The gap

ktx's format has features that Ossie doesn't directly model:

  • grain (row-level uniqueness definition)
  • sql sources with CTEs
  • Overlay sources (adding measures/joins without a base table)
  • segments (reusable named filters)
  • joins[].relationship (many_to_one, one_to_many, etc.)
  • usage statistics (query frequency, common filters/group-bys)

These could be handled via Ossie's custom_extensions mechanism, or they could be reasons to keep the native format and only add import/export.

Questions for the community

  1. Is there interest in adopting Ossie as ktx's storage format, or is the native YAML format too different to make this practical?
  2. If full adoption is too disruptive, would import/export converters (keeping ktx's native format internally) be a reasonable middle ground?
  3. Has anyone else in the community looked at Ossie? Any experience or concerns to share?
  4. Given that Ossie is still at 0.2.0.dev0, is it too early to invest in this direction?

Which area?

CLI / setup

Want to contribute this?

  • Yes, I'd like to open a PR
  • I'd like guidance, then I'll open a PR
  • No, just reporting the idea

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageNeeds maintainer review before work begins

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions