docs: comprehensive documentation uplift#3
Merged
Merged
Conversation
- Added @doc group: metadata to all 38 public functions in Dux module: constructors, transforms, aggregation, joins, reshape, sorting, io, materialization, distribution - Updated mix.exs docs config: groups_for_docs, groups_for_modules (removed deleted Dux.Remote, added Telemetry/Flame), groups_for_extras, nest_modules_by_prefix, Mermaid via before_closing_body_tag - Placeholder files for new guides (data-io, transformations, joins) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrote guides as runnable Livebooks using embedded datasets: - getting-started.livemd — penguins dataset, admonitions, progressive - data-io.livemd — NEW: CSV/Parquet/NDJSON/globs, SQL escape hatch - transformations.livemd — NEW: filter/mutate/window/sort deep dive - joins-and-reshape.livemd — NEW: nycflights star schema, all join types, pivot_wider/longer, concat_rows - distributed-queries.livemd — rewritten with distribute API, Mermaid diagram, FLAME, telemetry - graph-analytics.livemd — rewritten with karate club dataset Updated README: removed Rust/NIF references, pure Elixir + ADBC. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete cheatsheet overhaul: - col-2 for macro vs raw SQL (filter, mutate, aggregation, pivot) - col-2 for read vs write IO - All join types with examples - Materialization section with all output functions - Distributed section with FLAME - Graph section with all algorithms + distribution - Nx interop - DuckDB-specific features (MEDIAN, PERCENTILE_CONT, window functions) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dux.Datasets provides clean access to embedded CC0 datasets: - penguins (344 rows), gapminder (1,704), flights (6,099), airlines (16), airports (1,458), planes (3,322) Datasets ship in priv/datasets/ (included in hex package) so guides work with Mix.install — no path hacks needed. All 5 guides verified cell-by-cell: - getting-started: 10/10 cells pass - transformations: 8/8 cells pass - joins-and-reshape: 7/7 cells pass - graph-analytics: 6/6 cells pass - data-io: 4/4 cells pass 435 tests, 0 failures, 0 credo issues. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Moduledoc-only module explaining the distributed system: architecture diagram (Mermaid), components (Worker, Coordinator, Broadcast), data flow (S3-native reads, IPC transfer), FLAME link. Added back to groups_for_modules in docs config. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrote from a terse reference card to a proper introduction: - Quick start with penguins dataset - How it works: build → compile → execute lifecycle - Expression syntax with interpolation explanation - _with variants for raw DuckDB SQL - Distribution overview with link to Dux.Remote - Embedded datasets reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Function docs improved across Dux and Dux.Graph: - local/1: explain inverse of distribute, add doctest - from_csv/2: add doctest with airlines dataset - from_parquet/2: add round-trip doctest - from_ndjson/2: explain format, add doctest - to_csv/3, to_ndjson/3: add examples, document return value - ungroup/1: explain context, add doctest - to_tensor/2: proper @doc with type conversion docs - head/2: explicit default of 10, add no-arg doctest - Graph.from_edgelist/2: add options section and doctest - Graph.degree/1: explain both endpoints, add doctest - Graph.vertex_count/1, edge_count/1: add doctests - Graph.triangle_count/1: document distributed support Fixed module grouping: added Dux.Remote.Shuffle and Dux.Datasets to groups_for_modules. 43 doctests (up from 37), all passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Best-in-class documentation for Dux — function groups, rewritten guides with real datasets, cheatsheet overhaul, comprehensive function docs with doctests.
Function documentation
DuxandDux.Graphnow has at least one meaningful doctest@doc group:metadata on all 38 public functions: Constructors, Transforms, Aggregation, Joins, Reshape, Sorting, IO, Materialization, Distributionlocal/1,from_csv/2,from_parquet/2,from_ndjson/2,to_csv/3,to_ndjson/3,ungroup/1,to_tensor/2,head/2,from_edgelist/2,degree/1,vertex_count/1,edge_count/1,triangle_count/1Duxmoduledoc: build → compile → execute lifecycle, quick start with penguins, distribution overviewDux.Remotemoduledoc with Mermaid architecture diagramGuides (all runnable Livebooks, verified cell-by-cell)
Dux.Datasets module
Ships CC0 datasets in
priv/datasets/so guides work withMix.install:Cheatsheet
Complete rewrite with ex_doc features:
{: .col-2}for macro vs raw SQL side-by-sideex_doc configuration
groups_for_docsbefore_closing_body_tagnest_modules_by_prefixforDux.Remote.*Other
Test plan
mix docsbuilds without errorsmix credo --strictzero issuesmix compile --warnings-as-errorsclean🤖 Generated with Claude Code