Architecture¶
semantido is a small library with a linear pipeline and no state. Reading it end to end takes about twenty minutes.
semantido/
├── concepts.py # canonical import path for the registry objects
├── decorators/
│ └── semantic_table.py # writes dunders onto the class
├── models/
│ ├── semantic_base.py # SemanticBase mixin — sync_semantic_layer()
│ └── declarative_base.py # Base, SemanticDeclarativeBase
├── generators/
│ ├── semantic_bridge.py # extraction engine: mappers -> SemanticLayer
│ ├── semantic_layer.py # the dataclasses + enums (the IR)
│ ├── concept_registry.py # ConceptRegistry, Concept, relations, mappings
│ └── utils/
│ ├── sqlalchemy_mapping.py # type normalisation, metadata reads, FK resolution
│ └── time_grain.py # grain normalisation + type compatibility
├── lint/
│ └── linter.py # lint_layer() — SL001–SL010, Tier-2 static
└── exporters/
├── json_exporter.py
├── markdown_exporter.py # tiered: schema / enriched / concepts
├── ossie_exporter.py
├── skos_exporter.py # SKOS Turtle concept scheme
├── groundings_exporter.py # meaning/deployment split — groundings.yaml
└── utils/
└── markdown.py # shared block renderers
The pipeline¶
@semantic_table → dunders on the class
<col>_* attributes → plain class attributes
│
▼
SemanticBase.sync_semantic_layer()
│
▼
SQLAlchemySemanticBridge.sync_from_models()
├─ walk the declarative registry
├─ _extract_table ← extract_table_metadata (dunders)
├─ _extract_column ← extract_column_metadata (<col>_* attrs)
│ ← resolve_foreign_key (mapper)
│ ← normalize_time_grain
└─ _extract_relationships ← join conditions + direction (mapper)
│
▼
SemanticLayer
│
┌──────────┬─────────┼──────────────┬─────────────────┐
▼ ▼ ▼ ▼ ▼
to_json to_markdown to_ossie_* to_skos_turtle to_groundings_*
│
▼
semantido.lint · lint_layer(layer, groundings)
The registry rides along: sync_semantic_layer(concept_registry=...) validates every concept= reference and attaches the registry to the layer, so exporters can render the bundle and the linter can check the seams.
Three stages: author (attributes on classes), extract (bridge → IR), render (exporter → text). Each is independently testable, and the seam between extract and render is why the same layer can serve a prompt and a BI tool.
Why attributes instead of a column wrapper¶
The obvious design is a SemanticColumn(...) wrapper replacing Column(...). semantido reads <column>_description class attributes instead. This trades IDE support for reach.
A wrapper only annotates columns you construct. The attribute convention annotates any mapped column — from a mixin, an inherited base, a __table__ reflection, a hybrid property. Given that reflection against an existing warehouse is a first-class use case (see dbt guide), the wrapper design would have excluded it.
The cost is real and worth naming: typos are silent. That's what the coverage test is for.
Why the IR exists¶
The SemanticLayer dataclass tree could have been skipped — the bridge could render Markdown directly. It exists because:
- Exporters multiply. N exporters × one IR beat N × mapper-walking code.
- It's the test surface. Asserting on
layer.tables["orders"].time_dimensionis a better test than grepping rendered Markdown. - It's the extension point. Filter it, scope it, inject a glossary — all before export. The privacy filtering pattern only works because there's an object between extraction and rendering.
Plain dataclasses, no validation framework, no ORM of its own.
Where determinism comes from¶
Two structural properties, no discipline required:
- No I/O in the path. No database, no network, no LLM. Nothing that can vary.
- Ordering follows the mapper, which follows declaration order in your files.
See Determinism.
The bridge is cached, the layer is not¶
get_semantic_bridge() lazily builds one bridge per base and caches it on the class (_semantic_bridge). sync_semantic_layer() clears and re-extracts on every call — it is a full re-walk, not an incremental update.
That's inexpensive but not free. Call it once at startup; don't call it per request. See Build once.
What isn't here¶
No CLI. No server. No config file. No plugin system. No database driver. No query engine.
The library does one thing — mappers in, document out — and the surface is deliberately small enough that the whole thing fits in your head.