/projects/airia-datastores
← All projectsAiria.DataStores.Common
One query surface over six database engines, shipped as a package.
R&D Engineer — author, from the first commit · Jul 2025 – Oct 2025
Overview
A shared library that answers one question for every database an enterprise might point at an AI agent: how do you run a query and read a schema without the caller knowing which engine it is talking to.
What I did
I started this repository and wrote its first version — the interfaces, the providers, the pooling and the package pipeline that publishes it.
- The provider interface, and the relational implementations behind it.
- Schema metadata retrieval, as part of the contract rather than an extra.
- Connection pooling and the factory that hands out pooled stores.
- The document-store provider and its client wrapper.
- Unit tests and the publish workflow that versions the package.
Other engineers added providers and fixes on top of it after the first release.
The problem
The connector needed to query whatever database a customer happened to run, and the platform needed exactly the same thing from its own side. Written twice, that is two provider matrices, two sets of connection-string quirks and two places for a TLS default to be wrong — and they drift, because nobody fixes a bug in the copy they are not looking at.
Architecture
A caller asks a factory for a store of a given type and hands it connection parameters as a dictionary rather than a pre-built connection string, so nothing upstream has to know each engine’s spelling. The factory returns a pooled store; the store exposes the same two operations — execute a query, describe the tables — whatever driver is underneath. Document stores get a sibling interface, because pretending a collection is a table would be a lie the caller eventually pays for.
- CallerThe connector or the platform, holding connection parameters and a query.
- FactoryResolves the engine type to an implementation.
- Connection poolHands back a live store and reclaims it after use, capped per configuration.
- StoreTwo operations only: execute a query, describe the tables.
- Engine driverThe vendor client, and the only place an engine’s quirks are allowed to live.
Engines behind the one interface
| Engine | Family |
|---|---|
| PostgreSQL | Relational |
| SQL Server | Relational |
| MySQL | Relational |
| Snowflake | Warehouse |
| Databricks | Warehouse |
| MongoDB | Document |
What it does
- Six engines behind one interface, with the document store kept honestly separate.
- Schema description is part of the contract, not something bolted on later.
- Connection pooling behind the factory, so no caller manages a lifetime it did not open.
- Connection parameters as a dictionary — the library, not the caller, knows each engine’s spelling.
- Published as a versioned package, consumed by both the connector and the platform.
Engineering decisions
Reading the schema is part of the interface
A human writing SQL already knows the tables. A model does not, and asking it to guess produces queries that fail in ways that look like the database is broken. Making schema description a first-class operation alongside query execution is what turns the library from a connection helper into something an agent can actually be pointed at.
Parameters as a dictionary, never a connection string
Every engine spells the same idea differently — host versus server, the port that is implied, how encryption is requested. Accepting a built string would push that trivia into every caller and, worse, make each caller responsible for the security defaults. Taking a dictionary keeps one place where a wrong default can be fixed for everybody.
A published package, not shared source
The connector and the platform are separate repositories on separate release cadences. Copying the source would have been faster on day one and would have guaranteed divergence by month two. A versioned package makes the shared thing an actual dependency: an upgrade is a deliberate act with a number attached, and a fix reaches both consumers or neither.
Pooling belongs to the library, and its ceiling is configuration
Callers that open connections directly leak them under load, and the leak surfaces as an unrelated timeout somewhere else. Putting the pool behind the factory makes the correct thing the default thing. The maximum is a setting rather than a constant because the right ceiling for a connector on one customer machine is not the right ceiling for the platform — and the first default shipped turned out to be too low, and was raised five-fold.