Skip to content

Resources

Resources represent what data can be accessed. In Alien Giraffe this means two things working together: a registered datasource (the connection) and its catalog (what the scanner found inside it).

A request’s scope names datasources and datasets directly — see Policies. This page covers how those datasources get registered and cataloged in the first place.

Type Description
postgres PostgreSQL database
oracle Oracle database
mysql MySQL database
mssql SQL Server database — admin registration, health checks, and catalog scanning are supported; env-enzo materialization and the proxied request type are not yet supported for this type
snowflake Snowflake warehouse
s3 Amazon S3 bucket (CSV, JSON, JSONL, Parquet)
ssh Remote files over SSH (CSV/JSON)

Each type has its own config schema (host/port/database for the SQL types, bucket/region/prefix for S3, and so on) — see Configure Data Sources for the exact fields and admin API payloads.

Datasources are managed through the admin datasource API (admin role gets read-only access, super_admin can create/edit/disable):

  • GET /v1/admin/datasources
  • POST /v1/admin/datasources (super_admin)
  • PATCH /v1/admin/datasources/:id (super_admin)
  • DELETE /v1/admin/datasources/:id (super_admin, soft-disable — the record and its historical catalog entries are preserved, and the name becomes available for reuse)

In the dashboard this is the Admin → Settings → Datasources form: config is edited as key/value fields, not raw JSON, and any field can be marked is_secret: true.

Credentials are never stored in plain config. Each datasource carries a secret_refs pointer into a pluggable backend:

Backend How it works
java-keystore (current default) Credentials live in a Java KeyStore file, referenced by alias. Configured with A10E_KEYSTORE_PATH / A10E_KEYSTORE_PASSWORD.
file Secret values are written under A10E_SECRETS_FILE_ROOT and referenced by path.
aws-secrets-manager Secret values are stored in AWS Secrets Manager.

Secret fields are write-only: they’re accepted on create/update but never returned by the admin API or shown in the UI afterward. On update, an existing secret value is preserved unless you explicitly replace or clear it. At runtime — including scanner runs — secret-marked fields are resolved from secret_refs and merged into the effective config before the datasource driver uses it.

Once a datasource is registered, the catalog scanner connects to it and builds the catalog:

  1. Connection — credentials are resolved from the secret backend.
  2. Schema discovery — table/column structure is extracted with database-specific queries (information_schema for Postgres/MySQL/MSSQL, ALL_TABLES for Oracle, native listing for S3).
  3. Data sampling — a configurable number of rows are sampled (default 100).
  4. De-identification — PII in the sample is masked using A10E_DEIDENTIFY_SECRET_KEY.
  5. Change detection — a SHA-256 hash of the schema is stored, so rescans can detect drift.
  6. Catalog update — the result (schema, sample, AI summary) is written to the catalog table.

The target field on a datasource controls scope: a single table name, a glob pattern (CARD*, matched with Go’s path.Match syntax), or a comma-separated list of patterns. For S3, patterns are passed to DuckDB’s native glob expansion.

AI is off by default: leave AI_PROVIDER unset (or provider = "" in a10e.toml) for deployments that must not send catalog data to an AI provider. The scanner still discovers schema deterministically via information_schema/DuckDB — but:

  • no AI provider key is required
  • new catalog entries get an empty summary, written by hand later in the dashboard’s Catalog Targets editor
  • rescans never overwrite a hand-written summary
  • “Create with AI” and NL-to-SQL return 501 and are hidden in the dashboard

See Catalog Scanning for how to run the scanner end to end.

  • Policies — how a resource’s scope combines with the other four components
  • Subjects — who can register and read datasources (super_admin/admin)
  • Constraints — how a request’s scope over a resource gets approved and time-bounded
  • Channels — how approved data from a resource is actually delivered
  • Context — the justification tied to a request that references a resource