[2026-05-30] personal os schema, state snapshot, catchup plan, wiki ingest process
This commit is contained in:
Alexey Martemyanov
@@ -0,0 +1,115 @@
|
||||
---
|
||||
source: raw/schema.sql
|
||||
content_hash: 71b47e47b483834c887de14c76d5b16506d90ad64198d76a92a7b3731132dab8
|
||||
namespace: work
|
||||
last_synced: 2026-05-30
|
||||
confidence: 0.9
|
||||
tags: [personal-os, postgres, schema, asana]
|
||||
---
|
||||
|
||||
# Personal OS — Postgres Schema
|
||||
|
||||
The Personal OS Postgres database holds Asana sync state, agent annotations,
|
||||
file references for wiki ingest, the LLM-synthesised wiki, and a semantic
|
||||
memory store. Extensions required: `vector` (pgvector) and `pg_trgm`.
|
||||
|
||||
## Core Asana tables
|
||||
|
||||
### `tasks`
|
||||
One row per Asana task GID; upserted on every sync. Tracks identity, due
|
||||
dates, completion, assignee, project, and a `source` enum that records how
|
||||
the task was discovered:
|
||||
- `my_tasks` — assigned to me, lives in a My Tasks section
|
||||
- `following` — I'm a follower (commented, CC'd)
|
||||
- `delegated` — I created it, someone else owns it
|
||||
- `project` — visible via a project I'm on, none of the above
|
||||
|
||||
Bookkeeping columns: `fetched_at`, `stories_fetched_before` (cursor used so
|
||||
we only pull new stories), `raw_json` (fallback for unmodeled fields).
|
||||
Indexes cover `modified_at`, `due_on` (partial — incomplete only),
|
||||
`assignee_gid`, `source`, `completed`, and a trigram index on `name` for
|
||||
fuzzy match.
|
||||
|
||||
### `stories`
|
||||
Raw Asana event log per task. The key resource subtypes are
|
||||
`comment_added`, `assigned`, `due_date_changed`, `section_changed`,
|
||||
`dependency_added`, `attachment_added`, `marked_complete`. Both plain text
|
||||
and raw HTML are stored — HTML is needed to extract embedded task GIDs.
|
||||
|
||||
### `task_edges`
|
||||
Directed graph of relationships between tasks. `relation_type` covers
|
||||
`subtask`, `dependency` (blocked by), `dependent` (blocking), `project_sibling`,
|
||||
and `mention` (referenced inside a story). `related_gid` may not yet
|
||||
exist in `tasks`, so it's not a FK.
|
||||
|
||||
### `task_annotations`
|
||||
Agent-written notes about tasks. `annotation_type` is one of `irrelevant`,
|
||||
`watching`, `needs_action`, `snoozed`. `annotated_by` is `agent` or `user`.
|
||||
|
||||
### `sync_state`
|
||||
One row per logical sync stream (`my_tasks`, `following`, `delegated`,
|
||||
`project:{gid}`, `workspace_events`). Stores `cursor` (ISO timestamp or
|
||||
events `sync_token`) and `full_sync_at` so partial syncs can fall back to
|
||||
a full pull when needed.
|
||||
|
||||
### `task_embeddings`
|
||||
1024-dim Voyage embedding per task, indexed via HNSW with cosine ops.
|
||||
|
||||
### `active_tasks` view
|
||||
Convenience view: non-completed tasks that aren't annotated `irrelevant`
|
||||
or currently snoozed. Ordered by source priority (my_tasks → delegated →
|
||||
following → project), then `due_on`, then `modified_at`.
|
||||
|
||||
## Phase 0 additions (April 2026)
|
||||
|
||||
All core tables gained a `namespace TEXT NOT NULL DEFAULT 'work'` column —
|
||||
the three allowed values are `work`, `personal`, `family`. Applied via
|
||||
ALTER TABLE; documented as commented-out statements at the bottom of the
|
||||
file for fresh installs.
|
||||
|
||||
`tasks` also gained `possibly_deleted BOOLEAN` and `last_seen_in_full_sync
|
||||
TIMESTAMPTZ` — used by `generate-status.js` to filter out tasks that have
|
||||
disappeared from Asana between full syncs.
|
||||
|
||||
## Wiki + file ingest tables
|
||||
|
||||
### `file_references`
|
||||
Tracks files and URLs that should be ingested into the wiki. Holds a macOS
|
||||
security-scoped `bookmark_data` BYTEA, `last_known_path` (resolved
|
||||
cache), optional `url`, `content_hash` (SHA256), `mime_type`, `title`, and
|
||||
a `modification_log` JSONB of `{ts, hash_before, hash_after}` entries.
|
||||
The `wiki_stale` flag drives whether the wiki-ingest prompt processes the
|
||||
file on its next run.
|
||||
|
||||
### `wiki_pages`
|
||||
LLM-synthesised wiki content (not raw copies). Columns: `namespace`,
|
||||
`title`, `content` (markdown), `sources` JSONB pointing back at
|
||||
`file_references`, `confidence`, `superseded_by` (chains old versions),
|
||||
`stale`, `last_synced_hash`, and a 1536-dim `embedding`. Tagged for
|
||||
ivfflat index but the index is held off until the table has 1000+ rows
|
||||
for usable recall.
|
||||
|
||||
> Note: per Decision 8 in [[personal-os-state-2026-04-27]], the live
|
||||
> Personal OS no longer uses these DB tables for wiki ingest — the vault
|
||||
> is the source of truth and SHA256 lives in frontmatter. The tables
|
||||
> remain in the schema for compatibility / future re-use.
|
||||
|
||||
### `memory_store`
|
||||
Semantic memory accumulated from Discord, Claude sessions, and manual
|
||||
entries. `type` ∈ {`fact`, `preference`, `decision`, `person`}; `source` ∈
|
||||
{`discord`, `claude`, `manual`}. Includes `entities` JSONB, `confidence`,
|
||||
`embedding` (1536), and an optional `expires_at` (NULL = permanent).
|
||||
|
||||
## Key Points
|
||||
|
||||
- All namespaces gated to `work | personal | family` via CHECK constraints.
|
||||
- pgvector embeddings live on three tables: `task_embeddings`, `wiki_pages`,
|
||||
`memory_store`. Only `task_embeddings` ships with an HNSW index;
|
||||
the wiki/memory ivfflat indexes are commented out until data volume
|
||||
justifies them.
|
||||
- The `active_tasks` view is the canonical "what should I look at" surface
|
||||
for agents.
|
||||
|
||||
## Related
|
||||
[[personal-os-state-2026-04-27]] [[personal-os-catchup-plan-2026-04-27]]
|
||||
[[wiki-ingest-process]] [[ddg-asana-workflow]]
|
||||
Reference in New Issue
Block a user