mossJARVIS

// 01 · SYS // IDENTITY

Overview

// 02 · MEMORY ARCHITECTURE

Every turn, two memories, three writes

1. 01Directive inTyped in the command bar, or spoken after the wake word.
2. 02Dual recallWorking memory (top 5, token overlap with a recency tiebreak) and the Moss local index (top 5) are searched in parallel.
3. 03Context outResults are labelled working or long-term and sent to OpenRouter with the directive.
4. 04Strict JSON backThe model answers with { response, facts[], tasks[] }, validated before anything is stored. Prose answers still show; extraction is skipped.
5. 05Disk firstThe turn lands in .jarvis-data/second-brain.json (atomic write, mode 0600).
6. 06Moss localThen into a Moss SessionIndex, saved to second-brain.moss.
7. 07Moss CloudThen addDocs + loadIndex on jarvis-second-brain, if quota allows. A failure here loses nothing.

Try it above: flip CUT THE CLOUD to QUOTA EXCEEDED and send a directive. The cloud write fails, the HUD flashes red, and the turn is still on disk and in the local index.

// 03 · LOCAL EMBEDDING

How a thought becomes 384 numbers

No embedding API, no model download. localEmbedding() turns text into word features (w:launch) and character trigrams of ^word$, hashes each one FNV-1a style into one of 384 buckets with a sign from a second hash, and L2-normalises the result. Those vectors go to Moss as custom embeddings, so retrieval works even when hosted embedding quota is gone. The visualiser below runs that exact function.

// 04 · SECOND BRAIN

A cognitive archive you can open

Memories

Semantic search, type filters, an optional recency re-rank (half-life 45 days: 0.45 + 0.55·e^(-age/45)), full text, semantic neighbours, delete a vector.

Graph

A source-level knowledge graph from the embeddings. Each source keeps its 6 strongest neighbours; edges under 0.08 cosine are dropped.

Ingest

Markdown, text, reStructuredText, Org, CSV, logs, HTML, PDF, Word, ChatGPT and Claude exports, public articles, YouTube transcripts.

Remember

Write a fact, decision, preference or event straight into memory, no model call.

Chunks are heading-aware and capped near 3,600 characters, with deterministic IDs (source + section + part), so re-importing a file updates it instead of duplicating it. Prompt-role markers are neutralised and private-network URLs are rejected. Limits: 25 MB a file, 60 MB a batch.

Demo sources, embedded and linked with the same rules in your browser. Drag to pan, use the buttons (or pinch) to zoom, pick a source to inspect it.

// 05 · SUBSYSTEMS

What else it does

• TASK MATRIX

  Tasks and a morning briefing

  Extracted tasks are stored as Moss documents with type, status, due date, recurrence and priority. BRIEFING reads the open ones and asks for a short briefing: overdue first, then due today, then high priority. With no open tasks it answers without calling a model.

• VOICE MATRIX

  Voice when you want it

  Picovoice Porcupine listens for “Jarvis” on device, Cheetah streams speech to text and ends after about 1.15 s of silence, and ElevenLabs speaks the reply. It switches on only when those keys exist; text needs just Moss and OpenRouter.

• OVERLAY

  A real desktop overlay

  A Tauri 2 window: frameless, transparent, resizable, always on top, dragged by the HUD title bar. Jarvis.app bundles the Next.js standalone server, Moss's native Apple Silicon module and the Node executable. A Rust launcher starts and owns the server on 127.0.0.1.

• STATE MACHINE

  A HUD that shows its work

  booting, idle, listening, thinking, speaking, offline, each with its own reactor animation. Telemetry shows Moss query latency and how many vectors were recalled. Missing credentials or an API error turn the whole HUD red.

// 06 · BUILD LOG

How it's built

• ›One API route, POST /api/jarvis, with actions for status, models, init, turn, chat, briefing, memory search and the brain workspace; plus /api/jarvis/tts and a multipart /api/jarvis/brain/ingest.
• ›The working buffer keeps the latest 40 raw turns, and every exchange also becomes a durable conversation-summary document. Document types: system, conversation-turn, conversation-summary, fact, task.
• ›Voice runs Porcupine and Cheetah through their Web SDKs in workers. If ElevenLabs fails, it falls back to the OS en-GB voice.

CREDIT Adapts the architecture and ingestion ideas of moss-brain by Aarush Nigam (MIT license), reimplemented natively in TypeScript. moss-brain on GitHub (opens in a new tab)