gmail-hook/AGENTS.md

Agent notes — gmail-hook

npm workspaces: apps/api, apps/web, apps/watcher. Goal is e-receipt email → parsed spending → PocketBase, not a generic Gmail client.

Layout

App	Role
apps/api	Google OAuth, session cookie, Gmail read API. Tokens in SQLite apps/api/data/app.db.
apps/web	React login / inbox UI (Vite).
apps/watcher	Gmail Pub/Sub push consumer. Matches receipts, parses JSON, writes spendings.

Shared MIME helpers live in each app (mime.js). Watcher reuses the same SQLite tokens as the API.

Watcher pipeline

POST /pubsub/gmail → history.list (messageAdded) → per message:

1. logEvent + fetchAndLogMessage — fetch outer mail; collect emlMessages if any
2. Work items: if .eml / message/rfc822 attachments exist → process each attachment only (outer is a wrapper). If none → process the outer message as usual.
3. For each work item (sequential, in-process — no job queue): matchRule → parseMatchedMessage → saveSpending

Handlers return { pass: {...} } to merge into ctx or { break: true } to stop. Parse full message.body (and message.html when needed), never the 500-char log bodyPreview.

Attached .eml mails use nested From/Subject from Gmail’s expanded message/rfc822 part. When nested HTML/text is not inlined (body.data missing, only attachmentId), the watcher fetches those attachments (hydrateRfc822Message) before parse. Synthetic message_id looks like outerId#partId; dedupe remains on invoice_id.

Match rules

Resolve original sender:

1. If body has ---------- Forwarded message ---------, use its From / Subject / Date.
2. Else use Gmail envelope.

kind	From address	Subject
livin.qr_payment.success	noreply.livin@bankmandiri.co.id	Pembayaran Berhasil!
grab.ereceipt.ride	no-reply@grab.com	Your Grab E-Receipt
grab.ereceipt.tip	same From/Subject as ride; detected from tip body copy	tip invoice_id = {Booking ID}:tip
grab.ereceipt.food	same From/Subject; body has GrabFood / Selamat menikmati makanan	Pesanan ID as invoice_id
bri.transfer.success	bankbri@bri.co.id	Pemindahan Dana Sesama Rekening BRI
bri.qris.success	bankbri@bri.co.id	Pembelian QRIS Berhasil
tokopedia.order.completed	noreply@tokopedia.com	Pesanan Selesai

Production path is direct provider mail. Forwards still work via the Fwd header. Do not match on the forwarder envelope From.

Rules: apps/watcher/src/rules/. Parsers: apps/watcher/src/parsers/.

Parsing notes

• Required for a spending row: amount + reference (invoice id). Fail soft (log + break) if missing — never throw (Pub/Sub must ack).
• Livin: support clean forwarded lines and jammed HTML-stripped labels (Tanggal13 Sep 2026, Tanggal15 Agu 2026). Indonesian month names/abbreviations (Agu/Agustus, Des, …) are accepted. When merchant and city are glued on one line, keep the whole string as merchant; only split location when it is already a separate line ending in - ID. Do not maintain a city-name list.
• Grab: plain text often has date-only Picked up on …; pickup/dropoff times (10:47AM) live in HTML. Watcher keeps message.html and merges the first standalone AM/PM time into occurredAt (WIB). Tip receipts (Your tip goes a long way…, timestamp like 13 Sep 26 10:47 +0700) become grab.ereceipt.tip with invoice_id {bookingId}:tip so they do not collide with the ride.
• BRI / BRImo: transfer (Pemindahan Dana Sesama Rekening BRI) — amount from Nominal, invoice_id from Nomor Referensi, description from Nama Tujuan, datetime from Tanggal. QRIS (Pembelian QRIS Berhasil) — amount from Nominal (fallback Total Transaksi), description from Nama Merchant, datetime from Tanggal Transaksi. Other BRI subjects stay out of scope until samples appear.
• Tokopedia: Pesanan Selesai — one row per No.Invoice; amount from Total belanja (not fee lines); description = {Toko} - Tokopedia; line items (1+) in details.items; trx_date from Tanggal Terima (date-only).
• Open-ended text (names, products, driver, compliments) is best-effort optional details.

PocketBase

Env: POCKETBASE_URL, POCKETBASE_USER, POCKETBASE_PASSWORD (a _superusers account). Collection: spendings.

• Unique invoice_id — duplicate insert → log skip, no throw.
• Leave category empty; user fills it in PocketBase admin.
• Store flat columns for querying; details for kind leftovers; parsed for the full watcher envelope JSON (audit snapshot). Keep message_id so the original Gmail message can be re-fetched if needed — do not store raw email body by default.
• Schema ensure runs on watcher boot (ensureSpendingsSchema).
• Never commit secrets.

Parse failures and non-duplicate skips (incomplete record, save errors) POST to ntfy (NTFY_URL, default https://n.0dev.web.id/system) with messageId and message body. Duplicates do not notify.

Run

npm install
npm run dev            # api + web
# sign in once via UI so SQLite has tokens
npm run dev:watcher
npm test -w watcher    # unit tests (no Gmail/Pub/Sub)

Manual / backfill (forwarded old receipts)

Gmail Date on a forward is “today”; trx_date still comes from the receipt body. Duplicates skip on unique invoice_id. Forward-as-attachment (multiple *.eml) is expanded the same way as the watcher.

# List id / from / subject (from Gmail API)
npm run list-messages --
npm run list-messages -- --query 'newer_than:14d has:attachment filename:eml' --max 100

# Dry-run last 30 days of inbox (default query: in:inbox newer_than:30d)
npm run process-messages -- --dry-run

# Process a wrapper mail that has .eml attachments
npm run process-messages -- --ids OUTER_MESSAGE_ID

# Process and save
npm run process-messages -- --query 'newer_than:2d subject:"Pembayaran Berhasil!"'
npm run process-messages -- --ids MESSAGE_ID_1,MESSAGE_ID_2
npm run process-messages -- --user eleven16th@gmail.com --max 100

Adding a provider

1. Rule module under src/rules/ (kind, fromAddress, subject, source).
2. Parser under src/parsers/; register in parse-matched-message.js.
3. Fixture + tests under test/.
4. Map into spendings in pocketbase/map.js (source, description, details, parsed).