feat: add gmail-hook monorepo with watcher, api, and web

Ship the receipt pipeline (Livin/Grab/BRI), PocketBase spendings, PM2 ecosystem, and a proper .gitignore that excludes secrets, SQLite data, and media dumps.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
2026-09-13 22:58:47 +07:00
co-authored by Cursor
parent 8fa9f4350e
commit 46a2e2a7b0
67 changed files with 12153 additions and 0 deletions
+35
View File
@@ -0,0 +1,35 @@
# Google OAuth Web client (gmail.readonly)
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=
# Must match an Authorized redirect URI in Google Cloud Console.
# Local (Vite proxies /callback to Fastify):
GOOGLE_REDIRECT_URI=http://localhost:5173/callback
# Production:
# GOOGLE_REDIRECT_URI=https://oauth.0dev.web.id/callback
# 32+ random characters. Used to derive the session cookie key.
SESSION_SECRET=change-me-to-a-long-random-string
PORT=3000
NODE_ENV=development
# Watcher (apps/watcher) — Gmail Pub/Sub push
WATCHER_PORT=3001
# Full topic name, e.g. projects/my-gcp-project/topics/gmail-push
# A short name like gmail-push is expanded if GOOGLE_CLOUD_PROJECT is set.
GOOGLE_PUBSUB_TOPIC=
# GOOGLE_CLOUD_PROJECT=
# Shared secret. Push URL: https://host/pubsub/gmail?token=...
PUBSUB_VERIFICATION_TOKEN=
# Defaults to apps/api/data/app.db (same OAuth tokens as the web app)
# SQLITE_PATH=
# PocketBase (watcher writes parsed receipts to collection `spendings`)
# Use a _superusers account (admin), not a users-collection login.
POCKETBASE_URL=
POCKETBASE_USER=
POCKETBASE_PASSWORD=
# ntfy alerts for parse failures / non-duplicate skips (default: system topic)
# NTFY_URL=https://n.0dev.web.id/system
+47
View File
@@ -0,0 +1,47 @@
# Dependencies
node_modules/
# Build output
dist/
build/
*.tsbuildinfo
# Env / secrets (keep example)
.env
.env.*
!.env.example
# Local SQLite / runtime data
apps/api/data/
*.db
*.db-wal
*.db-shm
# Logs
*.log
npm-debug.log*
pnpm-debug.log*
yarn-debug.log*
yarn-error.log*
# OS / editor
.DS_Store
Thumbs.db
.idea/
.vscode/
*.swp
*.swo
*~
# Test / coverage
coverage/
.nyc_output/
# Accidental media dumps
*.mp4
*.mov
*.webm
*.mkv
# PM2 local dumps (process list lives on the host)
.pm2/
+102
View File
@@ -0,0 +1,102 @@
# Agent notes — gmail-hook
npm workspaces: `apps/api`, `apps/web`, `apps/watcher`. Goal is **e-receipt email → parsed spending → PocketBase**, not a generic Gmail client.
## Layout
| App | Role |
|---|---|
| `apps/api` | Google OAuth, session cookie, Gmail read API. Tokens in SQLite `apps/api/data/app.db`. |
| `apps/web` | React login / inbox UI (Vite). |
| `apps/watcher` | Gmail Pub/Sub push consumer. Matches receipts, parses JSON, writes `spendings`. |
Shared MIME helpers live in each app (`mime.js`). Watcher reuses the same SQLite tokens as the API.
## Watcher pipeline
`POST /pubsub/gmail` → `history.list` (`messageAdded`) → per message:
1. `logEvent` + `fetchAndLogMessage` — fetch outer mail; collect `emlMessages` if any
2. **Work items:** if `.eml` / `message/rfc822` attachments exist → process **each attachment only** (outer is a wrapper). If none → process the outer message as usual.
3. For each work item (sequential, in-process — no job queue): `matchRule` → `parseMatchedMessage` → `saveSpending`
Handlers return `{ pass: {...} }` to merge into ctx or `{ break: true }` to stop. **Parse full `message.body`** (and `message.html` when needed), never the 500-char log `bodyPreview`.
Attached `.eml` mails use nested From/Subject from Gmail’s expanded `message/rfc822` part. When nested HTML/text is not inlined (`body.data` missing, only `attachmentId`), the watcher fetches those attachments (`hydrateRfc822Message`) before parse. Synthetic `message_id` looks like `outerId#partId`; dedupe remains on `invoice_id`.
## Match rules
Resolve original sender:
1. If body has `---------- Forwarded message ---------`, use its From / Subject / Date.
2. Else use Gmail envelope.
| kind | From address | Subject |
|---|---|---|
| `livin.qr_payment.success` | `noreply.livin@bankmandiri.co.id` | `Pembayaran Berhasil!` |
| `grab.ereceipt.ride` | `no-reply@grab.com` | `Your Grab E-Receipt` |
| `grab.ereceipt.tip` | same From/Subject as ride; detected from tip body copy | tip invoice_id = `{Booking ID}:tip` |
| `grab.ereceipt.food` | same From/Subject; body has GrabFood / `Selamat menikmati makanan` | `Pesanan ID` as invoice_id |
| `bri.transfer.success` | `bankbri@bri.co.id` | `Pemindahan Dana Sesama Rekening BRI` |
Production path is **direct provider mail**. Forwards still work via the Fwd header. Do **not** match on the forwarder envelope From.
Rules: `apps/watcher/src/rules/`. Parsers: `apps/watcher/src/parsers/`.
## Parsing notes
- **Required for a spending row:** `amount` + `reference` (invoice id). Fail soft (log + break) if missing — never throw (Pub/Sub must ack).
- **Livin:** support clean forwarded lines and jammed HTML-stripped labels (`Tanggal13 Sep 2026`, `Tanggal15 Agu 2026`). Indonesian month names/abbreviations (`Agu`/`Agustus`, `Des`, …) are accepted. When merchant and city are glued on one line, keep the whole string as `merchant`; only split location when it is already a separate line ending in `- ID`. Do not maintain a city-name list.
- **Grab:** plain text often has date-only `Picked up on …`; pickup/dropoff times (`10:47AM`) live in HTML. Watcher keeps `message.html` and merges the first standalone AM/PM time into `occurredAt` (WIB). Tip receipts (`Your tip goes a long way…`, timestamp like `13 Sep 26 10:47 +0700`) become `grab.ereceipt.tip` with `invoice_id` `{bookingId}:tip` so they do not collide with the ride.
- **BRI / BRImo:** `Pemindahan Dana Sesama Rekening BRI` — amount from `Nominal` (not `Total`), `invoice_id` from `Nomor Referensi`, `description` from `Nama Tujuan`, datetime from combined `Tanggal` (`13 September 2026 , 19:00:00 WIB`). Other BRI subjects are out of scope until samples appear.
- Open-ended text (names, products, driver, compliments) is best-effort optional `details`.
## PocketBase
Env: `POCKETBASE_URL`, `POCKETBASE_USER`, `POCKETBASE_PASSWORD` (a `_superusers` account). Collection: `spendings`.
- Unique `invoice_id` — duplicate insert → log skip, no throw.
- Leave `category` empty; user fills it in PocketBase admin.
- Store flat columns for querying; `details` for kind leftovers; `parsed` for the full watcher envelope JSON (audit snapshot). Keep `message_id` so the original Gmail message can be re-fetched if needed — do not store raw email body by default.
- Schema ensure runs on watcher boot (`ensureSpendingsSchema`).
- Never commit secrets.
Parse failures and non-duplicate skips (incomplete record, save errors) POST to ntfy (`NTFY_URL`, default `https://n.0dev.web.id/system`) with `messageId` and message body. Duplicates do not notify.
## Run
```bash
npm install
npm run dev # api + web
# sign in once via UI so SQLite has tokens
npm run dev:watcher
npm test -w watcher # unit tests (no Gmail/Pub/Sub)
```
### Manual / backfill (forwarded old receipts)
Gmail Date on a forward is “today”; `trx_date` still comes from the receipt body. Duplicates skip on unique `invoice_id`. Forward-as-attachment (multiple `*.eml`) is expanded the same way as the watcher.
```bash
# List id / from / subject (from Gmail API)
npm run list-messages --
npm run list-messages -- --query 'newer_than:14d has:attachment filename:eml' --max 100
# Dry-run recent Livin/Grab subjects (default query: newer_than:7d)
npm run process-messages -- --dry-run
# Process a wrapper mail that has .eml attachments
npm run process-messages -- --ids OUTER_MESSAGE_ID
# Process and save
npm run process-messages -- --query 'newer_than:2d subject:"Pembayaran Berhasil!"'
npm run process-messages -- --ids MESSAGE_ID_1,MESSAGE_ID_2
npm run process-messages -- --user eleven16th@gmail.com --max 100
```
## Adding a provider
1. Rule module under `src/rules/` (kind, fromAddress, subject, source).
2. Parser under `src/parsers/`; register in `parse-matched-message.js`.
3. Fixture + tests under `test/`.
4. Map into spendings in `pocketbase/map.js` (`source`, `description`, `details`, `parsed`).
+17
View File
@@ -0,0 +1,17 @@
{
"name": "api",
"private": true,
"type": "module",
"scripts": {
"dev": "node --watch src/index.js",
"start": "node src/index.js"
},
"dependencies": {
"@fastify/secure-session": "^8.2.0",
"@fastify/static": "^8.2.0",
"better-sqlite3": "^12.2.0",
"dotenv": "^17.2.2",
"fastify": "^5.6.0",
"googleapis": "^159.0.0"
}
}
+38
View File
@@ -0,0 +1,38 @@
import { config } from "dotenv";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const __dirname = dirname(fileURLToPath(import.meta.url));
export const rootDir = resolve(__dirname, "../../..");
export const apiDir = resolve(__dirname, "..");
export const webDistDir = resolve(__dirname, "../../web/dist");
config({ path: resolve(rootDir, ".env"), quiet: true });
export const PORT = Number(process.env.PORT || 3000);
export const NODE_ENV = process.env.NODE_ENV || "development";
export const GOOGLE_CLIENT_ID = process.env.GOOGLE_CLIENT_ID || "";
export const GOOGLE_CLIENT_SECRET = process.env.GOOGLE_CLIENT_SECRET || "";
export const GOOGLE_REDIRECT_URI =
process.env.GOOGLE_REDIRECT_URI || "http://localhost:5173/callback";
export const SESSION_SECRET = process.env.SESSION_SECRET || "";
export const GMAIL_SCOPE = "https://www.googleapis.com/auth/gmail.readonly";
export const OAUTH_SCOPES = [
GMAIL_SCOPE,
"openid",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/userinfo.profile",
];
export function assertConfig() {
const missing = [];
if (!GOOGLE_CLIENT_ID) missing.push("GOOGLE_CLIENT_ID");
if (!GOOGLE_CLIENT_SECRET) missing.push("GOOGLE_CLIENT_SECRET");
if (!SESSION_SECRET || SESSION_SECRET.length < 16) {
missing.push("SESSION_SECRET (16+ characters)");
}
if (missing.length) {
throw new Error(`Missing required env: ${missing.join(", ")}`);
}
}
+82
View File
@@ -0,0 +1,82 @@
import { mkdirSync } from "node:fs";
import { resolve } from "node:path";
import Database from "better-sqlite3";
import { apiDir } from "./config.js";
const dataDir = resolve(apiDir, "data");
mkdirSync(dataDir, { recursive: true });
const db = new Database(resolve(dataDir, "app.db"));
db.pragma("journal_mode = WAL");
db.pragma("foreign_keys = ON");
db.exec(`
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
google_sub TEXT NOT NULL UNIQUE,
email TEXT NOT NULL,
name TEXT,
picture TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS oauth_tokens (
user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
access_token TEXT NOT NULL,
refresh_token TEXT,
expiry INTEGER NOT NULL
);
`);
const upsertUserStmt = db.prepare(`
INSERT INTO users (google_sub, email, name, picture)
VALUES (@googleSub, @email, @name, @picture)
ON CONFLICT(google_sub) DO UPDATE SET
email = excluded.email,
name = excluded.name,
picture = excluded.picture
RETURNING *
`);
const saveTokensStmt = db.prepare(`
INSERT INTO oauth_tokens (user_id, access_token, refresh_token, expiry)
VALUES (@userId, @accessToken, @refreshToken, @expiry)
ON CONFLICT(user_id) DO UPDATE SET
access_token = excluded.access_token,
refresh_token = COALESCE(excluded.refresh_token, oauth_tokens.refresh_token),
expiry = excluded.expiry
`);
const getUserByIdStmt = db.prepare(`SELECT * FROM users WHERE id = ?`);
const getTokensStmt = db.prepare(`SELECT * FROM oauth_tokens WHERE user_id = ?`);
export function upsertUser({ googleSub, email, name, picture }) {
return upsertUserStmt.get({ googleSub, email, name, picture });
}
export function saveTokens(userId, { accessToken, refreshToken, expiry }) {
saveTokensStmt.run({
userId,
accessToken,
refreshToken: refreshToken || null,
expiry,
});
}
export function getUserById(id) {
return getUserByIdStmt.get(id) || null;
}
export function getTokens(userId) {
return getTokensStmt.get(userId) || null;
}
export function publicUser(row) {
if (!row) return null;
return {
id: row.id,
email: row.email,
name: row.name,
picture: row.picture,
};
}
+89
View File
@@ -0,0 +1,89 @@
import { google } from "googleapis";
import {
GOOGLE_CLIENT_ID,
GOOGLE_CLIENT_SECRET,
GOOGLE_REDIRECT_URI,
OAUTH_SCOPES,
} from "./config.js";
import { getTokens, saveTokens } from "./db.js";
import { HttpError } from "./http.js";
export function createOAuthClient() {
return new google.auth.OAuth2(
GOOGLE_CLIENT_ID,
GOOGLE_CLIENT_SECRET,
GOOGLE_REDIRECT_URI,
);
}
export function authUrl(state) {
return createOAuthClient().generateAuthUrl({
access_type: "offline",
prompt: "consent",
include_granted_scopes: true,
scope: OAUTH_SCOPES,
state,
});
}
export async function exchangeCode(code) {
const client = createOAuthClient();
const { tokens } = await client.getToken(code);
client.setCredentials(tokens);
const oauth2 = google.oauth2({ version: "v2", auth: client });
const { data: profile } = await oauth2.userinfo.get();
if (!profile.id || !profile.email) {
throw new HttpError(400, "Google profile did not include email");
}
return {
profile: {
googleSub: profile.id,
email: profile.email,
name: profile.name || profile.email,
picture: profile.picture || null,
},
tokens: {
accessToken: tokens.access_token,
refreshToken: tokens.refresh_token || null,
expiry: tokens.expiry_date || Date.now() + 3600 * 1000,
},
};
}
function persistClientTokens(userId, client) {
client.on("tokens", (tokens) => {
if (!tokens.access_token) return;
saveTokens(userId, {
accessToken: tokens.access_token,
refreshToken: tokens.refresh_token || null,
expiry: tokens.expiry_date || Date.now() + 3600 * 1000,
});
});
}
export async function gmailClientForUser(userId) {
const stored = getTokens(userId);
if (!stored?.access_token && !stored?.refresh_token) {
throw new HttpError(401, "Google account is not connected");
}
const client = createOAuthClient();
client.setCredentials({
access_token: stored.access_token,
refresh_token: stored.refresh_token,
expiry_date: stored.expiry,
});
persistClientTokens(userId, client);
try {
await client.getAccessToken();
} catch {
throw new HttpError(
401,
"Gmail access expired or was revoked. Sign in again.",
);
}
return google.gmail({ version: "v1", auth: client });
}
+14
View File
@@ -0,0 +1,14 @@
export class HttpError extends Error {
constructor(statusCode, message) {
super(message);
this.statusCode = statusCode;
}
}
export function requireUser(request) {
const userId = request.session.get("userId");
if (!userId) {
throw new HttpError(401, "Not authenticated");
}
return userId;
}
+84
View File
@@ -0,0 +1,84 @@
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import Fastify from "fastify";
import fastifySecureSession from "@fastify/secure-session";
import fastifyStatic from "@fastify/static";
import {
NODE_ENV,
PORT,
SESSION_SECRET,
assertConfig,
webDistDir,
} from "./config.js";
import { HttpError } from "./http.js";
import { registerAuthRoutes } from "./routes/auth.js";
import { registerMessageRoutes } from "./routes/messages.js";
assertConfig();
const app = Fastify({
logger: true,
trustProxy: true,
});
await app.register(fastifySecureSession, {
key: createHash("sha256").update(SESSION_SECRET).digest(),
cookieName: "session",
expiry: 60 * 60 * 24 * 7,
cookie: {
path: "/",
httpOnly: true,
sameSite: "lax",
secure: NODE_ENV === "production",
},
});
app.get("/api/health", async () => ({ ok: true }));
await registerAuthRoutes(app);
await registerMessageRoutes(app);
app.setErrorHandler((err, request, reply) => {
const status = err.statusCode || 500;
if (status >= 500) {
request.log.error(err);
}
const message =
err instanceof HttpError || status < 500
? err.message
: "Internal server error";
reply.code(status).send({ error: message });
});
const serveFrontend = existsSync(webDistDir);
if (serveFrontend) {
await app.register(fastifyStatic, {
root: webDistDir,
wildcard: false,
});
app.setNotFoundHandler((request, reply) => {
if (
request.method !== "GET" ||
request.url.startsWith("/api") ||
request.url.startsWith("/auth") ||
request.url.startsWith("/callback")
) {
reply.code(404).send({ error: "Not found" });
return;
}
return reply.sendFile("index.html");
});
}
try {
await app.listen({ port: PORT, host: "0.0.0.0" });
app.log.info(
`API listening on http://localhost:${PORT}` +
(serveFrontend ? " (serving web dist)" : " (API only; use Vite in dev)"),
);
} catch (err) {
app.log.error(err);
process.exit(1);
}
+69
View File
@@ -0,0 +1,69 @@
function decodeBase64Url(data) {
if (!data) return "";
const pad = data.length % 4 === 0 ? "" : "=".repeat(4 - (data.length % 4));
return Buffer.from(
data.replace(/-/g, "+").replace(/_/g, "/") + pad,
"base64",
).toString("utf8");
}
function stripHtml(html) {
return html
.replace(/<style[\s\S]*?<\/style>/gi, "")
.replace(/<script[\s\S]*?<\/script>/gi, "")
.replace(/<br\s*\/?>/gi, "\n")
.replace(/<\/p>/gi, "\n\n")
.replace(/<\/div>/gi, "\n")
.replace(/<\/tr>/gi, "\n")
.replace(/<[^>]+>/g, "")
.replace(/&nbsp;/g, " ")
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x27;/gi, "'")
.replace(/\n{3,}/g, "\n\n")
.trim();
}
function walkParts(payload, acc = { text: "", html: "" }) {
if (!payload) return acc;
const mime = payload.mimeType || "";
const data = payload.body?.data;
if (data) {
if (mime === "text/plain") acc.text += decodeBase64Url(data);
else if (mime === "text/html") acc.html += decodeBase64Url(data);
}
for (const part of payload.parts || []) {
walkParts(part, acc);
}
return acc;
}
export function extractBody(payload) {
const { text, html } = walkParts(payload);
if (text.trim()) return text.trim();
if (html.trim()) return stripHtml(html);
return "";
}
export function header(payload, name) {
const match = payload?.headers?.find(
(h) => h.name?.toLowerCase() === name.toLowerCase(),
);
return match?.value || "";
}
export function summarizeMessage(message) {
const payload = message.payload || {};
return {
id: message.id,
threadId: message.threadId,
from: header(payload, "From"),
to: header(payload, "To"),
subject: header(payload, "Subject") || "(no subject)",
date: header(payload, "Date"),
snippet: message.snippet || "",
};
}
+53
View File
@@ -0,0 +1,53 @@
import { randomBytes } from "node:crypto";
import { getUserById, publicUser, saveTokens, upsertUser } from "../db.js";
import { HttpError, requireUser } from "../http.js";
import { authUrl, exchangeCode } from "../google.js";
export async function registerAuthRoutes(app) {
app.get("/auth/google", async (request, reply) => {
const state = randomBytes(16).toString("hex");
request.session.set("oauthState", state);
return reply.redirect(authUrl(state));
});
app.get("/callback", async (request, reply) => {
const { code, state, error } = request.query;
if (error) {
request.log.warn({ error }, "Google OAuth error");
return reply.redirect("/login?error=denied");
}
if (!code || typeof code !== "string") {
return reply.redirect("/login?error=missing_code");
}
const expected = request.session.get("oauthState");
if (!expected || state !== expected) {
return reply.redirect("/login?error=invalid_state");
}
request.session.set("oauthState", undefined);
try {
const { profile, tokens } = await exchangeCode(code);
const user = upsertUser(profile);
saveTokens(user.id, tokens);
request.session.set("userId", user.id);
return reply.redirect("/");
} catch (err) {
request.log.error(err);
return reply.redirect("/login?error=exchange_failed");
}
});
app.post("/auth/logout", async (request, reply) => {
request.session.delete();
return reply.send({ ok: true });
});
app.get("/api/me", async (request) => {
const userId = requireUser(request);
const user = getUserById(userId);
if (!user) {
throw new HttpError(401, "Not authenticated");
}
return publicUser(user);
});
}
+92
View File
@@ -0,0 +1,92 @@
import { gmailClientForUser } from "../google.js";
import { HttpError, requireUser } from "../http.js";
import { extractBody, summarizeMessage } from "../mime.js";
function fromGoogleError(err) {
const status = Number(err.code || err.response?.status || err.statusCode);
if (status === 401 || status === 403) {
return new HttpError(
401,
"Gmail access expired or was revoked. Sign in again.",
);
}
if (status === 404) {
return new HttpError(404, "Message not found");
}
return err;
}
export async function registerMessageRoutes(app) {
app.get("/api/messages", async (request) => {
const userId = requireUser(request);
const gmail = await gmailClientForUser(userId);
const q = typeof request.query.q === "string" ? request.query.q : "";
const pageToken =
typeof request.query.pageToken === "string"
? request.query.pageToken
: undefined;
const maxResults = Math.min(
50,
Math.max(1, Number(request.query.maxResults) || 20),
);
let list;
try {
list = await gmail.users.messages.list({
userId: "me",
q: q || undefined,
maxResults,
pageToken,
});
} catch (err) {
throw fromGoogleError(err);
}
const refs = list.data.messages || [];
const messages = [];
const settled = await Promise.allSettled(
refs.map((ref) =>
gmail.users.messages.get({
userId: "me",
id: ref.id,
format: "metadata",
metadataHeaders: ["From", "To", "Subject", "Date"],
}),
),
);
for (const result of settled) {
if (result.status === "fulfilled") {
messages.push(summarizeMessage(result.value.data));
}
}
return {
messages,
nextPageToken: list.data.nextPageToken || null,
resultSizeEstimate: list.data.resultSizeEstimate || 0,
};
});
app.get("/api/messages/:id", async (request) => {
const userId = requireUser(request);
const gmail = await gmailClientForUser(userId);
const { id } = request.params;
let data;
try {
({ data } = await gmail.users.messages.get({
userId: "me",
id,
format: "full",
}));
} catch (err) {
throw fromGoogleError(err);
}
return {
...summarizeMessage(data),
body: extractBody(data.payload),
};
});
}
+28
View File
@@ -0,0 +1,28 @@
/**
* PM2 ecosystem for the Gmail receipt watcher.
*
* Loads secrets from the monorepo root `.env` via dotenv in src/config.js.
* Override port with WATCHER_PORT (default 3001; production on rpi4 uses 8083).
*
* pm2 start ecosystem.config.cjs
* pm2 save
*/
module.exports = {
apps: [
{
name: "gmail-watcher",
cwd: __dirname,
script: "src/index.js",
interpreter: "node",
instances: 1,
exec_mode: "fork",
autorestart: true,
max_restarts: 10,
restart_delay: 2000,
env: {
NODE_ENV: "production",
WATCHER_PORT: "8083",
},
},
],
};
+19
View File
@@ -0,0 +1,19 @@
{
"name": "watcher",
"private": true,
"type": "module",
"scripts": {
"dev": "node --watch src/index.js",
"start": "node src/index.js",
"test": "node --test test/**/*.test.js",
"process-messages": "node scripts/process-messages.js",
"list-messages": "node scripts/list-messages.js"
},
"dependencies": {
"better-sqlite3": "^12.2.0",
"dotenv": "^17.2.2",
"fastify": "^5.6.0",
"googleapis": "^159.0.0",
"node-html-parser": "^9.0.4"
}
}
+1414
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -0,0 +1,2 @@
allowBuilds:
better-sqlite3: true
+140
View File
@@ -0,0 +1,140 @@
#!/usr/bin/env node
/**
* List Gmail messages: id, from, subject (metadata only).
*
* Usage:
* npm run list-messages -w watcher
* npm run list-messages -w watcher -- --query 'newer_than:14d has:attachment filename:eml'
* npm run list-messages -w watcher -- --max 100 --json
*
* Options:
* --query <q> Gmail search (default: newer_than:7d)
* --user <email> Mailbox (default: first user with tokens)
* --max <n> Max messages (default 50)
* --json Print JSON lines instead of a table
*/
import { GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET } from "../src/config.js";
import { getUserByEmail, listUsersWithTokens } from "../src/db.js";
import { gmailClientForUser } from "../src/google.js";
import { header } from "../src/mime.js";
const DEFAULT_QUERY = "newer_than:7d";
function parseArgs(argv) {
const out = {
query: null,
user: null,
max: 50,
json: false,
help: false,
};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === "--help" || a === "-h") out.help = true;
else if (a === "--json") out.json = true;
else if (a === "--query") out.query = argv[++i];
else if (a === "--user") out.user = argv[++i];
else if (a === "--max") out.max = Math.max(1, Number(argv[++i]) || 50);
else if (a.startsWith("--query=")) out.query = a.slice("--query=".length);
else if (a.startsWith("--user=")) out.user = a.slice("--user=".length);
else if (a.startsWith("--max=")) {
out.max = Math.max(1, Number(a.slice("--max=".length)) || 50);
}
}
return out;
}
async function listMessageIds(gmail, query, max) {
const ids = [];
let pageToken;
while (ids.length < max) {
const { data } = await gmail.users.messages.list({
userId: "me",
q: query,
maxResults: Math.min(50, max - ids.length),
pageToken,
});
for (const m of data.messages || []) {
if (m.id) ids.push(m.id);
if (ids.length >= max) break;
}
pageToken = data.nextPageToken;
if (!pageToken) break;
}
return ids;
}
function metaFromPayload(payload) {
return {
from: header(payload, "From") || "",
subject: header(payload, "Subject") || "(no subject)",
date: header(payload, "Date") || "",
};
}
async function main() {
const args = parseArgs(process.argv.slice(2));
if (args.help) {
console.log(`Usage: list-messages [--query q] [--user email] [--max n] [--json]
Default query: ${DEFAULT_QUERY}`);
process.exit(0);
}
if (!GOOGLE_CLIENT_ID || !GOOGLE_CLIENT_SECRET) {
console.error("Missing GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET in .env");
process.exit(1);
}
const users = listUsersWithTokens();
if (!users.length) {
console.error("No signed-in users with tokens. Sign in via the web UI first.");
process.exit(1);
}
let user = users[0];
if (args.user) {
const byEmail = getUserByEmail(args.user);
const byId = users.find((u) => String(u.id) === String(args.user));
user = byEmail || byId || null;
if (!user) {
console.error(`User not found: ${args.user}`);
process.exit(1);
}
}
const gmail = await gmailClientForUser(user.id);
const query = args.query || DEFAULT_QUERY;
const ids = await listMessageIds(gmail, query, args.max);
if (!args.json) {
console.error(`# mailbox=${user.email} query=${JSON.stringify(query)} count=${ids.length}`);
}
for (const id of ids) {
const { data } = await gmail.users.messages.get({
userId: "me",
id,
format: "metadata",
metadataHeaders: ["From", "Subject", "Date"],
});
const meta = metaFromPayload(data.payload || {});
const row = {
id,
from: meta.from,
subject: meta.subject,
date: meta.date,
};
if (args.json) {
console.log(JSON.stringify(row));
} else {
console.log(`${row.id}\t${row.from}\t${row.subject}`);
}
}
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
+243
View File
@@ -0,0 +1,243 @@
#!/usr/bin/env node
/**
* Manually process Gmail messages through match → parse → PocketBase save.
*
* Use when forwarding old receipts: Gmail Date may be "today", but trx_date
* comes from the receipt body (Tanggal/Jam, Picked up on, …). Existing
* invoice_id rows are skipped (unique constraint).
*
* Also expands .eml / message/rfc822 attachments (forward-as-attachment).
*
* Usage:
* npm run process-messages -w watcher -- --query 'newer_than:2d subject:"Pembayaran Berhasil!"'
* npm run process-messages -w watcher -- --ids 1a099bd61efd234d,1a099b8b9d239e5e
* npm run process-messages -w watcher -- --dry-run --max 20
*
* Options:
* --query <q> Gmail search (default targets Livin/Grab subjects, newer_than:7d)
* --ids <a,b> Explicit message IDs (skips list search)
* --user <email> Mailbox to use (default: first user with tokens)
* --max <n> Max messages to process (default 50)
* --dry-run Parse and log only; do not write PocketBase
*/
import {
GOOGLE_CLIENT_ID,
GOOGLE_CLIENT_SECRET,
POCKETBASE_URL,
} from "../src/config.js";
import { getUserByEmail, listUsersWithTokens } from "../src/db.js";
import { gmailClientForUser } from "../src/google.js";
import { fetchAndLogMessage } from "../src/handlers/fetch-and-log-message.js";
import { matchRule } from "../src/handlers/match-rule.js";
import { parseMatchedMessage } from "../src/handlers/parse-matched-message.js";
import { saveSpending } from "../src/handlers/save-spending.js";
import { runPipeline } from "../src/pipeline.js";
import { processFetchedMessage } from "../src/process-fetched.js";
import { pocketbaseConfigured } from "../src/pocketbase/client.js";
const DEFAULT_QUERY =
'newer_than:7d (subject:"Pembayaran Berhasil!" OR subject:"Your Grab E-Receipt" OR subject:"Pemindahan Dana Sesama Rekening BRI")';
function parseArgs(argv) {
const out = {
query: null,
ids: [],
user: null,
max: 50,
dryRun: false,
help: false,
};
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === "--help" || a === "-h") out.help = true;
else if (a === "--dry-run") out.dryRun = true;
else if (a === "--query") out.query = argv[++i];
else if (a === "--ids") {
out.ids = String(argv[++i] || "")
.split(",")
.map((s) => s.trim())
.filter(Boolean);
} else if (a === "--user") out.user = argv[++i];
else if (a === "--max") out.max = Math.max(1, Number(argv[++i]) || 50);
else if (a.startsWith("--query=")) out.query = a.slice("--query=".length);
else if (a.startsWith("--ids=")) {
out.ids = a
.slice("--ids=".length)
.split(",")
.map((s) => s.trim())
.filter(Boolean);
} else if (a.startsWith("--user=")) out.user = a.slice("--user=".length);
else if (a.startsWith("--max=")) {
out.max = Math.max(1, Number(a.slice("--max=".length)) || 50);
}
}
return out;
}
function makeLog() {
const line = (level, obj, msg) => {
const payload =
typeof obj === "string" ? { msg: obj } : { ...(obj || {}), msg };
const { msg: message, ...rest } = payload;
console.log(
JSON.stringify({
level,
time: new Date().toISOString(),
...rest,
msg: message || msg,
}),
);
};
return {
info: (obj, msg) => line("info", obj, msg),
warn: (obj, msg) => line("warn", obj, msg),
error: (obj, msg) => line("error", obj, msg),
};
}
async function listMessageIds(gmail, query, max) {
const ids = [];
let pageToken;
while (ids.length < max) {
const { data } = await gmail.users.messages.list({
userId: "me",
q: query,
maxResults: Math.min(50, max - ids.length),
pageToken,
});
for (const m of data.messages || []) {
if (m.id) ids.push(m.id);
if (ids.length >= max) break;
}
pageToken = data.nextPageToken;
if (!pageToken) break;
}
return ids;
}
async function main() {
const args = parseArgs(process.argv.slice(2));
if (args.help) {
console.log(`Usage: process-messages [--query q] [--ids id1,id2] [--user email] [--max n] [--dry-run]
Default query: ${DEFAULT_QUERY}`);
process.exit(0);
}
if (!GOOGLE_CLIENT_ID || !GOOGLE_CLIENT_SECRET) {
console.error("Missing GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET in .env");
process.exit(1);
}
const users = listUsersWithTokens();
if (!users.length) {
console.error("No signed-in users with tokens. Sign in via the web UI first.");
process.exit(1);
}
let user = users[0];
if (args.user) {
const byEmail = getUserByEmail(args.user);
const byId = users.find((u) => String(u.id) === String(args.user));
user = byEmail || byId || null;
if (!user) {
console.error(`User not found: ${args.user}`);
process.exit(1);
}
}
const log = makeLog();
const gmail = await gmailClientForUser(user.id);
let messageIds = args.ids;
if (!messageIds.length) {
const query = args.query || DEFAULT_QUERY;
log.info({ query, max: args.max, user: user.email }, "process-messages: listing");
messageIds = await listMessageIds(gmail, query, args.max);
} else {
messageIds = messageIds.slice(0, args.max);
}
log.info(
{
count: messageIds.length,
user: user.email,
dryRun: args.dryRun,
pocketbase: pocketbaseConfigured() ? POCKETBASE_URL : null,
},
"process-messages: start",
);
const fetchHandlers = [fetchAndLogMessage];
const processHandlers = args.dryRun
? [matchRule, parseMatchedMessage]
: [matchRule, parseMatchedMessage, saveSpending];
const summary = {
processed: 0,
matched: 0,
saved: 0,
duplicate: 0,
skipped: 0,
errors: 0,
};
for (const messageId of messageIds) {
try {
const fetched = await runPipeline(fetchHandlers, {
notification: { emailAddress: user.email, historyId: null },
userId: user.id,
messageId,
gmail,
log,
});
if (!fetched.message) {
summary.skipped++;
continue;
}
const results = await processFetchedMessage(fetched, processHandlers);
for (const result of results) {
summary.processed++;
if (!result.rule) {
summary.skipped++;
continue;
}
summary.matched++;
if (args.dryRun) {
log.info(
{
messageId: result.messageId,
kind: result.parsed?.kind,
invoice_id: result.parsed?.reference,
amount: result.parsed?.amount,
occurredAt: result.parsed?.occurredAt,
},
"process-messages: dry-run parsed",
);
continue;
}
if (result.saved) summary.saved++;
else if (result.duplicate) summary.duplicate++;
else if (result.error) summary.errors++;
else summary.skipped++;
}
} catch (err) {
summary.errors++;
log.error(
{ messageId, err: err.message || String(err) },
"process-messages: failed",
);
}
}
log.info(summary, "process-messages: done");
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
+73
View File
@@ -0,0 +1,73 @@
import { config } from "dotenv";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const __dirname = dirname(fileURLToPath(import.meta.url));
export const rootDir = resolve(__dirname, "../../..");
export const watcherDir = resolve(__dirname, "..");
export const defaultSqlitePath = resolve(__dirname, "../../api/data/app.db");
config({ path: resolve(rootDir, ".env"), quiet: true });
export const PORT = Number(process.env.WATCHER_PORT || 3001);
export const NODE_ENV = process.env.NODE_ENV || "development";
export const GOOGLE_CLIENT_ID = process.env.GOOGLE_CLIENT_ID || "";
export const GOOGLE_CLIENT_SECRET = process.env.GOOGLE_CLIENT_SECRET || "";
export const GOOGLE_REDIRECT_URI =
process.env.GOOGLE_REDIRECT_URI || "http://localhost:5173/callback";
export const GOOGLE_CLOUD_PROJECT =
process.env.GOOGLE_CLOUD_PROJECT || process.env.GCLOUD_PROJECT || "";
export const PUBSUB_VERIFICATION_TOKEN =
process.env.PUBSUB_VERIFICATION_TOKEN || "";
export const SQLITE_PATH = process.env.SQLITE_PATH || defaultSqlitePath;
export const POCKETBASE_URL = (process.env.POCKETBASE_URL || "").replace(
/\/$/,
"",
);
export const POCKETBASE_USER = process.env.POCKETBASE_USER || "";
export const POCKETBASE_PASSWORD = process.env.POCKETBASE_PASSWORD || "";
/** ntfy topic URL, e.g. https://n.0dev.web.id/system */
export const NTFY_URL = (process.env.NTFY_URL || "https://n.0dev.web.id/system").replace(
/\/$/,
"",
);
export const WATCH_RENEW_MS = 12 * 60 * 60 * 1000;
const PUBSUB_TOPIC_RE = /^projects\/[^/]+\/topics\/[^/]+$/;
export function resolvePubsubTopic(
topic,
{ projectId = GOOGLE_CLOUD_PROJECT } = {},
) {
const trimmed = (topic || "").trim();
if (!trimmed) return "";
if (PUBSUB_TOPIC_RE.test(trimmed)) return trimmed;
if (trimmed.includes("/")) return trimmed;
if (!projectId) return trimmed;
return `projects/${projectId}/topics/${trimmed}`;
}
export const GOOGLE_PUBSUB_TOPIC = resolvePubsubTopic(
process.env.GOOGLE_PUBSUB_TOPIC || "",
);
export function assertConfig() {
const missing = [];
if (!GOOGLE_CLIENT_ID) missing.push("GOOGLE_CLIENT_ID");
if (!GOOGLE_CLIENT_SECRET) missing.push("GOOGLE_CLIENT_SECRET");
if (!GOOGLE_PUBSUB_TOPIC) missing.push("GOOGLE_PUBSUB_TOPIC");
if (NODE_ENV === "production" && !PUBSUB_VERIFICATION_TOKEN) {
missing.push("PUBSUB_VERIFICATION_TOKEN");
}
if (missing.length) {
throw new Error(`Missing required env: ${missing.join(", ")}`);
}
if (!PUBSUB_TOPIC_RE.test(GOOGLE_PUBSUB_TOPIC)) {
throw new Error(
`GOOGLE_PUBSUB_TOPIC must be a full Pub/Sub name like projects/my-gcp-project/topics/gmail-push (got "${process.env.GOOGLE_PUBSUB_TOPIC || ""}")`,
);
}
}
+102
View File
@@ -0,0 +1,102 @@
import { mkdirSync } from "node:fs";
import { dirname } from "node:path";
import Database from "better-sqlite3";
import { SQLITE_PATH } from "./config.js";
mkdirSync(dirname(SQLITE_PATH), { recursive: true });
const db = new Database(SQLITE_PATH);
db.pragma("journal_mode = WAL");
db.pragma("foreign_keys = ON");
db.pragma("busy_timeout = 5000");
db.exec(`
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
google_sub TEXT NOT NULL UNIQUE,
email TEXT NOT NULL,
name TEXT,
picture TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS oauth_tokens (
user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
access_token TEXT NOT NULL,
refresh_token TEXT,
expiry INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS watch_state (
user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
history_id TEXT NOT NULL,
expiration INTEGER NOT NULL
);
`);
const getUserByIdStmt = db.prepare(`SELECT * FROM users WHERE id = ?`);
const getUserByEmailStmt = db.prepare(
`SELECT * FROM users WHERE LOWER(email) = LOWER(?)`,
);
const getTokensStmt = db.prepare(`SELECT * FROM oauth_tokens WHERE user_id = ?`);
const listUsersWithTokensStmt = db.prepare(`
SELECT u.*
FROM users u
INNER JOIN oauth_tokens t ON t.user_id = u.id
`);
const saveTokensStmt = db.prepare(`
INSERT INTO oauth_tokens (user_id, access_token, refresh_token, expiry)
VALUES (@userId, @accessToken, @refreshToken, @expiry)
ON CONFLICT(user_id) DO UPDATE SET
access_token = excluded.access_token,
refresh_token = COALESCE(excluded.refresh_token, oauth_tokens.refresh_token),
expiry = excluded.expiry
`);
const getWatchStateStmt = db.prepare(
`SELECT * FROM watch_state WHERE user_id = ?`,
);
const saveWatchStateStmt = db.prepare(`
INSERT INTO watch_state (user_id, history_id, expiration)
VALUES (@userId, @historyId, @expiration)
ON CONFLICT(user_id) DO UPDATE SET
history_id = excluded.history_id,
expiration = excluded.expiration
`);
export function getUserById(id) {
return getUserByIdStmt.get(id) || null;
}
export function getUserByEmail(email) {
if (!email) return null;
return getUserByEmailStmt.get(email) || null;
}
export function getTokens(userId) {
return getTokensStmt.get(userId) || null;
}
export function listUsersWithTokens() {
return listUsersWithTokensStmt.all();
}
export function saveTokens(userId, { accessToken, refreshToken, expiry }) {
saveTokensStmt.run({
userId,
accessToken,
refreshToken: refreshToken || null,
expiry,
});
}
export function getWatchState(userId) {
return getWatchStateStmt.get(userId) || null;
}
export function saveWatchState(userId, { historyId, expiration }) {
saveWatchStateStmt.run({
userId,
historyId: String(historyId),
expiration: Number(expiration),
});
}
+105
View File
@@ -0,0 +1,105 @@
import { google } from "googleapis";
import {
GOOGLE_CLIENT_ID,
GOOGLE_CLIENT_SECRET,
GOOGLE_PUBSUB_TOPIC,
GOOGLE_REDIRECT_URI,
} from "./config.js";
import { getTokens, getWatchState, saveTokens, saveWatchState } from "./db.js";
import { HttpError } from "./http.js";
export function createOAuthClient() {
return new google.auth.OAuth2(
GOOGLE_CLIENT_ID,
GOOGLE_CLIENT_SECRET,
GOOGLE_REDIRECT_URI,
);
}
function persistClientTokens(userId, client) {
client.on("tokens", (tokens) => {
if (!tokens.access_token) return;
saveTokens(userId, {
accessToken: tokens.access_token,
refreshToken: tokens.refresh_token || null,
expiry: tokens.expiry_date || Date.now() + 3600 * 1000,
});
});
}
export async function gmailClientForUser(userId) {
const stored = getTokens(userId);
if (!stored?.access_token && !stored?.refresh_token) {
throw new HttpError(401, "Google account is not connected");
}
const client = createOAuthClient();
client.setCredentials({
access_token: stored.access_token,
refresh_token: stored.refresh_token,
expiry_date: stored.expiry,
});
persistClientTokens(userId, client);
try {
await client.getAccessToken();
} catch {
throw new HttpError(
401,
"Gmail access expired or was revoked. Sign in again.",
);
}
return google.gmail({ version: "v1", auth: client });
}
export async function startWatch(userId, { resetCursor = false } = {}) {
const gmail = await gmailClientForUser(userId);
const { data } = await gmail.users.watch({
userId: "me",
requestBody: {
topicName: GOOGLE_PUBSUB_TOPIC,
labelIds: ["INBOX"],
},
});
const existing = getWatchState(userId);
const historyId =
resetCursor || !existing?.history_id
? String(data.historyId)
: existing.history_id;
saveWatchState(userId, {
historyId,
expiration: Number(data.expiration) || Date.now() + 7 * 24 * 3600 * 1000,
});
return { ...data, historyId };
}
export async function listHistory(gmail, startHistoryId) {
const messageIds = new Set();
let pageToken;
let latestHistoryId = String(startHistoryId);
do {
const { data } = await gmail.users.history.list({
userId: "me",
startHistoryId: String(startHistoryId),
pageToken,
historyTypes: ["messageAdded"],
});
if (data.historyId) latestHistoryId = String(data.historyId);
for (const record of data.history || []) {
for (const added of record.messagesAdded || []) {
const id = added.message?.id;
if (id) messageIds.add(id);
}
}
pageToken = data.nextPageToken;
} while (pageToken);
return { messageIds: [...messageIds], historyId: latestHistoryId };
}
export function googleStatus(err) {
return Number(err.code || err.response?.status || err.statusCode || 0);
}
@@ -0,0 +1,76 @@
import {
extractBody,
extractHtml,
hydrateRfc822Message,
listRfc822Parts,
summarizeMessage,
} from "../mime.js";
import { googleStatus } from "../google.js";
const BODY_PREVIEW = 500;
export async function fetchAndLogMessage(ctx) {
const { gmail, messageId, log } = ctx;
try {
const { data } = await gmail.users.messages.get({
userId: "me",
id: messageId,
format: "full",
});
const message = {
...summarizeMessage(data),
body: extractBody(data.payload),
html: extractHtml(data.payload),
};
const rfcParts = listRfc822Parts(data.payload);
const emlMessages = [];
for (let index = 0; index < rfcParts.length; index++) {
const part = rfcParts[index];
const hydrated = await hydrateRfc822Message(gmail, message.id, part, index);
emlMessages.push(hydrated);
}
const emptyEmls = emlMessages.filter((m) => !m.body && !m.html).length;
log.info(
{
messageId,
from: message.from,
subject: message.subject,
date: message.date,
snippet: message.snippet,
bodyPreview: (message.body || "").slice(0, BODY_PREVIEW),
emlCount: emlMessages.length,
emptyEmlCount: emptyEmls || undefined,
},
"gmail-watch: message",
);
if (emlMessages.length) {
log.info(
{
messageId,
emlCount: emlMessages.length,
filenames: emlMessages.map((m) => m.attachmentFilename),
},
"gmail-watch: eml attachments",
);
}
return {
pass: {
message,
emlMessages,
gmailPayload: data.payload,
},
};
} catch (err) {
const status = googleStatus(err);
if (status === 404) {
log.warn({ messageId }, "gmail-watch: message not found");
return { break: true };
}
throw err;
}
}
+12
View File
@@ -0,0 +1,12 @@
export async function logEvent(ctx) {
ctx.log.info(
{
emailAddress: ctx.notification?.emailAddress,
historyId: ctx.notification?.historyId,
messageId: ctx.messageId,
userId: ctx.userId,
},
"gmail-watch: event",
);
return { pass: { loggedAt: Date.now() } };
}
+58
View File
@@ -0,0 +1,58 @@
import { extractEmail, resolveOriginal, subjectMatches } from "../lib/email.js";
import { rules } from "../rules/index.js";
/**
* Match a fetched message against known receipt rules using original From + Subject.
* Breaks the pipeline when nothing matches.
*/
export async function matchRule(ctx) {
const { message, log, messageId } = ctx;
if (!message) {
log.warn({ messageId }, "gmail-watch: match skipped (no message)");
return { break: true };
}
const original = resolveOriginal(message);
const fromEmail = extractEmail(original.from);
const rule = rules.find(
(r) =>
fromEmail === r.fromAddress.toLowerCase() &&
subjectMatches(original.subject, r.subject),
);
if (!rule) {
log.info(
{
messageId,
from: original.from,
subject: original.subject,
viaForward: original.viaForward,
},
"gmail-watch: no rule match",
);
return { break: true };
}
log.info(
{
messageId,
kind: rule.kind,
from: original.from,
subject: original.subject,
viaForward: original.viaForward,
},
"gmail-watch: rule matched",
);
return {
pass: {
rule,
original: {
from: original.from,
subject: original.subject,
date: original.date,
},
},
};
}
@@ -0,0 +1,76 @@
import { parseBriTransfer } from "../parsers/bri.js";
import { parseGrabReceipt } from "../parsers/grab.js";
import { parseLivinReceipt } from "../parsers/livin.js";
import { notifyReceiptIssue } from "../notify/ntfy.js";
const PARSERS = {
"livin.qr_payment.success": parseLivinReceipt,
"grab.ereceipt.ride": parseGrabReceipt,
"bri.transfer.success": parseBriTransfer,
};
/**
* Parse a matched message body into the shared envelope JSON.
* On failure, logs a warning, notifies ntfy, and breaks (Pub/Sub stays healthy).
*/
export async function parseMatchedMessage(ctx) {
const { message, rule, original, log, messageId, notification } = ctx;
const parse = PARSERS[rule?.kind];
if (!parse) {
log.warn({ messageId, kind: rule?.kind }, "gmail-watch: no parser for kind");
await notifyReceiptIssue(
{
reason: "no parser for kind",
messageId,
kind: rule?.kind,
message,
},
log,
);
return { break: true };
}
const fields = parse(message.body || "", { html: message.html || "" });
if (!fields) {
log.warn(
{ messageId, kind: rule.kind },
"gmail-watch: parse failed",
);
await notifyReceiptIssue(
{
reason: "parse failed",
messageId,
kind: rule.kind,
message,
},
log,
);
return { break: true };
}
// Grab tip shares From/Subject with ride; parser sets kind/reference.
const kind = fields.kind || rule.kind;
const parsed = {
kind,
messageId,
mailbox: notification?.emailAddress || "",
matched: {
from: message.from || "",
subject: message.subject || "",
},
original: {
from: original?.from || "",
subject: original?.subject || "",
date: original?.date || "",
},
amount: fields.amount,
currency: fields.currency || "IDR",
reference: fields.reference,
occurredAt: fields.occurredAt,
details: fields.details || {},
};
log.info({ messageId, kind: parsed.kind, parsed }, "gmail-watch: parsed");
return { pass: { parsed } };
}
+126
View File
@@ -0,0 +1,126 @@
import { pocketbaseConfigured, createRecord } from "../pocketbase/client.js";
import { toSpendingRecord } from "../pocketbase/map.js";
import { notifyReceiptIssue } from "../notify/ntfy.js";
/**
* Insert a parsed receipt into PocketBase spendings.
* Unique invoice_id conflicts are logged and skipped (no throw).
* Non-duplicate skips / errors notify ntfy.
*/
export async function saveSpending(ctx) {
const { parsed, rule, log, messageId, message } = ctx;
if (!parsed) {
log.warn({ messageId }, "gmail-watch: save skipped (no parsed)");
await notifyReceiptIssue(
{
reason: "save skipped (no parsed)",
messageId,
message,
},
log,
);
return { break: true };
}
if (!pocketbaseConfigured()) {
log.warn(
{ messageId, kind: parsed.kind },
"gmail-watch: pocketbase not configured; parsed only",
);
await notifyReceiptIssue(
{
reason: "pocketbase not configured",
messageId,
kind: parsed.kind,
message,
extra: { parsed },
},
log,
);
return { pass: { saved: false } };
}
const record = toSpendingRecord(parsed, rule);
if (!record.invoice_id || record.amount == null || !record.trx_date) {
log.warn(
{ messageId, kind: parsed.kind, record },
"gmail-watch: save skipped (incomplete record)",
);
await notifyReceiptIssue(
{
reason: "save skipped (incomplete record)",
messageId,
kind: parsed.kind,
message,
record,
extra: { parsed },
},
log,
);
return { break: true };
}
try {
const { ok, status, data } = await createRecord("spendings", record);
if (ok) {
log.info(
{
messageId,
kind: parsed.kind,
invoice_id: record.invoice_id,
recordId: data.id,
},
"gmail-watch: spending saved",
);
return { pass: { saved: true, spendingId: data.id } };
}
// Unique constraint / duplicate — no ntfy
const msg = String(data?.message || "");
const isUnique =
status === 400 &&
(/unique|invoice_id/i.test(msg) ||
data?.data?.invoice_id ||
JSON.stringify(data).includes("unique"));
if (isUnique) {
log.info(
{ messageId, invoice_id: record.invoice_id, status },
"gmail-watch: spending duplicate skipped",
);
return { pass: { saved: false, duplicate: true } };
}
log.error(
{ messageId, status, data },
"gmail-watch: spending save failed",
);
await notifyReceiptIssue(
{
reason: "spending save failed",
messageId,
kind: parsed.kind,
message,
record,
extra: { status, data },
},
log,
);
return { pass: { saved: false, error: true } };
} catch (err) {
log.error({ err, messageId }, "gmail-watch: spending save error");
await notifyReceiptIssue(
{
reason: "spending save error",
messageId,
kind: parsed.kind,
message,
record,
extra: { error: err.message || String(err) },
},
log,
);
return { pass: { saved: false, error: true } };
}
}
+6
View File
@@ -0,0 +1,6 @@
export class HttpError extends Error {
constructor(statusCode, message) {
super(message);
this.statusCode = statusCode;
}
}
+92
View File
@@ -0,0 +1,92 @@
import Fastify from "fastify";
import { GOOGLE_PUBSUB_TOPIC, PORT, WATCH_RENEW_MS, assertConfig } from "./config.js";
import { listUsersWithTokens } from "./db.js";
import { startWatch } from "./google.js";
import { fetchAndLogMessage } from "./handlers/fetch-and-log-message.js";
import { logEvent } from "./handlers/log-event.js";
import { matchRule } from "./handlers/match-rule.js";
import { parseMatchedMessage } from "./handlers/parse-matched-message.js";
import { saveSpending } from "./handlers/save-spending.js";
import { HttpError } from "./http.js";
import { pocketbaseConfigured } from "./pocketbase/client.js";
import { ensureSpendingsSchema } from "./pocketbase/schema.js";
import { registerPubsubRoutes } from "./routes/pubsub.js";
assertConfig();
const fetchHandlers = [logEvent, fetchAndLogMessage];
const processHandlers = [matchRule, parseMatchedMessage, saveSpending];
const app = Fastify({
logger: true,
trustProxy: true,
});
app.get("/health", async () => ({ ok: true }));
await registerPubsubRoutes(app, { fetchHandlers, processHandlers });
app.setErrorHandler((err, request, reply) => {
const status = err.statusCode || 500;
if (status >= 500) {
request.log.error(err);
}
const message =
err instanceof HttpError || status < 500
? err.message
: "Internal server error";
reply.code(status).send({ error: message });
});
async function renewAllWatches() {
const users = listUsersWithTokens();
if (!users.length) {
app.log.warn("gmail-watch: no signed-in users with tokens");
return;
}
for (const user of users) {
try {
await startWatch(user.id);
app.log.info(
{ userId: user.id, email: user.email },
"gmail-watch: watch started",
);
} catch (err) {
app.log.error({ err, userId: user.id, email: user.email }, "gmail-watch: watch failed");
}
}
}
let renewTimer;
app.addHook("onClose", async () => {
if (renewTimer) clearInterval(renewTimer);
});
try {
await app.listen({ port: PORT, host: "0.0.0.0" });
app.log.info(
{ topic: GOOGLE_PUBSUB_TOPIC },
`Watcher listening on http://localhost:${PORT}`,
);
if (pocketbaseConfigured()) {
try {
await ensureSpendingsSchema(app.log);
} catch (err) {
app.log.error({ err }, "pocketbase: schema ensure failed");
}
} else {
app.log.warn("pocketbase: not configured; spendings will not be saved");
}
await renewAllWatches();
renewTimer = setInterval(() => {
renewAllWatches().catch((err) => {
app.log.error({ err }, "gmail-watch: renew cycle failed");
});
}, WATCH_RENEW_MS);
renewTimer.unref?.();
} catch (err) {
app.log.error(err);
process.exit(1);
}
+66
View File
@@ -0,0 +1,66 @@
/**
* Extract a bare email address from a From header value.
* e.g. `"Livin'" <noreply.livin@bankmandiri.co.id>` → `noreply.livin@bankmandiri.co.id`
*/
export function extractEmail(from) {
if (!from) return "";
const angle = from.match(/<([^>]+)>/);
if (angle) return angle[1].trim().toLowerCase();
const bare = from.match(/[\w.+-]+@[\w.-]+\.\w+/);
return bare ? bare[0].toLowerCase() : "";
}
/**
* Parse a Gmail "Forwarded message" header block from the body.
* Returns null if no forwarded header is present.
*/
export function parseForwardedHeader(body) {
if (!body) return null;
const marker = /-{2,}\s*Forwarded message\s*-{2,}/i;
const idx = body.search(marker);
if (idx < 0) return null;
const after = body.slice(idx);
const headerMatch = after.match(
/From:\s*(.+)\r?\nDate:\s*(.+)\r?\nSubject:\s*(.+)\r?\nTo:\s*(.+)/i,
);
if (!headerMatch) return null;
return {
from: headerMatch[1].trim(),
date: headerMatch[2].trim(),
subject: headerMatch[3].trim(),
to: headerMatch[4].trim(),
};
}
/**
* Resolve the original sender/subject/date for matching.
* Prefer forwarded header when present; otherwise use the Gmail envelope.
*/
export function resolveOriginal(message) {
const forwarded = parseForwardedHeader(message?.body || "");
if (forwarded) {
return {
from: forwarded.from,
subject: forwarded.subject,
date: forwarded.date,
viaForward: true,
};
}
return {
from: message?.from || "",
subject: message?.subject || "",
date: message?.date || "",
viaForward: false,
};
}
export function subjectMatches(actual, expected) {
if (!actual || !expected) return false;
const a = actual.trim().toLowerCase();
const e = expected.trim().toLowerCase();
// Strip common Fwd:/Re: prefixes for comparison of the core subject
const strip = (s) => s.replace(/^(fwd|re):\s*/i, "").trim();
return a === e || strip(a) === strip(e) || a.includes(e);
}
+168
View File
@@ -0,0 +1,168 @@
/** Parse Indonesian Rupiah strings like "Rp 22.000,00" or "Rp16.600" into integers. */
export function parseIdrAmount(raw) {
if (raw == null) return null;
const s = String(raw).replace(/\s/g, "");
const match = s.match(/Rp?\s*([\d.]+(?:,\d{2})?)/i) || s.match(/([\d.]+(?:,\d{2})?)/);
if (!match) return null;
// Indonesian: dots are thousands separators, comma is decimal
const whole = match[1].replace(/\./g, "").replace(/,\d+$/, "");
const n = Number(whole);
return Number.isFinite(n) ? n : null;
}
const MONTHS = {
// English
jan: 1,
january: 1,
feb: 2,
february: 2,
mar: 3,
march: 3,
apr: 4,
april: 4,
may: 5,
jun: 6,
june: 6,
jul: 7,
july: 7,
aug: 8,
august: 8,
sep: 9,
sept: 9,
september: 9,
oct: 10,
october: 10,
nov: 11,
november: 11,
dec: 12,
december: 12,
// Indonesian (Livin uses e.g. "15 Agu 2026")
januari: 1,
februari: 2,
maret: 3,
mei: 5,
juni: 6,
juli: 7,
agustus: 8,
agu: 8,
ags: 8,
oktober: 10,
okt: 10,
nop: 11,
desember: 12,
des: 12,
};
/**
* Parse dates like "13 Sep 2026", "13 September 2026", or Indonesian
* "15 Agu 2026" into YYYY-MM-DD. Also accepts 2-digit years ("13 Sep 26").
*/
export function parseDayMonthYear(raw) {
if (!raw) return null;
const m = String(raw)
.trim()
.match(/(\d{1,2})\s+([A-Za-z]+)\s+(\d{2,4})\b/);
if (!m) return null;
const day = Number(m[1]);
const month = MONTHS[m[2].toLowerCase()];
let year = Number(m[3]);
if (m[3].length === 2) year += year >= 70 ? 1900 : 2000;
if (!month || !day || !year) return null;
return `${year}-${String(month).padStart(2, "0")}-${String(day).padStart(2, "0")}`;
}
/**
* Parse Grab tip timestamps like "13 Sep 26 10:47 +0700" → ISO with offset.
*/
export function parseGrabDateTime(raw) {
if (!raw) return null;
const m = String(raw)
.trim()
.match(
/(\d{1,2})\s+([A-Za-z]+)\s+(\d{2,4})\s+(\d{1,2}):(\d{2})(?::(\d{2}))?\s*([+-]\d{4})?/,
);
if (!m) return null;
const dateYmd = parseDayMonthYear(`${m[1]} ${m[2]} ${m[3]}`);
if (!dateYmd) return null;
const hh = String(Number(m[4])).padStart(2, "0");
const mm = m[5];
const ss = m[6] || "00";
const offset = m[7]
? `${m[7].slice(0, 3)}:${m[7].slice(3)}`
: "+07:00";
return `${dateYmd}T${hh}:${mm}:${ss}${offset}`;
}
/**
* Parse time like "10:43:51 WIB" into HH:MM:SS.
*/
export function parseTimeWib(raw) {
if (!raw) return null;
const m = String(raw).trim().match(/(\d{1,2}):(\d{2})(?::(\d{2}))?/);
if (!m) return null;
const hh = String(Number(m[1])).padStart(2, "0");
const mm = m[2];
const ss = m[3] || "00";
return `${hh}:${mm}:${ss}`;
}
/** Combine YYYY-MM-DD + HH:MM:SS as WIB (+07:00) ISO string. */
export function toWibIso(dateYmd, timeHms) {
if (!dateYmd) return null;
if (!timeHms) return dateYmd;
return `${dateYmd}T${timeHms}+07:00`;
}
/**
* Parse "10:47AM" / "10:47 AM" into 24h HH:MM:SS.
*/
export function parseAmPmTime(raw) {
if (!raw) return null;
const m = String(raw)
.trim()
.match(/^(\d{1,2}):(\d{2})(?::(\d{2}))?\s*([AaPp][Mm])$/);
if (!m) return null;
let hour = Number(m[1]);
const minute = m[2];
const second = m[3] || "00";
const ap = m[4].toUpperCase();
if (ap === "AM") {
if (hour === 12) hour = 0;
} else if (hour !== 12) {
hour += 12;
}
return `${String(hour).padStart(2, "0")}:${minute}:${second}`;
}
/**
* Capture value after a label, allowing optional whitespace between label and value
* (including jammed HTML-stripped forms like Tanggal13 Sep 2026).
*/
export function captureAfterLabel(body, label, { until } = {}) {
const escaped = label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const untilPart = until
? `(?=${until.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")})`
: "(?=\\n|$)";
const re = new RegExp(`${escaped}\\s*([^\\n]*?)${untilPart}`, "i");
const m = body.match(re);
return m ? m[1].trim() : null;
}
/**
* Capture a multi-line block after a label until the next known label or blank run.
*/
export function captureBlockAfterLabel(body, label, nextLabels = []) {
const escaped = label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const next =
nextLabels.length > 0
? nextLabels.map((l) => l.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")).join("|")
: null;
const until = next ? `(?=${next}|$)` : "$";
const re = new RegExp(`${escaped}\\s*([\\s\\S]*?)${until}`, "i");
const m = body.match(re);
if (!m) return null;
return m[1]
.split(/\r?\n/)
.map((l) => l.trim())
.filter(Boolean);
}
+256
View File
@@ -0,0 +1,256 @@
function decodeBase64Url(data) {
if (!data) return "";
const pad = data.length % 4 === 0 ? "" : "=".repeat(4 - (data.length % 4));
return Buffer.from(
data.replace(/-/g, "+").replace(/_/g, "/") + pad,
"base64",
).toString("utf8");
}
function stripHtml(html) {
return html
.replace(/<style[\s\S]*?<\/style>/gi, "")
.replace(/<script[\s\S]*?<\/script>/gi, "")
.replace(/<br\s*\/?>/gi, "\n")
.replace(/<\/p>/gi, "\n\n")
.replace(/<\/div>/gi, "\n")
.replace(/<\/tr>/gi, "\n")
.replace(/<[^>]+>/g, "")
.replace(/&nbsp;/g, " ")
.replace(/&amp;/g, "&")
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x27;/gi, "'")
.replace(/\n{3,}/g, "\n\n")
.trim();
}
function walkParts(payload, acc = { text: "", html: "" }) {
if (!payload) return acc;
const mime = payload.mimeType || "";
const data = payload.body?.data;
if (data) {
if (mime === "text/plain") acc.text += decodeBase64Url(data);
else if (mime === "text/html") acc.html += decodeBase64Url(data);
}
for (const part of payload.parts || []) {
walkParts(part, acc);
}
return acc;
}
/** Return both MIME text and html parts (decoded). */
export function extractMimeParts(payload) {
const { text, html } = walkParts(payload);
return {
text: text.trim(),
html: html.trim(),
};
}
export function extractBody(payload) {
const { text, html } = extractMimeParts(payload);
if (text) return text;
if (html) return stripHtml(html);
return "";
}
export function extractHtml(payload) {
return extractMimeParts(payload).html;
}
export function header(payload, name) {
const match = payload?.headers?.find(
(h) => h.name?.toLowerCase() === name.toLowerCase(),
);
return match?.value || "";
}
export function summarizeMessage(message) {
const payload = message.payload || {};
return {
id: message.id,
threadId: message.threadId,
from: header(payload, "From"),
to: header(payload, "To"),
subject: header(payload, "Subject") || "(no subject)",
date: header(payload, "Date"),
snippet: message.snippet || "",
};
}
function isRfc822Part(part) {
if (!part) return false;
const mime = (part.mimeType || "").toLowerCase();
const filename = (part.filename || "").toLowerCase();
return mime === "message/rfc822" || filename.endsWith(".eml");
}
/**
* Collect message/rfc822 (or *.eml) attachment parts. Does not recurse into
* nested .eml inside .eml — only direct attachment parts under the tree.
*/
export function listRfc822Parts(payload, acc = []) {
if (!payload) return acc;
for (const part of payload.parts || []) {
if (isRfc822Part(part)) {
acc.push(part);
continue; // do not recurse into nested eml
}
listRfc822Parts(part, acc);
}
return acc;
}
function findNestedHeaderSource(part) {
if (!part) return null;
if (header(part, "From") || header(part, "Subject")) return part;
for (const child of part.parts || []) {
const found = findNestedHeaderSource(child);
if (found) return found;
}
return null;
}
/**
* Build a synthetic message from a Gmail-expanded message/rfc822 part.
* When nested text/html is only available via attachmentId (common on large
* batches), call hydrateRfc822Message instead.
*/
export function messageFromRfc822Part(part, { parentMessageId, index = 0 } = {}) {
const headerSource = findNestedHeaderSource(part) || part;
const partId = part.partId || String(index);
const id = parentMessageId ? `${parentMessageId}#${partId}` : partId;
return {
id,
threadId: null,
from: header(headerSource, "From"),
to: header(headerSource, "To"),
subject: header(headerSource, "Subject") || "(no subject)",
date: header(headerSource, "Date"),
snippet: "",
body: extractBody(part),
html: extractHtml(part),
attachmentFilename: part.filename || "",
parentMessageId: parentMessageId || null,
_rfc822Part: part,
};
}
/** Collect text/plain + text/html nodes that need attachments.get. */
export function listNestedBodyAttachmentRefs(part, acc = []) {
if (!part) return acc;
const mime = (part.mimeType || "").toLowerCase();
const attachId = part.body?.attachmentId;
if (attachId && (mime === "text/plain" || mime === "text/html") && !part.body?.data) {
acc.push({ mime, attachmentId: attachId });
}
for (const child of part.parts || []) {
listNestedBodyAttachmentRefs(child, acc);
}
return acc;
}
/**
* Decode a simple multipart .eml (no nested multipart recursion beyond one level
* of multipart/*) into { text, html }. Used when only the top-level rfc822 blob
* is downloadable.
*/
export function extractMimePartsFromRawEml(raw) {
if (!raw) return { text: "", html: "" };
let text = "";
let html = "";
// Split on MIME boundaries (best-effort)
const boundaryMatch = raw.match(/boundary="?([^";\r\n]+)"?/i);
const chunks = boundaryMatch
? raw.split(new RegExp(`\\r?\\n--${boundaryMatch[1].replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}(?:--)?\\r?\\n`))
: [raw];
for (const chunk of chunks) {
const headerEnd = chunk.search(/\r?\n\r?\n/);
if (headerEnd < 0) continue;
const headers = chunk.slice(0, headerEnd);
let body = chunk.slice(headerEnd).replace(/^\r?\n\r?\n/, "");
const ct = headers.match(/Content-Type:\s*([^\r\n;]+)/i)?.[1]?.trim().toLowerCase() || "";
const cte = headers.match(/Content-Transfer-Encoding:\s*([^\r\n]+)/i)?.[1]?.trim().toLowerCase() || "";
if (cte === "base64") {
body = Buffer.from(body.replace(/\s+/g, ""), "base64").toString("utf8");
} else if (cte === "quoted-printable") {
body = body
.replace(/=\r?\n/g, "")
.replace(/=([0-9A-Fa-f]{2})/g, (_, h) => String.fromCharCode(parseInt(h, 16)));
}
if (ct === "text/plain") text += body;
else if (ct === "text/html") html += body;
}
return { text: text.trim(), html: html.trim() };
}
/**
* Ensure an rfc822-derived message has body/html. Fetches nested Gmail
* attachments when Gmail did not inline the content.
*/
export async function hydrateRfc822Message(gmail, parentMessageId, part, index = 0) {
const message = messageFromRfc822Part(part, { parentMessageId, index });
if (message.body || message.html) {
delete message._rfc822Part;
return message;
}
const refs = listNestedBodyAttachmentRefs(part);
let text = "";
let html = "";
for (const ref of refs) {
const { data } = await gmail.users.messages.attachments.get({
userId: "me",
messageId: parentMessageId,
id: ref.attachmentId,
});
const decoded = decodeBase64Url(data.data);
if (ref.mime === "text/plain") text += decoded;
else if (ref.mime === "text/html") html += decoded;
}
// Fallback: download the whole .eml blob
if (!text && !html && part.body?.attachmentId) {
const { data } = await gmail.users.messages.attachments.get({
userId: "me",
messageId: parentMessageId,
id: part.body.attachmentId,
});
const raw = decodeBase64Url(data.data);
const parts = extractMimePartsFromRawEml(raw);
text = parts.text;
html = parts.html;
}
message.html = html.trim();
message.body = text.trim() || (html ? stripHtml(html) : "");
delete message._rfc822Part;
return message;
}
/**
* Expand a fetched Gmail payload into work items:
* - if .eml / message/rfc822 attachments exist → those only
* - else → the outer message
*/
export function expandWorkItems(outerMessage, payload) {
const parts = listRfc822Parts(payload);
if (!parts.length) return [outerMessage];
return parts.map((part, index) =>
messageFromRfc822Part(part, {
parentMessageId: outerMessage.id,
index,
}),
);
}
+102
View File
@@ -0,0 +1,102 @@
import { NTFY_URL } from "../config.js";
/**
* POST a notification to ntfy. Never throws — failures are logged only.
*/
export async function notifyNtfy(
{
title,
message,
priority = "default",
tags = [],
},
log = console,
) {
if (!NTFY_URL) return { skipped: true };
try {
const headers = {
Title: title.slice(0, 250),
Priority: String(priority),
"Content-Type": "text/plain; charset=utf-8",
};
if (tags.length) headers.Tags = tags.join(",");
const res = await fetch(NTFY_URL, {
method: "POST",
headers,
body: message,
});
if (!res.ok) {
const text = await res.text().catch(() => "");
log.warn?.(
{ status: res.status, text: text.slice(0, 200) },
"ntfy: publish failed",
);
return { ok: false, status: res.status };
}
return { ok: true };
} catch (err) {
log.warn?.({ err: err.message || String(err) }, "ntfy: publish error");
return { ok: false, error: true };
}
}
/** Build a notification body for a failed/skipped receipt work item. */
export function formatReceiptAlert({
reason,
messageId,
kind,
message,
record,
extra,
}) {
const lines = [
`reason: ${reason}`,
`messageId: ${messageId || ""}`,
`kind: ${kind || ""}`,
`from: ${message?.from || ""}`,
`subject: ${message?.subject || ""}`,
`date: ${message?.date || ""}`,
`attachment: ${message?.attachmentFilename || ""}`,
`parentMessageId: ${message?.parentMessageId || ""}`,
];
if (record) {
lines.push(`record: ${JSON.stringify(record)}`);
}
if (extra) {
lines.push(`extra: ${JSON.stringify(extra)}`);
}
const body = message?.body || "";
lines.push("", "----- message body -----", body || "(empty)");
if (message?.html) {
lines.push(
"",
"----- message html (length) -----",
String(message.html.length),
);
}
// ntfy payloads should stay reasonably sized
const text = lines.join("\n");
const max = 12000;
if (text.length <= max) return text;
return `${text.slice(0, max)}\n\n… truncated (${text.length} chars total)`;
}
export async function notifyReceiptIssue(payload, log) {
const body = formatReceiptAlert(payload);
return notifyNtfy(
{
title: `gmail-hook: ${payload.reason}`,
message: body,
priority: "high",
tags: ["warning", "email"],
},
log,
);
}
+68
View File
@@ -0,0 +1,68 @@
import {
captureAfterLabel,
captureBlockAfterLabel,
parseDayMonthYear,
parseIdrAmount,
parseTimeWib,
toWibIso,
} from "../lib/parse-helpers.js";
/**
* Parse BRImo / Bank BRI transfer success body (plain text).
* Required: Nominal + Nomor Referensi.
*/
export function parseBriTransfer(body) {
if (!body) return null;
const payerMatch = body.match(/Halo[,\s*]*([^*\n,]+)/i);
const payerName = payerMatch ? payerMatch[1].replace(/\*/g, "").trim() : null;
const tanggalRaw = captureAfterLabel(body, "Tanggal");
const dateYmd = parseDayMonthYear(tanggalRaw);
const timeHms = parseTimeWib(tanggalRaw);
const occurredAt = toWibIso(dateYmd, timeHms);
// Prefer Nominal over Total (Total = Nominal + Biaya Admin).
const nominalRaw = captureAfterLabel(body, "Nominal");
const amount = parseIdrAmount(nominalRaw);
const reference =
captureAfterLabel(body, "Nomor Referensi") ||
body.match(/Nomor\s+Referensi\s*([^\s\n]+)/i)?.[1] ||
null;
if (amount == null || !reference) return null;
const namaTujuan = captureAfterLabel(body, "Nama Tujuan");
const jenisTransaksi = captureAfterLabel(body, "Jenis Transaksi");
const bankTujuan = captureAfterLabel(body, "Bank Tujuan");
const nomorTujuan = captureAfterLabel(body, "Nomor Tujuan");
const catatanRaw = captureAfterLabel(body, "Catatan");
const biayaAdmin = parseIdrAmount(captureAfterLabel(body, "Biaya Admin"));
const fundingLines = captureBlockAfterLabel(body, "Sumber Dana", [
"Jenis Transaksi",
]);
const fundingSource =
fundingLines?.join(" ").replace(/\s+/g, " ").trim() || null;
const details = {};
if (payerName) details.payerName = payerName;
if (namaTujuan) details.namaTujuan = namaTujuan.trim();
if (jenisTransaksi) details.jenisTransaksi = jenisTransaksi.trim();
if (bankTujuan) details.bankTujuan = bankTujuan.trim();
if (nomorTujuan) details.nomorTujuan = nomorTujuan.trim();
if (catatanRaw != null && catatanRaw.trim() && catatanRaw.trim() !== "-") {
details.catatan = catatanRaw.trim();
}
if (biayaAdmin != null) details.biayaAdmin = biayaAdmin;
if (fundingSource) details.fundingSource = fundingSource;
return {
amount,
currency: "IDR",
reference: reference.trim(),
occurredAt,
details,
};
}
+247
View File
@@ -0,0 +1,247 @@
import { parse } from "node-html-parser";
import {
captureAfterLabel,
captureBlockAfterLabel,
parseAmPmTime,
parseDayMonthYear,
parseGrabDateTime,
parseIdrAmount,
toWibIso,
} from "../lib/parse-helpers.js";
function stripForwardedHeader(body) {
const m = body.match(
/-{2,}\s*Forwarded message\s*-{2,}[\s\S]*?\nTo:[^\n]*\n+/i,
);
if (!m) return body;
return body.slice(m.index + m[0].length);
}
/** Tip receipts share From/Subject with ride e-receipts but different body copy. */
export function isGrabTipBody(body) {
if (!body) return false;
if (/your tip goes a long way/i.test(body)) return true;
if (/thanks!\s*your tip/i.test(body)) return true;
// Tip layout: has a Tip section, no ride pickup line
if (
!/picked up on/i.test(body) &&
!/hope you enjoyed your ride/i.test(body) &&
!/selamat menikmati makanan/i.test(body) &&
!/grabfood/i.test(body) &&
/\btip\b/i.test(body) &&
/total paid/i.test(body)
) {
return true;
}
return false;
}
/** GrabFood order receipts (ID/EN) — same From/Subject as rides. */
export function isGrabFoodBody(body) {
if (!body) return false;
if (/selamat menikmati makanan/i.test(body)) return true;
if (/jenis kendaraan:\s*\r?\n\s*GrabFood/i.test(body)) return true;
if (/\bGrabFood\b/i.test(body) && /pesanan id/i.test(body)) return true;
return false;
}
function extractProduct(content) {
const productMatch = content.match(
/^\s*(Bike\s+\w+|Car\s+\w+|Grab\w*|Food|Express|Mart)[^\n]*/im,
);
return productMatch ? productMatch[1].trim() : null;
}
function extractBookingId(body) {
return (
captureAfterLabel(body, "Booking ID:") ||
body.match(/Booking ID:\s*([A-Z0-9-]+)/i)?.[1] ||
null
);
}
function extractTotalPaid(body) {
return (
captureAfterLabel(body, "Total Paid") ||
body.match(/Total Paid\s*\r?\n\s*(Rp[\d.,]+)/i)?.[1]
);
}
/**
* Grab ride HTML puts pickup/dropoff as standalone "10:47AM" divs next to
* locations. Plain text only has "Picked up on 13 September 2026" (date only).
* Returns { pickup, dropoff } as HH:MM:SS (24h) when found.
*/
export function extractGrabTripTimesFromHtml(html) {
if (!html) return { pickup: null, dropoff: null };
const root = parse(html);
const times = [];
for (const el of root.querySelectorAll("div")) {
const t = (el.text || "").replace(/\u00a0/g, " ").trim();
if (!/^\d{1,2}:\d{2}(?::\d{2})?\s*[AaPp][Mm]$/.test(t)) continue;
const hms = parseAmPmTime(t);
if (hms) times.push(hms);
}
return {
pickup: times[0] || null,
dropoff: times[1] || null,
};
}
function parseRide(body, content, html) {
const product = extractProduct(content);
const pickedUpRaw = captureAfterLabel(body, "Picked up on");
const dateYmd = parseDayMonthYear(pickedUpRaw);
const { pickup, dropoff } = extractGrabTripTimesFromHtml(html);
const occurredAt = toWibIso(dateYmd, pickup) || dateYmd;
const bookingId = extractBookingId(body);
const amount = parseIdrAmount(extractTotalPaid(body));
if (amount == null || !bookingId) return null;
const details = { variant: "ride" };
if (product) details.product = product;
if (dropoff) details.dropoffAt = toWibIso(dateYmd, dropoff);
const driverMatch = body.match(
/Receipt issued by the driver\s+([^.\n]+)/i,
);
if (driverMatch) details.driver = driverMatch[1].trim();
const ratingMatch = body.match(
/(\d(?:\.\d)?)\s*(?:\[image: stars\]|★|stars)/i,
);
if (ratingMatch) details.driverRating = Number(ratingMatch[1]);
const complimentsBlock = body.match(
/Compliments for driver\s*\r?\n([\s\S]*?)(?=\r?\nBreakdown|\r?\nPassenger|$)/i,
);
if (complimentsBlock) {
const compliments = complimentsBlock[1]
.split(/\r?\n/)
.map((l) => l.trim())
.filter((l) => l && !/^\[image/i.test(l));
if (compliments.length) details.compliments = compliments;
}
const passenger = captureAfterLabel(body, "Passenger");
if (passenger) {
details.passenger = passenger.replace(/\s*Profile.*$/i, "").trim();
}
return {
variant: "ride",
kind: "grab.ereceipt.ride",
amount,
currency: "IDR",
reference: bookingId.trim(),
occurredAt,
details,
};
}
function parseTip(body, content) {
const product = extractProduct(content);
const bookingId = extractBookingId(body);
const amount = parseIdrAmount(extractTotalPaid(body));
if (amount == null || !bookingId) return null;
// Tip timestamp sits on its own line, e.g. "13 Sep 26 10:47 +0700"
const tipTs =
body.match(
/(\d{1,2}\s+[A-Za-z]+\s+\d{2,4}\s+\d{1,2}:\d{2}(?::\d{2})?\s*[+-]\d{4})/,
)?.[1] || null;
const occurredAt = parseGrabDateTime(tipTs);
const details = { variant: "tip" };
if (product) details.product = product;
return {
variant: "tip",
kind: "grab.ereceipt.tip",
amount,
currency: "IDR",
// Separate invoice from the ride spend for the same Booking ID
reference: `${bookingId.trim()}:tip`,
occurredAt,
details,
};
}
function parseFood(body, content) {
const orderId =
captureAfterLabel(body, "Pesanan ID") ||
body.match(/Pesanan ID\s*\r?\n\s*([A-Z0-9-]+)/i)?.[1] ||
captureAfterLabel(body, "Order ID") ||
null;
const totalRaw =
body.match(/TOTAL\s*\(INCL\.?\s*TAX\)\s*IDR\s*([\d.]+)/i)?.[0] ||
body.match(/TOTAL\s*\r?\n\s*IDR\s*([\d.]+)/i)?.[0] ||
body.match(/IDR\s*[\d.]+/)?.[0];
const amount = parseIdrAmount(totalRaw);
if (amount == null || !orderId) return null;
const occurredAt = parseGrabDateTime(
body.match(
/(\d{1,2}\s+[A-Za-z]+\s+\d{2,4}\s+\d{1,2}:\d{2}(?::\d{2})?\s*[+-]\d{4})/,
)?.[1],
);
const details = { variant: "food", product: "Food" };
const vehicle =
captureAfterLabel(body, "Jenis Kendaraan:") ||
body.match(/Jenis Kendaraan:\s*\r?\n\s*([^\n]+)/i)?.[1];
if (vehicle) details.vehicleType = vehicle.trim();
const forName =
captureAfterLabel(body, "Diterbitkan untuk") ||
body.match(/Diterbitkan untuk\s*\r?\n\s*([^\n]+)/i)?.[1];
if (forName) details.passenger = forName.trim();
const merchantLines = captureBlockAfterLabel(body, "Pesanan Dari:", [
"Profil:",
"Detail Tagihan",
"Detail Pembayaran",
]);
if (merchantLines?.length) {
details.merchant = merchantLines[0];
if (merchantLines.length > 1) {
details.location = merchantLines
.slice(1)
.join(" ")
.replace(/<https?:\/\/[^>]+>/g, "")
.replace(/\s+/g, " ")
.trim();
}
}
const payment =
captureAfterLabel(body, "Detail Pembayaran:") ||
body.match(/Detail Pembayaran:\s*\r?\n\s*([^\n]+)/i)?.[1];
if (payment && !/Jumlah/i.test(payment)) {
details.paymentMethod = payment.trim();
}
return {
variant: "food",
kind: "grab.ereceipt.food",
amount,
currency: "IDR",
reference: orderId.trim(),
occurredAt,
details,
};
}
export function parseGrabReceipt(body, { html = "" } = {}) {
if (!body) return null;
const content = stripForwardedHeader(body);
if (isGrabTipBody(body)) return parseTip(body, content);
if (isGrabFoodBody(body)) return parseFood(body, content);
return parseRide(body, content, html);
}
+94
View File
@@ -0,0 +1,94 @@
import {
captureAfterLabel,
captureBlockAfterLabel,
parseDayMonthYear,
parseIdrAmount,
parseTimeWib,
toWibIso,
} from "../lib/parse-helpers.js";
/**
* Split Penerima lines into merchant + location when the body already
* separates them. If HTML-stripped glue merges them into one line
* (e.g. SNACK BAROKAHJAKARTA BARAT - ID), keep the whole string as merchant
* — do not guess Indonesian city names.
*/
export function splitMerchantLocation(lines) {
if (!lines || !lines.length) {
return { merchant: null, location: null };
}
if (lines.length >= 2) {
const last = lines[lines.length - 1];
if (/-\s*ID\s*$/i.test(last)) {
return {
merchant: lines.slice(0, -1).join(" ").trim() || null,
location: last.trim(),
};
}
}
return { merchant: lines.join(" ").trim() || null, location: null };
}
export function parseLivinReceipt(body) {
if (!body) return null;
const payerMatch = body.match(/Halo\s+([^,\n]+)/i);
const payerName = payerMatch ? payerMatch[1].trim() : null;
const penerimaLines = captureBlockAfterLabel(body, "Penerima", [
"Tanggal",
"Jam",
"Nominal",
]);
const { merchant, location } = splitMerchantLocation(penerimaLines);
const tanggalRaw = captureAfterLabel(body, "Tanggal");
const jamRaw = captureAfterLabel(body, "Jam");
const dateYmd = parseDayMonthYear(tanggalRaw);
const timeHms = parseTimeWib(jamRaw);
const occurredAt = toWibIso(dateYmd, timeHms);
const nominalRaw = captureAfterLabel(body, "Nominal Transaksi");
const amount = parseIdrAmount(nominalRaw);
const reference =
captureAfterLabel(body, "No. Referensi") ||
body.match(/No\.\s*Referensi\s*([^\s\n]+)/i)?.[1] ||
null;
if (amount == null || !reference) return null;
const details = {};
if (payerName) details.payerName = payerName;
if (merchant) details.merchant = merchant;
if (location) details.location = location;
const qrisRef =
captureAfterLabel(body, "No. Ref. QRIS") ||
body.match(/No\.\s*Ref\.\s*QRIS\s*([^\s\n]+)/i)?.[1];
const merchantPan = captureAfterLabel(body, "Merchant PAN");
const customerPan = captureAfterLabel(body, "Customer PAN");
const acquirer = captureAfterLabel(body, "Pengakuisisi");
const terminalId = captureAfterLabel(body, "Terminal ID");
const fundingLines = captureBlockAfterLabel(body, "Sumber Dana", [
"Simpan email",
]);
const fundingSource = fundingLines?.join(" ").replace(/\s+/g, " ").trim() || null;
if (qrisRef) details.qrisRef = qrisRef;
if (merchantPan) details.merchantPan = merchantPan;
if (customerPan) details.customerPan = customerPan;
if (acquirer) details.acquirer = acquirer;
if (terminalId) details.terminalId = terminalId.trim();
if (fundingSource) details.fundingSource = fundingSource;
return {
amount,
currency: "IDR",
reference: reference.trim(),
occurredAt,
details,
};
}
+12
View File
@@ -0,0 +1,12 @@
export async function runPipeline(handlers, ctx) {
let current = { ...ctx };
for (const handler of handlers) {
const result = await handler(current);
if (!result) continue;
if (result.pass && typeof result.pass === "object" && !Array.isArray(result.pass)) {
current = { ...current, ...result.pass };
}
if (result.break) break;
}
return current;
}
+91
View File
@@ -0,0 +1,91 @@
import {
POCKETBASE_PASSWORD,
POCKETBASE_URL,
POCKETBASE_USER,
} from "../config.js";
let cachedToken = null;
let cachedTokenAt = 0;
const TOKEN_TTL_MS = 50 * 60 * 1000; // refresh before typical 1h expiry
export function pocketbaseConfigured() {
return Boolean(POCKETBASE_URL && POCKETBASE_USER && POCKETBASE_PASSWORD);
}
async function authSuperuser() {
const res = await fetch(
`${POCKETBASE_URL}/api/collections/_superusers/auth-with-password`,
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
identity: POCKETBASE_USER,
password: POCKETBASE_PASSWORD,
}),
},
);
const data = await res.json().catch(() => ({}));
if (!res.ok || !data.token) {
throw new Error(
`PocketBase auth failed (${res.status}): ${data.message || "unknown"}`,
);
}
cachedToken = data.token;
cachedTokenAt = Date.now();
return cachedToken;
}
async function getToken() {
if (cachedToken && Date.now() - cachedTokenAt < TOKEN_TTL_MS) {
return cachedToken;
}
return authSuperuser();
}
export async function pbRequest(path, { method = "GET", body, retry = true } = {}) {
if (!pocketbaseConfigured()) {
throw new Error("PocketBase is not configured");
}
const token = await getToken();
const res = await fetch(`${POCKETBASE_URL}${path}`, {
method,
headers: {
Authorization: token,
...(body ? { "Content-Type": "application/json" } : {}),
},
body: body ? JSON.stringify(body) : undefined,
});
if (res.status === 401 && retry) {
cachedToken = null;
await authSuperuser();
return pbRequest(path, { method, body, retry: false });
}
const text = await res.text();
let data = {};
try {
data = text ? JSON.parse(text) : {};
} catch {
data = { raw: text };
}
return { ok: res.ok, status: res.status, data };
}
export async function getCollection(name) {
return pbRequest(`/api/collections/${name}`);
}
export async function updateCollection(name, patch) {
return pbRequest(`/api/collections/${name}`, {
method: "PATCH",
body: patch,
});
}
export async function createRecord(collection, record) {
return pbRequest(`/api/collections/${collection}/records`, {
method: "POST",
body: record,
});
}
+52
View File
@@ -0,0 +1,52 @@
/**
* Map parsed receipt envelope → PocketBase spendings record.
*/
export function toSpendingRecord(parsed, rule) {
const source = rule?.source || parsed.kind?.split(".")[0] || "";
let description = "";
if (source === "livin") {
description = parsed.details?.merchant || "";
} else if (source === "bri") {
description = parsed.details?.namaTujuan || "";
} else if (source === "grab") {
if (parsed.kind === "grab.ereceipt.food" || parsed.details?.variant === "food") {
const merchant = parsed.details?.merchant;
description = merchant ? `Grab Food ${merchant}` : "Grab Food";
} else {
const product = parsed.details?.product;
const base = product ? `Grab ${product}` : "Grab";
description =
parsed.kind === "grab.ereceipt.tip" || parsed.details?.variant === "tip"
? `${base} tip`
: base;
}
}
return {
invoice_id: parsed.reference,
amount: parsed.amount,
trx_date: toPocketBaseDate(parsed.occurredAt),
source,
description,
category: "",
message_id: parsed.messageId || "",
details: parsed.details || {},
// Full watcher envelope snapshot for audit / debugging (not the raw email body).
parsed,
};
}
/**
* PocketBase date fields accept "YYYY-MM-DD HH:mm:ss.SSSZ" (UTC) or date-only.
*/
export function toPocketBaseDate(occurredAt) {
if (!occurredAt) return null;
// Date-only (Grab)
if (/^\d{4}-\d{2}-\d{2}$/.test(occurredAt)) {
return `${occurredAt} 00:00:00.000Z`;
}
const d = new Date(occurredAt);
if (Number.isNaN(d.getTime())) return null;
// PocketBase: "2026-09-13 07:48:31.000Z"
return d.toISOString().replace("T", " ");
}
+121
View File
@@ -0,0 +1,121 @@
import { getCollection, pocketbaseConfigured, updateCollection } from "./client.js";
const SPENDINGS = "spendings";
const DESIRED_FIELDS = [
{
name: "invoice_id",
type: "text",
required: true,
min: 0,
},
{
name: "amount",
type: "number",
required: true,
},
{
name: "trx_date",
type: "date",
required: true,
},
{
name: "source",
type: "text",
required: true,
min: 0,
},
{
name: "description",
type: "text",
required: false,
min: 0,
},
{
name: "category",
type: "text",
required: false,
min: 0,
},
{
name: "message_id",
type: "text",
required: false,
min: 0,
},
{
name: "details",
type: "json",
required: false,
},
{
name: "parsed",
type: "json",
required: false,
},
];
const UNIQUE_INVOICE_INDEX =
"CREATE UNIQUE INDEX IF NOT EXISTS idx_spendings_invoice_id ON spendings (invoice_id)";
/**
* Ensure spendings has the fields and unique invoice_id index we need.
* Safe on an empty collection. Idempotent.
*/
export async function ensureSpendingsSchema(log = console) {
if (!pocketbaseConfigured()) {
log.warn?.("pocketbase: skipped schema ensure (not configured)");
return { skipped: true };
}
const { ok, status, data } = await getCollection(SPENDINGS);
if (!ok) {
throw new Error(
`pocketbase: cannot load spendings collection (${status}): ${data.message || ""}`,
);
}
const existing = data.fields || [];
const byName = new Map(existing.map((f) => [f.name, f]));
// Keep system/autodate fields; merge desired custom fields by name.
const systemNames = new Set(["id", "created", "updated"]);
const nextFields = existing.filter((f) => systemNames.has(f.name));
for (const desired of DESIRED_FIELDS) {
const prev = byName.get(desired.name);
if (prev) {
nextFields.push({
...prev,
type: desired.type,
required: desired.required,
...(desired.min != null ? { min: desired.min } : {}),
});
} else {
nextFields.push({ ...desired });
}
}
// Preserve any other unknown custom fields the user may have added.
for (const f of existing) {
if (systemNames.has(f.name)) continue;
if (DESIRED_FIELDS.some((d) => d.name === f.name)) continue;
nextFields.push(f);
}
const indexes = Array.isArray(data.indexes) ? [...data.indexes] : [];
if (!indexes.some((idx) => /invoice_id/i.test(String(idx)))) {
indexes.push(UNIQUE_INVOICE_INDEX);
}
const patch = { fields: nextFields, indexes };
const result = await updateCollection(SPENDINGS, patch);
if (!result.ok) {
throw new Error(
`pocketbase: failed to update spendings schema (${result.status}): ${result.data.message || JSON.stringify(result.data)}`,
);
}
log.info?.({ collection: SPENDINGS }, "pocketbase: spendings schema ensured");
return { ok: true };
}
+32
View File
@@ -0,0 +1,32 @@
import { runPipeline } from "./pipeline.js";
/**
* After fetch: if .eml attachments exist, process each; otherwise process outer.
* `fetchResult` is the ctx after [logEvent, fetchAndLogMessage].
*/
export async function processFetchedMessage(fetchResult, processHandlers) {
if (!fetchResult?.message) return [];
const workItems = fetchResult.emlMessages?.length
? fetchResult.emlMessages
: [fetchResult.message];
const results = [];
for (const item of workItems) {
const result = await runPipeline(processHandlers, {
...fetchResult,
message: item,
messageId: item.id,
// Clear prior match/parse/save state from a previous item
rule: undefined,
original: undefined,
parsed: undefined,
saved: undefined,
duplicate: undefined,
error: undefined,
spendingId: undefined,
});
results.push(result);
}
return results;
}
+169
View File
@@ -0,0 +1,169 @@
import { timingSafeEqual } from "node:crypto";
import { PUBSUB_VERIFICATION_TOKEN } from "../config.js";
import { getUserByEmail, getWatchState, saveWatchState } from "../db.js";
import {
gmailClientForUser,
googleStatus,
listHistory,
startWatch,
} from "../google.js";
import { HttpError } from "../http.js";
import { runPipeline } from "../pipeline.js";
import { processFetchedMessage } from "../process-fetched.js";
const userLocks = new Map();
function withUserLock(userId, fn) {
const prev = userLocks.get(userId) || Promise.resolve();
const next = prev.then(fn, fn);
userLocks.set(
userId,
next.then(
() => {},
() => {},
),
);
return next;
}
function tokenFromRequest(request) {
const q = request.query?.token;
if (typeof q === "string" && q) return q;
const header =
request.headers["x-pubsub-verification-token"] ||
request.headers["x-goog-pubsub-verification-token"];
if (typeof header === "string" && header) return header;
const auth = request.headers.authorization;
if (typeof auth === "string" && auth.startsWith("Bearer ")) {
return auth.slice(7);
}
return "";
}
function tokensMatch(provided, expected) {
const a = Buffer.from(provided);
const b = Buffer.from(expected);
if (a.length !== b.length) return false;
return timingSafeEqual(a, b);
}
function verifyPush(request) {
if (!PUBSUB_VERIFICATION_TOKEN) return;
if (!tokensMatch(tokenFromRequest(request), PUBSUB_VERIFICATION_TOKEN)) {
throw new HttpError(403, "Invalid Pub/Sub verification token");
}
}
function decodeNotification(body) {
const data = body?.message?.data;
if (typeof data !== "string" || !data) return null;
try {
const json = Buffer.from(data, "base64").toString("utf8");
const parsed = JSON.parse(json);
if (!parsed?.emailAddress || parsed.historyId == null) return null;
return {
emailAddress: parsed.emailAddress,
historyId: String(parsed.historyId),
};
} catch {
return null;
}
}
async function processNotification(log, { fetchHandlers, processHandlers }, notification) {
const user = getUserByEmail(notification.emailAddress);
if (!user) {
log.warn(
{ email: notification.emailAddress },
"gmail-watch: unknown mailbox, acking",
);
return;
}
await withUserLock(user.id, () =>
handleUserNotification(log, { fetchHandlers, processHandlers }, user, notification),
);
}
async function handleUserNotification(
log,
{ fetchHandlers, processHandlers },
user,
notification,
) {
const gmail = await gmailClientForUser(user.id);
const state = getWatchState(user.id);
const startHistoryId = state?.history_id;
if (!startHistoryId) {
log.info(
{ userId: user.id },
"gmail-watch: no cursor yet, starting watch and skipping backlog",
);
await startWatch(user.id, { resetCursor: true });
return;
}
let result;
try {
result = await listHistory(gmail, startHistoryId);
} catch (err) {
const status = googleStatus(err);
if (status === 404) {
log.warn(
{ userId: user.id, startHistoryId },
"gmail-watch: history expired, re-watching without backfill",
);
await startWatch(user.id, { resetCursor: true });
return;
}
throw err;
}
for (const messageId of result.messageIds) {
const fetched = await runPipeline(fetchHandlers, {
notification,
userId: user.id,
messageId,
gmail,
log,
});
await processFetchedMessage(fetched, processHandlers);
}
saveWatchState(user.id, {
historyId: result.historyId || notification.historyId,
expiration: state.expiration || Date.now() + 7 * 24 * 3600 * 1000,
});
}
export async function registerPubsubRoutes(app, { fetchHandlers, processHandlers }) {
app.post("/pubsub/gmail", async (request, reply) => {
verifyPush(request);
const notification = decodeNotification(request.body);
if (!notification) {
request.log.warn("gmail-watch: undecodable Pub/Sub payload, acking");
return reply.code(204).send();
}
try {
await processNotification(
request.log,
{ fetchHandlers, processHandlers },
notification,
);
} catch (err) {
const status = googleStatus(err) || err.statusCode || 500;
if (status === 401 || status === 403) {
throw new HttpError(
500,
"Gmail access expired or was revoked. Sign in again.",
);
}
throw err;
}
return reply.code(204).send();
});
}
+6
View File
@@ -0,0 +1,6 @@
export const briTransfer = {
kind: "bri.transfer.success",
source: "bri",
fromAddress: "bankbri@bri.co.id",
subject: "Pemindahan Dana Sesama Rekening BRI",
};
+6
View File
@@ -0,0 +1,6 @@
export const grabEreceipt = {
kind: "grab.ereceipt.ride",
source: "grab",
fromAddress: "no-reply@grab.com",
subject: "Your Grab E-Receipt",
};
+5
View File
@@ -0,0 +1,5 @@
import { briTransfer } from "./bri-transfer.js";
import { grabEreceipt } from "./grab-ereceipt.js";
import { livinQrPayment } from "./livin-qr-payment.js";
export const rules = [livinQrPayment, grabEreceipt, briTransfer];
@@ -0,0 +1,6 @@
export const livinQrPayment = {
kind: "livin.qr_payment.success",
source: "livin",
fromAddress: "noreply.livin@bankmandiri.co.id",
subject: "Pembayaran Berhasil!",
};
+327
View File
@@ -0,0 +1,327 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
expandWorkItems,
listRfc822Parts,
messageFromRfc822Part,
} from "../src/mime.js";
import { processFetchedMessage } from "../src/process-fetched.js";
import { matchRule } from "../src/handlers/match-rule.js";
import { parseMatchedMessage } from "../src/handlers/parse-matched-message.js";
const silentLog = {
info() {},
warn() {},
error() {},
};
function textPart(text, partId = "0.0") {
return {
partId,
mimeType: "text/plain",
body: {
data: Buffer.from(text, "utf8")
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, ""),
},
};
}
function htmlPart(html, partId = "0.1") {
return {
partId,
mimeType: "text/html",
body: {
data: Buffer.from(html, "utf8")
.toString("base64")
.replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, ""),
},
};
}
function rfc822Part({
partId,
filename,
from,
subject,
date,
bodyText,
bodyHtml,
}) {
return {
partId,
mimeType: "message/rfc822",
filename,
parts: [
{
partId: `${partId}.0`,
mimeType: "multipart/alternative",
headers: [
{ name: "From", value: from },
{ name: "Subject", value: subject },
{ name: "Date", value: date },
],
parts: [
textPart(bodyText, `${partId}.0.0`),
...(bodyHtml ? [htmlPart(bodyHtml, `${partId}.0.1`)] : []),
],
},
],
};
}
const grabRideBody = `Bike Hemat
Hope you enjoyed your ride!
Picked up on 13 September 2026
Booking ID: A-TESTBOOKING1
Total Paid
Rp16.600
Receipt issued by the driver Mahendra.
Passenger
Bintang
`;
const grabFoodBody = `Selamat menikmati makanan Anda!
TOTAL
IDR 49000 TANGGAL | WAKTU
11 Sep 26 18:24 +0700
Detail Pesanan
Jenis Kendaraan:
GrabFood
Diterbitkan untuk
Bintang
Pesanan ID
00112712674-TESTFOOD1
Pesanan Dari:
Doner Kebab - Kalibata City
Profil:
Personal
Detail Tagihan
Detail Pembayaran:
OVO Cash
TOTAL (INCL. TAX) IDR 49000
`;
describe("listRfc822Parts", () => {
it("finds message/rfc822 attachments and does not treat outer as eml", () => {
const payload = {
mimeType: "multipart/mixed",
parts: [
{
mimeType: "multipart/alternative",
parts: [textPart("wrapper")],
},
rfc822Part({
partId: "1",
filename: "Your Grab E-Receipt.eml",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Wed, 9 Sep 2026 06:09:32 +0000",
bodyText: grabRideBody,
}),
rfc822Part({
partId: "2",
filename: "Your Grab E-Receipt.eml",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Thu, 10 Sep 2026 11:28:24 +0000",
bodyText: grabFoodBody,
}),
],
};
const parts = listRfc822Parts(payload);
assert.equal(parts.length, 2);
assert.equal(parts[0].filename, "Your Grab E-Receipt.eml");
});
});
describe("messageFromRfc822Part", () => {
it("builds synthetic message with nested From/Subject/body", () => {
const part = rfc822Part({
partId: "1",
filename: "Your Grab E-Receipt.eml",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Wed, 9 Sep 2026 06:09:32 +0000",
bodyText: grabRideBody,
});
const msg = messageFromRfc822Part(part, {
parentMessageId: "outer123",
index: 0,
});
assert.equal(msg.id, "outer123#1");
assert.equal(msg.from, "Grab <no-reply@grab.com>");
assert.equal(msg.subject, "Your Grab E-Receipt");
assert.match(msg.body, /Booking ID: A-TESTBOOKING1/);
assert.equal(msg.parentMessageId, "outer123");
assert.equal(msg.attachmentFilename, "Your Grab E-Receipt.eml");
});
});
describe("expandWorkItems", () => {
it("returns outer only when no eml", () => {
const outer = {
id: "outer",
from: "me@x.com",
subject: "hi",
body: "x",
html: "",
};
const payload = {
mimeType: "multipart/alternative",
parts: [textPart("hi")],
};
const items = expandWorkItems(outer, payload);
assert.equal(items.length, 1);
assert.equal(items[0], outer);
});
it("returns eml messages when attachments exist", () => {
const outer = {
id: "outer123",
from: "NSRB <nasyarobby@gmail.com>",
subject: "Test attachment",
body: "Regards",
html: "",
};
const payload = {
mimeType: "multipart/mixed",
parts: [
{ mimeType: "multipart/alternative", parts: [textPart("Regards")] },
rfc822Part({
partId: "1",
filename: "a.eml",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Wed, 9 Sep 2026 06:09:32 +0000",
bodyText: grabRideBody,
}),
],
};
const items = expandWorkItems(outer, payload);
assert.equal(items.length, 1);
assert.equal(items[0].from, "Grab <no-reply@grab.com>");
assert.equal(items[0].id, "outer123#1");
});
});
describe("processFetchedMessage fan-out", () => {
it("processes each eml attachment and skips outer wrapper", async () => {
const fetched = {
notification: { emailAddress: "eleven16th@gmail.com" },
log: silentLog,
message: {
id: "outer123",
from: "NSRB <nasyarobby@gmail.com>",
subject: "Test attachment",
body: "Regards",
html: "",
},
emlMessages: [
{
id: "outer123#1",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Wed, 9 Sep 2026 06:09:32 +0000",
body: grabRideBody,
html: "",
parentMessageId: "outer123",
},
{
id: "outer123#2",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
date: "Thu, 10 Sep 2026 11:28:24 +0000",
body: grabFoodBody,
html: "",
parentMessageId: "outer123",
},
],
};
const results = await processFetchedMessage(fetched, [
matchRule,
parseMatchedMessage,
]);
assert.equal(results.length, 2);
assert.equal(results[0].parsed?.kind, "grab.ereceipt.ride");
assert.equal(results[0].parsed?.reference, "A-TESTBOOKING1");
assert.equal(results[1].parsed?.kind, "grab.ereceipt.food");
assert.equal(results[1].parsed?.reference, "00112712674-TESTFOOD1");
});
it("processes outer message when no eml attachments", async () => {
const fetched = {
notification: { emailAddress: "eleven16th@gmail.com" },
log: silentLog,
message: {
id: "direct1",
from: `"Livin'" <noreply.livin@bankmandiri.co.id>`,
subject: "Pembayaran Berhasil!",
date: "Sun, 13 Sep 2026 14:48:36 +0700",
body: `Halo TEST,
Penerima
Shop
JAKARTA SELATAN - ID
Tanggal 13 Sep 2026
Jam 10:00:00 WIB
Nominal Transaksi Rp 10.000,00
No. Referensi 9999999999999999999`,
html: "",
},
emlMessages: [],
};
const results = await processFetchedMessage(fetched, [
matchRule,
parseMatchedMessage,
]);
assert.equal(results.length, 1);
assert.equal(results[0].parsed?.kind, "livin.qr_payment.success");
});
});
describe("matchRule on synthetic eml", () => {
it("matches Grab from nested headers without Fwd", async () => {
const result = await matchRule({
messageId: "outer#1",
log: silentLog,
message: {
id: "outer#1",
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
body: grabRideBody,
html: "",
},
});
assert.equal(result.pass.rule.kind, "grab.ereceipt.ride");
assert.equal(result.pass.original.from, "Grab <no-reply@grab.com>");
});
});
describe("extractMimePartsFromRawEml", () => {
it("pulls text/html from a simple multipart eml", async () => {
const { extractMimePartsFromRawEml } = await import("../src/mime.js");
const raw = [
'Content-Type: multipart/alternative; boundary="bnd"',
"",
"--bnd",
"Content-Type: text/plain; charset=UTF-8",
"",
"Hello plain",
"--bnd",
"Content-Type: text/html; charset=UTF-8",
"",
"<b>Hello html</b>",
"--bnd--",
"",
].join("\r\n");
const parts = extractMimePartsFromRawEml(raw);
assert.match(parts.text, /Hello plain/);
assert.match(parts.html, /Hello html/);
});
});
+233
View File
@@ -0,0 +1,233 @@
export const livinDirectBody = `Livin' by Mandiri
Pembayaran Berhasil
Halo NASYAROBBY NUGRAHA PUTRA,
Berikut adalah detail transaksi Anda dengan QR:
Penerima
SNACK BAROKAHJAKARTA BARAT - ID
Tanggal13 Sep 2026
Jam14:48:31 WIB
Nominal TransaksiRp 14.000,00
No. Referensi2609131121540314991
No. Ref. QRIS609532729372
Merchant PAN9360091436372513569
Customer PAN9360000822021345851
PengakuisisiGoPay
Terminal ID A01
Sumber Dana
Kartu Kredit - Mandiri Platinum ****4585
Simpan email ini sebagai ref`;
export const livinForwardedBody = `---------- Forwarded message ---------
From: Livin' <noreply.livin@bankmandiri.co.id>
Date: Sun, Sep 13, 2026 at 10:44 AM
Subject: Pembayaran Berhasil!
To: <nasyarobby@gmail.com>
[image: livinbymandiri]
Pembayaran Berhasil
Halo NASYAROBBY NUGRAHA PUTRA,
Berikut adalah detail transaksi Anda dengan QR:
Penerima
Eleven Chicken Cabang Gah
JAKARTA SELATAN - ID
Tanggal 13 Sep 2026
Jam 10:43:51 WIB
Nominal Transaksi Rp 22.000,00
No. Referensi 2609131121539559589`;
export const grabBody = `Bike Hemat
[image: Grab]
Hope you enjoyed your ride!
Picked up on 13 September 2026
Booking ID: A-9RXV4K2GWCLOAV
Total Paid
Rp16.600
Receipt issued by the driver Mahendra.
5.0 [image: stars]
Compliments for driver
Layanan Mantap
Breakdown
Fare
16.600
Fare
16.600
Total Paid
16.600
Passenger
Bintang
Profile
`;
export const grabForwardedBody = `---------- Forwarded message ---------
From: Grab <no-reply@grab.com>
Date: Sun, Sep 13, 2026 at 10:58 AM
Subject: Your Grab E-Receipt
To: <nasyarobby@gmail.com>
${grabBody}`;
export const grabTipForwardedBody = `---------- Forwarded message ---------
From: Grab <no-reply@grab.com>
Date: Sun, Sep 13, 2026 at 11:51 AM
Subject: Your Grab E-Receipt
To: <nasyarobby@gmail.com>
Bike Hemat
[image: Grab]
Thanks! Your tip goes a long way for your driver.
13 Sep 26 10:47 +0700
Total Paid
Rp2.000
Tip
From: Apartemen Kalibata City, Jl. Kalibata Raya No.1, Rawajati, Pancoran,
Kota Jakarta Selatan, DKI Jakarta, Indonesia, 12750
Booking ID: A-9RXV4K2GWCLOAV
`;
/** Minimal Grab ride HTML: pickup/dropoff times only exist here, not in plain text. */
export const grabRideHtml = `
<div>Picked up on 13 September 2026</div>
<div>Booking ID: A-9RXV4K2GWCLOAV</div>
<table>
<tr>
<td>
<div>Gate II Apartemen Kalibata City</div>
<div style="font-weight:300">10:47AM</div>
</td>
</tr>
<tr>
<td>
<div>Badan Koordinasi Penanaman Modal</div>
<div style="font-weight:300">10:58AM</div>
</td>
</tr>
</table>
`;
export const grabFoodForwardedBody = `---------- Forwarded message ---------
From: <no-reply@grab.com>
Date: Fri, Sep 11, 2026 at 7:01 PM
Subject: Your Grab E-Receipt
To: <nasyarobby@gmail.com>
[image: E-receipt]
Selamat menikmati makanan Anda!
TOTAL
IDR 49000 TANGGAL | WAKTU
11 Sep 26 18:24 +0700
Detail Pesanan
Jenis Kendaraan:
GrabFood
Diterbitkan untuk
Bintang
Pesanan ID
00112712674-C8EUEZDGEEAZTX
Pesanan Dari:
Doner Kebab - Kalibata City
Kalibata City, Foodcourt Teras Kuliner, Lt. Dasar Units Fb. 23 (Doner
Kebab), Jl. Raya Kalibata No. 1, Rawajati, Pancoran, Jakarta Selatan
Profil:
Personal
Detail Tagihan
Detail Pembayaran:
OVO Cash
Detail Pembayaran: Jumlah:
1x Fish n Chips IDR 45000
Subtotal IDR 45000
Restaurant packaging charge IDR 4000
TOTAL (INCL. TAX) IDR 49000
`;
export const briTransferDirectBody = `Halo,*NASYAROBBY NUGRAHA PUTRA*
Berikut ini adalah informasi transaksi yang telah Anda lakukan di Aplikasi
BRImo.
Transaksi Berhasil
Tanggal 13 September 2026 , 19:00:00 WIB
Nomor Referensi 212189333864
------------------------------
Sumber Dana NASYAROBBY NUGRAHA PUTRA
0121 **** **** 500
Jenis Transaksi Transfer Bank BRI
Bank Tujuan BANK BRI
Nomor Tujuan 1060 0100 2070 507
Nama Tujuan NELLY ARISANTI
Catatan -
------------------------------
Nominal Rp4.000.000
Biaya Admin Rp0
*Total* *Rp4.000.000*
Semoga informasi ini dapat bermanfaat bagi Anda.`;
export const briTransferForwardedBody = `Nasyarobby N. Putra
Boston University MSSD Program
---------- Forwarded message ---------
From: Bank BRI <BankBRI@bri.co.id>
Date: Sun, Sep 13, 2026, 7:00 PM
Subject: Pemindahan Dana Sesama Rekening BRI
To: <nsrb@bu.edu>
Halo,*NASYAROBBY NUGRAHA PUTRA*
Berikut ini adalah informasi transaksi yang telah Anda lakukan di Aplikasi
BRImo.
Transaksi Berhasil
Tanggal 13 September 2026 , 19:00:00 WIB
Nomor Referensi 212189333864
------------------------------
Sumber Dana NASYAROBBY NUGRAHA PUTRA
0121 **** **** 500
Jenis Transaksi Transfer Bank BRI
Bank Tujuan BANK BRI
Nomor Tujuan 1060 0100 2070 507
Nama Tujuan NELLY ARISANTI
Catatan -
------------------------------
Nominal Rp4.000.000
Biaya Admin Rp0
*Total* *Rp4.000.000*
`;
+171
View File
@@ -0,0 +1,171 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
extractEmail,
parseForwardedHeader,
resolveOriginal,
subjectMatches,
} from "../src/lib/email.js";
import { matchRule } from "../src/handlers/match-rule.js";
import {
briTransferDirectBody,
briTransferForwardedBody,
grabBody,
grabForwardedBody,
livinDirectBody,
livinForwardedBody,
} from "./fixtures.js";
const silentLog = {
info() {},
warn() {},
error() {},
};
describe("extractEmail", () => {
it("parses angle-bracket addresses", () => {
assert.equal(
extractEmail(`"Livin'" <noreply.livin@bankmandiri.co.id>`),
"noreply.livin@bankmandiri.co.id",
);
});
});
describe("parseForwardedHeader", () => {
it("reads original From/Subject from a Fwd block", () => {
const h = parseForwardedHeader(livinForwardedBody);
assert.ok(h);
assert.match(h.from, /noreply\.livin@bankmandiri\.co\.id/);
assert.equal(h.subject, "Pembayaran Berhasil!");
});
it("returns null when there is no Fwd block", () => {
assert.equal(parseForwardedHeader(livinDirectBody), null);
});
});
describe("resolveOriginal", () => {
it("uses envelope for direct mail", () => {
const o = resolveOriginal({
from: `"Livin'" <noreply.livin@bankmandiri.co.id>`,
subject: "Pembayaran Berhasil!",
date: "Sun, 13 Sep 2026 14:48:36 +0700 (WIB)",
body: livinDirectBody,
});
assert.equal(o.viaForward, false);
assert.equal(extractEmail(o.from), "noreply.livin@bankmandiri.co.id");
});
it("uses Fwd header for forwarded mail", () => {
const o = resolveOriginal({
from: "NSRB <nasyarobby@gmail.com>",
subject: "Fwd: Your Grab E-Receipt",
body: grabForwardedBody,
});
assert.equal(o.viaForward, true);
assert.equal(extractEmail(o.from), "no-reply@grab.com");
assert.equal(o.subject, "Your Grab E-Receipt");
});
});
describe("subjectMatches", () => {
it("matches exact and Fwd-prefixed subjects", () => {
assert.equal(subjectMatches("Pembayaran Berhasil!", "Pembayaran Berhasil!"), true);
assert.equal(subjectMatches("Fwd: Pembayaran Berhasil!", "Pembayaran Berhasil!"), true);
assert.equal(subjectMatches("Hello", "Pembayaran Berhasil!"), false);
});
});
describe("matchRule", () => {
it("matches direct Livin", async () => {
const result = await matchRule({
messageId: "1a099bd61efd234d",
log: silentLog,
message: {
from: `"Livin'" <noreply.livin@bankmandiri.co.id>`,
subject: "Pembayaran Berhasil!",
date: "Sun, 13 Sep 2026 14:48:36 +0700 (WIB)",
body: livinDirectBody,
},
});
assert.equal(result.pass.rule.kind, "livin.qr_payment.success");
});
it("matches forwarded Livin via original header", async () => {
const result = await matchRule({
messageId: "1a099b0c04c4397a",
log: silentLog,
message: {
from: "NSRB <nasyarobby@gmail.com>",
subject: "Fwd: Pembayaran Berhasil!",
body: livinForwardedBody,
},
});
assert.equal(result.pass.rule.kind, "livin.qr_payment.success");
});
it("matches direct Grab", async () => {
const result = await matchRule({
messageId: "grab-direct",
log: silentLog,
message: {
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
body: grabBody,
},
});
assert.equal(result.pass.rule.kind, "grab.ereceipt.ride");
});
it("matches forwarded Grab via original header", async () => {
const result = await matchRule({
messageId: "1a099b8b9d239e5e",
log: silentLog,
message: {
from: "NSRB <nasyarobby@gmail.com>",
subject: "Fwd: Your Grab E-Receipt",
body: grabForwardedBody,
},
});
assert.equal(result.pass.rule.kind, "grab.ereceipt.ride");
});
it("matches direct BRI transfer", async () => {
const result = await matchRule({
messageId: "bri-direct",
log: silentLog,
message: {
from: "Bank BRI <BankBRI@bri.co.id>",
subject: "Pemindahan Dana Sesama Rekening BRI",
body: briTransferDirectBody,
},
});
assert.equal(result.pass.rule.kind, "bri.transfer.success");
});
it("matches forwarded BRI transfer via original header", async () => {
const result = await matchRule({
messageId: "1a09b6cd9a3bc6bd",
log: silentLog,
message: {
from: "Nasyarobby Putra <nsrb@bu.edu>",
subject: "Fwd: Pemindahan Dana Sesama Rekening BRI",
body: briTransferForwardedBody,
},
});
assert.equal(result.pass.rule.kind, "bri.transfer.success");
});
it("breaks on unknown from/subject", async () => {
const result = await matchRule({
messageId: "x",
log: silentLog,
message: {
from: "Someone <other@example.com>",
subject: "Hello",
body: "hi",
},
});
assert.equal(result.break, true);
});
});
+23
View File
@@ -0,0 +1,23 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { formatReceiptAlert } from "../src/notify/ntfy.js";
describe("formatReceiptAlert", () => {
it("includes reason, messageId, and body", () => {
const text = formatReceiptAlert({
reason: "parse failed",
messageId: "abc#1",
kind: "grab.ereceipt.ride",
message: {
from: "Grab <no-reply@grab.com>",
subject: "Your Grab E-Receipt",
body: "Total Paid\nRp1.000",
html: "<b>x</b>",
},
});
assert.match(text, /reason: parse failed/);
assert.match(text, /messageId: abc#1/);
assert.match(text, /Total Paid/);
assert.match(text, /html \(length\)/);
});
});
+17
View File
@@ -0,0 +1,17 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { parseDayMonthYear } from "../src/lib/parse-helpers.js";
describe("parseDayMonthYear", () => {
it("parses English months", () => {
assert.equal(parseDayMonthYear("13 Sep 2026"), "2026-09-13");
assert.equal(parseDayMonthYear("13 September 2026"), "2026-09-13");
});
it("parses Indonesian months", () => {
assert.equal(parseDayMonthYear("15 Agu 2026"), "2026-08-15");
assert.equal(parseDayMonthYear("15 Agustus 2026"), "2026-08-15");
assert.equal(parseDayMonthYear("1 Des 26"), "2026-12-01");
assert.equal(parseDayMonthYear("1 Mei 2026"), "2026-05-01");
});
});
+276
View File
@@ -0,0 +1,276 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { parseBriTransfer } from "../src/parsers/bri.js";
import { parseGrabReceipt, extractGrabTripTimesFromHtml } from "../src/parsers/grab.js";
import { parseLivinReceipt, splitMerchantLocation } from "../src/parsers/livin.js";
import { toSpendingRecord, toPocketBaseDate } from "../src/pocketbase/map.js";
import { briTransfer } from "../src/rules/bri-transfer.js";
import { livinQrPayment } from "../src/rules/livin-qr-payment.js";
import { grabEreceipt } from "../src/rules/grab-ereceipt.js";
import {
briTransferDirectBody,
briTransferForwardedBody,
grabBody,
grabFoodForwardedBody,
grabForwardedBody,
grabRideHtml,
grabTipForwardedBody,
livinDirectBody,
livinForwardedBody,
} from "./fixtures.js";
describe("splitMerchantLocation", () => {
it("splits clean multi-line penerima", () => {
const r = splitMerchantLocation([
"Eleven Chicken Cabang Gah",
"JAKARTA SELATAN - ID",
]);
assert.equal(r.merchant, "Eleven Chicken Cabang Gah");
assert.equal(r.location, "JAKARTA SELATAN - ID");
});
it("keeps glued merchant+city as merchant only", () => {
const r = splitMerchantLocation(["SNACK BAROKAHJAKARTA BARAT - ID"]);
assert.equal(r.merchant, "SNACK BAROKAHJAKARTA BARAT - ID");
assert.equal(r.location, null);
});
});
describe("parseLivinReceipt", () => {
it("parses direct HTML-stripped body", () => {
const r = parseLivinReceipt(livinDirectBody);
assert.ok(r);
assert.equal(r.amount, 14000);
assert.equal(r.reference, "2609131121540314991");
assert.equal(r.occurredAt, "2026-09-13T14:48:31+07:00");
assert.equal(r.details.payerName, "NASYAROBBY NUGRAHA PUTRA");
assert.equal(r.details.merchant, "SNACK BAROKAHJAKARTA BARAT - ID");
assert.equal(r.details.location, undefined);
assert.equal(r.details.qrisRef, "609532729372");
assert.equal(r.details.acquirer, "GoPay");
});
it("parses forwarded clean body", () => {
const r = parseLivinReceipt(livinForwardedBody);
assert.ok(r);
assert.equal(r.amount, 22000);
assert.equal(r.reference, "2609131121539559589");
assert.equal(r.occurredAt, "2026-09-13T10:43:51+07:00");
assert.equal(r.details.merchant, "Eleven Chicken Cabang Gah");
assert.equal(r.details.location, "JAKARTA SELATAN - ID");
});
it("returns null without amount/reference", () => {
assert.equal(parseLivinReceipt("Halo someone\nPenerima\nFoo"), null);
});
it("parses Indonesian month abbreviation (Agu)", () => {
const body = `Halo NASYAROBBY NUGRAHA PUTRA,
Penerima
RM Bu Sum Ebony, KalibataJAKARTA SELATAN - ID
Tanggal15 Agu 2026
Jam09:45:25 WIB
Nominal TransaksiRp 15.000,00
No. Referensi2608151121545704363
No. Ref. QRIS608550990105`;
const r = parseLivinReceipt(body);
assert.ok(r);
assert.equal(r.amount, 15000);
assert.equal(r.reference, "2608151121545704363");
assert.equal(r.occurredAt, "2026-08-15T09:45:25+07:00");
});
});
describe("parseGrabReceipt", () => {
it("parses direct Grab body", () => {
const r = parseGrabReceipt(grabBody);
assert.ok(r);
assert.equal(r.kind, "grab.ereceipt.ride");
assert.equal(r.amount, 16600);
assert.equal(r.reference, "A-9RXV4K2GWCLOAV");
assert.equal(r.occurredAt, "2026-09-13");
assert.equal(r.details.product, "Bike Hemat");
assert.equal(r.details.driver, "Mahendra");
assert.equal(r.details.passenger, "Bintang");
assert.deepEqual(r.details.compliments, ["Layanan Mantap"]);
});
it("merges pickup time from HTML into occurredAt", () => {
const r = parseGrabReceipt(grabBody, { html: grabRideHtml });
assert.ok(r);
assert.equal(r.occurredAt, "2026-09-13T10:47:00+07:00");
assert.equal(r.details.dropoffAt, "2026-09-13T10:58:00+07:00");
});
it("extracts standalone AM/PM trip times from HTML", () => {
const t = extractGrabTripTimesFromHtml(grabRideHtml);
assert.equal(t.pickup, "10:47:00");
assert.equal(t.dropoff, "10:58:00");
});
it("parses forwarded Grab body", () => {
const r = parseGrabReceipt(grabForwardedBody);
assert.ok(r);
assert.equal(r.amount, 16600);
assert.equal(r.reference, "A-9RXV4K2GWCLOAV");
assert.equal(r.details.product, "Bike Hemat");
assert.equal(r.kind, "grab.ereceipt.ride");
});
it("parses Grab tip as a separate kind/invoice", () => {
const r = parseGrabReceipt(grabTipForwardedBody);
assert.ok(r);
assert.equal(r.kind, "grab.ereceipt.tip");
assert.equal(r.variant, "tip");
assert.equal(r.amount, 2000);
assert.equal(r.reference, "A-9RXV4K2GWCLOAV:tip");
assert.equal(r.occurredAt, "2026-09-13T10:47:00+07:00");
assert.equal(r.details.product, "Bike Hemat");
assert.equal(r.details.variant, "tip");
});
it("parses Grab Food receipt", () => {
const r = parseGrabReceipt(grabFoodForwardedBody);
assert.ok(r);
assert.equal(r.kind, "grab.ereceipt.food");
assert.equal(r.variant, "food");
assert.equal(r.amount, 49000);
assert.equal(r.reference, "00112712674-C8EUEZDGEEAZTX");
assert.equal(r.occurredAt, "2026-09-11T18:24:00+07:00");
assert.equal(r.details.merchant, "Doner Kebab - Kalibata City");
assert.equal(r.details.vehicleType, "GrabFood");
assert.equal(r.details.passenger, "Bintang");
assert.equal(r.details.paymentMethod, "OVO Cash");
});
});
describe("parseBriTransfer", () => {
it("parses direct BRImo transfer body", () => {
const r = parseBriTransfer(briTransferDirectBody);
assert.ok(r);
assert.equal(r.amount, 4000000);
assert.equal(r.reference, "212189333864");
assert.equal(r.occurredAt, "2026-09-13T19:00:00+07:00");
assert.equal(r.details.payerName, "NASYAROBBY NUGRAHA PUTRA");
assert.equal(r.details.namaTujuan, "NELLY ARISANTI");
assert.equal(r.details.jenisTransaksi, "Transfer Bank BRI");
assert.equal(r.details.bankTujuan, "BANK BRI");
assert.equal(r.details.nomorTujuan, "1060 0100 2070 507");
assert.equal(r.details.biayaAdmin, 0);
assert.equal(
r.details.fundingSource,
"NASYAROBBY NUGRAHA PUTRA 0121 **** **** 500",
);
assert.equal(r.details.catatan, undefined);
});
it("parses forwarded BRImo transfer body", () => {
const r = parseBriTransfer(briTransferForwardedBody);
assert.ok(r);
assert.equal(r.amount, 4000000);
assert.equal(r.reference, "212189333864");
assert.equal(r.details.namaTujuan, "NELLY ARISANTI");
});
it("returns null without amount/reference", () => {
assert.equal(parseBriTransfer("Halo someone\nTransaksi Berhasil"), null);
});
});
describe("toSpendingRecord", () => {
it("maps Livin envelope to spendings fields", () => {
const parsed = {
kind: "livin.qr_payment.success",
messageId: "1a099bd61efd234d",
amount: 14000,
reference: "2609131121540314991",
occurredAt: "2026-09-13T14:48:31+07:00",
details: {
merchant: "SNACK BAROKAHJAKARTA BARAT - ID",
qrisRef: "609532729372",
},
};
const rec = toSpendingRecord(parsed, livinQrPayment);
assert.equal(rec.invoice_id, "2609131121540314991");
assert.equal(rec.amount, 14000);
assert.equal(rec.source, "livin");
assert.equal(rec.description, "SNACK BAROKAHJAKARTA BARAT - ID");
assert.equal(rec.category, "");
assert.equal(rec.message_id, "1a099bd61efd234d");
assert.equal(rec.trx_date, toPocketBaseDate(parsed.occurredAt));
assert.deepEqual(rec.parsed, parsed);
});
it("maps Grab envelope to spendings fields", () => {
const parsed = {
kind: "grab.ereceipt.ride",
messageId: "1a099b8b9d239e5e",
amount: 16600,
reference: "A-9RXV4K2GWCLOAV",
occurredAt: "2026-09-13",
details: { product: "Bike Hemat", driver: "Mahendra" },
};
const rec = toSpendingRecord(parsed, grabEreceipt);
assert.equal(rec.invoice_id, "A-9RXV4K2GWCLOAV");
assert.equal(rec.source, "grab");
assert.equal(rec.description, "Grab Bike Hemat");
assert.equal(rec.trx_date, "2026-09-13 00:00:00.000Z");
assert.deepEqual(rec.parsed, parsed);
});
it("maps Grab tip envelope to spendings fields", () => {
const parsed = {
kind: "grab.ereceipt.tip",
messageId: "1a099e731cb511dc",
amount: 2000,
reference: "A-9RXV4K2GWCLOAV:tip",
occurredAt: "2026-09-13T10:47:00+07:00",
details: { product: "Bike Hemat", variant: "tip" },
};
const rec = toSpendingRecord(parsed, grabEreceipt);
assert.equal(rec.invoice_id, "A-9RXV4K2GWCLOAV:tip");
assert.equal(rec.source, "grab");
assert.equal(rec.description, "Grab Bike Hemat tip");
assert.equal(rec.trx_date, toPocketBaseDate(parsed.occurredAt));
});
it("maps Grab Food envelope to spendings fields", () => {
const parsed = {
kind: "grab.ereceipt.food",
messageId: "1a09a07cbd6f58cb",
amount: 49000,
reference: "00112712674-C8EUEZDGEEAZTX",
occurredAt: "2026-09-11T18:24:00+07:00",
details: {
variant: "food",
product: "Food",
merchant: "Doner Kebab - Kalibata City",
},
};
const rec = toSpendingRecord(parsed, grabEreceipt);
assert.equal(rec.invoice_id, "00112712674-C8EUEZDGEEAZTX");
assert.equal(rec.source, "grab");
assert.equal(rec.description, "Grab Food Doner Kebab - Kalibata City");
});
it("maps BRI transfer envelope to spendings fields", () => {
const parsed = {
kind: "bri.transfer.success",
messageId: "1a09b6cd9a3bc6bd",
amount: 4000000,
reference: "212189333864",
occurredAt: "2026-09-13T19:00:00+07:00",
details: {
namaTujuan: "NELLY ARISANTI",
jenisTransaksi: "Transfer Bank BRI",
},
};
const rec = toSpendingRecord(parsed, briTransfer);
assert.equal(rec.invoice_id, "212189333864");
assert.equal(rec.amount, 4000000);
assert.equal(rec.source, "bri");
assert.equal(rec.description, "NELLY ARISANTI");
assert.equal(rec.trx_date, toPocketBaseDate(parsed.occurredAt));
assert.deepEqual(rec.parsed, parsed);
});
});
+12
View File
@@ -0,0 +1,12 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Gmail Reader</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.jsx"></script>
</body>
</html>
+7
View File
@@ -0,0 +1,7 @@
{
"compilerOptions": {
"jsx": "react-jsx",
"module": "ESNext",
"moduleResolution": "bundler"
}
}
+22
View File
@@ -0,0 +1,22 @@
{
"name": "web",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^19.1.1",
"react-dom": "^19.1.1",
"react-router-dom": "^7.8.2"
},
"devDependencies": {
"@tailwindcss/vite": "^4.1.13",
"@vitejs/plugin-react": "^5.0.2",
"daisyui": "^5.1.13",
"tailwindcss": "^4.1.13",
"vite": "^7.1.5"
}
}
+56
View File
@@ -0,0 +1,56 @@
import { Navigate, Outlet, Route, Routes } from "react-router-dom";
import { useAuth } from "./auth.jsx";
import Layout from "./components/Layout.jsx";
import Inbox from "./pages/Inbox.jsx";
import Login from "./pages/Login.jsx";
import Message from "./pages/Message.jsx";
function Protected() {
const { user, loading } = useAuth();
if (loading) {
return (
<div className="grid min-h-screen place-items-center">
<span className="loading loading-spinner loading-lg" />
</div>
);
}
if (!user) return <Navigate to="/login" replace />;
return (
<Layout>
<Outlet />
</Layout>
);
}
function GuestOnly({ children }) {
const { user, loading } = useAuth();
if (loading) {
return (
<div className="grid min-h-screen place-items-center">
<span className="loading loading-spinner loading-lg" />
</div>
);
}
if (user) return <Navigate to="/" replace />;
return children;
}
export default function App() {
return (
<Routes>
<Route
path="/login"
element={
<GuestOnly>
<Login />
</GuestOnly>
}
/>
<Route element={<Protected />}>
<Route path="/" element={<Inbox />} />
<Route path="/messages/:id" element={<Message />} />
</Route>
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
);
}
+29
View File
@@ -0,0 +1,29 @@
export class AuthError extends Error {
constructor() {
super("Not authenticated");
this.name = "AuthError";
}
}
export async function api(path, options = {}) {
const headers = { ...(options.headers || {}) };
if (options.body && !headers["Content-Type"]) {
headers["Content-Type"] = "application/json";
}
const res = await fetch(path, {
credentials: "include",
...options,
headers,
});
if (res.status === 401) {
throw new AuthError();
}
const data = await res.json().catch(() => ({}));
if (!res.ok) {
throw new Error(data.error || res.statusText);
}
return data;
}
+49
View File
@@ -0,0 +1,49 @@
import { createContext, useContext, useEffect, useState } from "react";
import { api, AuthError } from "./api.js";
const AuthContext = createContext(null);
export function AuthProvider({ children }) {
const [user, setUser] = useState(null);
const [loading, setLoading] = useState(true);
useEffect(() => {
let cancelled = false;
api("/api/me")
.then((me) => {
if (!cancelled) setUser(me);
})
.catch((err) => {
if (!cancelled && !(err instanceof AuthError)) {
console.error(err);
}
if (!cancelled) setUser(null);
})
.finally(() => {
if (!cancelled) setLoading(false);
});
return () => {
cancelled = true;
};
}, []);
async function logout() {
try {
await api("/auth/logout", { method: "POST" });
} finally {
setUser(null);
}
}
return (
<AuthContext.Provider value={{ user, loading, logout }}>
{children}
</AuthContext.Provider>
);
}
export function useAuth() {
const ctx = useContext(AuthContext);
if (!ctx) throw new Error("useAuth must be used within AuthProvider");
return ctx;
}
+34
View File
@@ -0,0 +1,34 @@
import { Link } from "react-router-dom";
import { useAuth } from "../auth.jsx";
export default function Layout({ children }) {
const { user, logout } = useAuth();
return (
<div className="min-h-screen bg-base-200">
<header className="navbar border-b border-base-300 bg-base-100 px-4">
<div className="flex-1">
<Link to="/" className="text-lg font-semibold tracking-tight">
Gmail Reader
</Link>
</div>
<div className="flex items-center gap-3">
<span className="hidden text-sm opacity-70 sm:inline">
{user?.email}
</span>
{user?.picture ? (
<div className="avatar">
<div className="w-8 rounded-full">
<img src={user.picture} alt="" referrerPolicy="no-referrer" />
</div>
</div>
) : null}
<button type="button" className="btn btn-ghost btn-sm" onClick={logout}>
Log out
</button>
</div>
</header>
<main className="mx-auto max-w-4xl px-4 py-6">{children}</main>
</div>
);
}
+17
View File
@@ -0,0 +1,17 @@
@import "tailwindcss";
@plugin "daisyui" {
themes: corporate --default, business --prefersdark;
}
html,
body,
#root {
min-height: 100%;
}
.message-body {
white-space: pre-wrap;
word-break: break-word;
font-family: ui-sans-serif, system-ui, sans-serif;
line-height: 1.55;
}
+16
View File
@@ -0,0 +1,16 @@
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { BrowserRouter } from "react-router-dom";
import App from "./App.jsx";
import { AuthProvider } from "./auth.jsx";
import "./index.css";
createRoot(document.getElementById("root")).render(
<StrictMode>
<BrowserRouter>
<AuthProvider>
<App />
</AuthProvider>
</BrowserRouter>
</StrictMode>,
);
+169
View File
@@ -0,0 +1,169 @@
import { useEffect, useState } from "react";
import { Link, useSearchParams } from "react-router-dom";
import { api } from "../api.js";
function formatDate(value) {
if (!value) return "";
const parsed = new Date(value);
if (Number.isNaN(parsed.getTime())) return value;
return parsed.toLocaleString(undefined, {
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
});
}
export default function Inbox() {
const [searchParams, setSearchParams] = useSearchParams();
const q = searchParams.get("q") || "";
const [draft, setDraft] = useState(q);
const [messages, setMessages] = useState([]);
const [nextPageToken, setNextPageToken] = useState(null);
const [loading, setLoading] = useState(true);
const [loadingMore, setLoadingMore] = useState(false);
const [error, setError] = useState(null);
useEffect(() => {
setDraft(q);
}, [q]);
useEffect(() => {
let cancelled = false;
setLoading(true);
setError(null);
const params = new URLSearchParams();
if (q) params.set("q", q);
api(`/api/messages?${params}`)
.then((data) => {
if (cancelled) return;
setMessages(data.messages);
setNextPageToken(data.nextPageToken);
})
.catch((err) => {
if (!cancelled) setError(err.message);
})
.finally(() => {
if (!cancelled) setLoading(false);
});
return () => {
cancelled = true;
};
}, [q]);
function onSearch(event) {
event.preventDefault();
const next = draft.trim();
if (next) setSearchParams({ q: next });
else setSearchParams({});
}
async function loadMore() {
if (!nextPageToken) return;
setLoadingMore(true);
setError(null);
try {
const params = new URLSearchParams({ pageToken: nextPageToken });
if (q) params.set("q", q);
const data = await api(`/api/messages?${params}`);
setMessages((prev) => [...prev, ...data.messages]);
setNextPageToken(data.nextPageToken);
} catch (err) {
setError(err.message);
} finally {
setLoadingMore(false);
}
}
return (
<div className="space-y-4">
<form onSubmit={onSearch} className="join w-full">
<input
className="input join-item input-bordered w-full"
value={draft}
onChange={(e) => setDraft(e.target.value)}
placeholder='Search Gmail, e.g. from:alice subject:invoice'
aria-label="Search mail"
/>
<button type="submit" className="btn btn-primary join-item">
Search
</button>
</form>
{q ? (
<p className="text-sm opacity-70">
Results for <span className="font-mono">{q}</span>{" "}
<button
type="button"
className="link link-hover"
onClick={() => setSearchParams({})}
>
clear
</button>
</p>
) : (
<h2 className="text-sm font-medium opacity-70">Latest mail</h2>
)}
{error ? (
<div role="alert" className="alert alert-error text-sm">
{error}
</div>
) : null}
{loading ? (
<div className="flex justify-center py-16">
<span className="loading loading-spinner loading-lg" />
</div>
) : messages.length === 0 ? (
<div className="card bg-base-100">
<div className="card-body items-center text-center opacity-70">
<p>No messages found.</p>
</div>
</div>
) : (
<ul className="menu rounded-box bg-base-100 p-0 shadow">
{messages.map((message) => (
<li
key={message.id}
className="border-b border-base-200 last:border-b-0"
>
<Link
to={`/messages/${message.id}`}
className="flex flex-col items-stretch gap-1 py-3"
>
<div className="flex items-baseline justify-between gap-3">
<span className="truncate font-medium">{message.from}</span>
<time className="shrink-0 text-xs opacity-60">
{formatDate(message.date)}
</time>
</div>
<span className="truncate">{message.subject}</span>
<span className="line-clamp-1 text-sm opacity-60">
{message.snippet}
</span>
</Link>
</li>
))}
</ul>
)}
{nextPageToken && !loading ? (
<div className="flex justify-center">
<button
type="button"
className="btn btn-outline"
onClick={loadMore}
disabled={loadingMore}
>
{loadingMore ? (
<span className="loading loading-spinner" />
) : (
"Load more"
)}
</button>
</div>
) : null}
</div>
);
}
+35
View File
@@ -0,0 +1,35 @@
import { useSearchParams } from "react-router-dom";
const ERRORS = {
denied: "Google access was denied.",
missing_code: "OAuth callback was missing a code.",
invalid_state: "OAuth state did not match. Try signing in again.",
exchange_failed: "Could not complete Google sign-in.",
};
export default function Login() {
const [params] = useSearchParams();
const error = ERRORS[params.get("error")] || null;
return (
<div className="grid min-h-screen place-items-center bg-base-200 px-4">
<div className="card w-full max-w-md bg-base-100 shadow-xl">
<div className="card-body gap-4">
<h1 className="card-title text-2xl">Gmail Reader</h1>
<p className="text-sm opacity-70">
Sign in with Google to list, search, and read mail. Access is
read-only.
</p>
{error ? (
<div role="alert" className="alert alert-error text-sm">
{error}
</div>
) : null}
<a href="/auth/google" className="btn btn-primary">
Sign in with Google
</a>
</div>
</div>
</div>
);
}
+80
View File
@@ -0,0 +1,80 @@
import { useEffect, useState } from "react";
import { Link, useParams } from "react-router-dom";
import { api } from "../api.js";
function formatDate(value) {
if (!value) return "";
const parsed = new Date(value);
if (Number.isNaN(parsed.getTime())) return value;
return parsed.toLocaleString();
}
export default function Message() {
const { id } = useParams();
const [message, setMessage] = useState(null);
const [error, setError] = useState(null);
const [loading, setLoading] = useState(true);
useEffect(() => {
let cancelled = false;
setLoading(true);
setError(null);
api(`/api/messages/${encodeURIComponent(id)}`)
.then((data) => {
if (!cancelled) setMessage(data);
})
.catch((err) => {
if (!cancelled) setError(err.message);
})
.finally(() => {
if (!cancelled) setLoading(false);
});
return () => {
cancelled = true;
};
}, [id]);
return (
<div className="space-y-4">
<Link to="/" className="btn btn-ghost btn-sm">
← Inbox
</Link>
{loading ? (
<div className="flex justify-center py-16">
<span className="loading loading-spinner loading-lg" />
</div>
) : error ? (
<div role="alert" className="alert alert-error text-sm">
{error}
</div>
) : message ? (
<article className="card bg-base-100 shadow">
<div className="card-body gap-4">
<h1 className="card-title text-xl">{message.subject}</h1>
<dl className="grid gap-1 text-sm">
<div className="flex gap-2">
<dt className="opacity-60">From</dt>
<dd>{message.from}</dd>
</div>
{message.to ? (
<div className="flex gap-2">
<dt className="opacity-60">To</dt>
<dd>{message.to}</dd>
</div>
) : null}
<div className="flex gap-2">
<dt className="opacity-60">Date</dt>
<dd>{formatDate(message.date)}</dd>
</div>
</dl>
<div className="divider my-0" />
<div className="message-body text-sm">
{message.body || message.snippet || "(empty message)"}
</div>
</div>
</article>
) : null}
</div>
);
}
+16
View File
@@ -0,0 +1,16 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
import tailwindcss from "@tailwindcss/vite";
export default defineConfig({
plugins: [react(), tailwindcss()],
server: {
port: 5173,
strictPort: true,
proxy: {
"/api": "http://localhost:3000",
"/auth": "http://localhost:3000",
"/callback": "http://localhost:3000",
},
},
});
+5537
View File
File diff suppressed because it is too large Load Diff
+26
View File
@@ -0,0 +1,26 @@
{
"name": "gmail-summarizer",
"private": true,
"workspaces": [
"apps/*"
],
"scripts": {
"dev": "concurrently -k -n api,web -c cyan,magenta \"npm run dev -w api\" \"npm run dev -w web\"",
"dev:api": "npm run dev -w api",
"dev:web": "npm run dev -w web",
"dev:watcher": "npm run dev -w watcher",
"test": "npm run test -w watcher",
"test:watcher": "npm run test -w watcher",
"process-messages": "npm run process-messages -w watcher --",
"list-messages": "npm run list-messages -w watcher --",
"build": "npm run build -w web",
"start": "npm run start -w api",
"start:watcher": "npm run start -w watcher"
},
"engines": {
"node": ">=20"
},
"devDependencies": {
"concurrently": "^9.2.1"
}
}
+206
View File
@@ -0,0 +1,206 @@
lockfileVersion: '9.0'
settings:
autoInstallPeers: true
excludeLinksFromLockfile: false
importers:
.:
devDependencies:
concurrently:
specifier: ^9.2.1
version: 9.2.4
packages:
ansi-regex@5.0.1:
resolution: {integrity: sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==}
engines: {node: '>=8'}
ansi-styles@4.3.0:
resolution: {integrity: sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==}
engines: {node: '>=8'}
chalk@4.1.2:
resolution: {integrity: sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==}
engines: {node: '>=10'}
cliui@8.0.1:
resolution: {integrity: sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==}
engines: {node: '>=12'}
color-convert@2.0.1:
resolution: {integrity: sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==}
engines: {node: '>=7.0.0'}
color-name@1.1.4:
resolution: {integrity: sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==}
concurrently@9.2.4:
resolution: {integrity: sha512-TZ0CEhyzvFjgtAvHTusDMgj7wNdihCh7LLLrzdUOXIhdlnL2JBBGA9eJxR24rtqgmdjh3OA3hrN1rCHj6HM8qA==}
engines: {node: '>=18'}
hasBin: true
emoji-regex@8.0.0:
resolution: {integrity: sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==}
escalade@3.2.0:
resolution: {integrity: sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==}
engines: {node: '>=6'}
get-caller-file@2.0.5:
resolution: {integrity: sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==}
engines: {node: 6.* || 8.* || >= 10.*}
has-flag@4.0.0:
resolution: {integrity: sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==}
engines: {node: '>=8'}
is-fullwidth-code-point@3.0.0:
resolution: {integrity: sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==}
engines: {node: '>=8'}
require-directory@2.1.1:
resolution: {integrity: sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==}
engines: {node: '>=0.10.0'}
rxjs@7.8.2:
resolution: {integrity: sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==}
shell-quote@1.9.0:
resolution: {integrity: sha512-Iov+JwFv/2HcTpcwNMKd8+IWNb8tboQJNQTkAY/LLVK7gGH9jy+LGkVqPxfekHl+yMmiqXszdGWXgkfml7hjqA==}
engines: {node: '>= 0.4'}
string-width@4.2.3:
resolution: {integrity: sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==}
engines: {node: '>=8'}
strip-ansi@6.0.1:
resolution: {integrity: sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==}
engines: {node: '>=8'}
supports-color@7.2.0:
resolution: {integrity: sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==}
engines: {node: '>=8'}
supports-color@8.1.1:
resolution: {integrity: sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==}
engines: {node: '>=10'}
tree-kill@1.2.2:
resolution: {integrity: sha512-L0Orpi8qGpRG//Nd+H90vFB+3iHnue1zSSGmNOOCh1GLJ7rUKVwV2HvijphGQS2UmhUZewS9VgvxYIdgr+fG1A==}
hasBin: true
tslib@2.8.1:
resolution: {integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==}
wrap-ansi@7.0.0:
resolution: {integrity: sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==}
engines: {node: '>=10'}
y18n@5.0.8:
resolution: {integrity: sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==}
engines: {node: '>=10'}
yargs-parser@21.1.1:
resolution: {integrity: sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==}
engines: {node: '>=12'}
yargs@17.7.2:
resolution: {integrity: sha512-7dSzzRQ++CKnNI/krKnYRV7JKKPUXMEh61soaHKg9mrWEhzFWhFnxPxGl+69cD1Ou63C13NUPCnmIcrvqCuM6w==}
engines: {node: '>=12'}
snapshots:
ansi-regex@5.0.1: {}
ansi-styles@4.3.0:
dependencies:
color-convert: 2.0.1
chalk@4.1.2:
dependencies:
ansi-styles: 4.3.0
supports-color: 7.2.0
cliui@8.0.1:
dependencies:
string-width: 4.2.3
strip-ansi: 6.0.1
wrap-ansi: 7.0.0
color-convert@2.0.1:
dependencies:
color-name: 1.1.4
color-name@1.1.4: {}
concurrently@9.2.4:
dependencies:
chalk: 4.1.2
rxjs: 7.8.2
shell-quote: 1.9.0
supports-color: 8.1.1
tree-kill: 1.2.2
yargs: 17.7.2
emoji-regex@8.0.0: {}
escalade@3.2.0: {}
get-caller-file@2.0.5: {}
has-flag@4.0.0: {}
is-fullwidth-code-point@3.0.0: {}
require-directory@2.1.1: {}
rxjs@7.8.2:
dependencies:
tslib: 2.8.1
shell-quote@1.9.0: {}
string-width@4.2.3:
dependencies:
emoji-regex: 8.0.0
is-fullwidth-code-point: 3.0.0
strip-ansi: 6.0.1
strip-ansi@6.0.1:
dependencies:
ansi-regex: 5.0.1
supports-color@7.2.0:
dependencies:
has-flag: 4.0.0
supports-color@8.1.1:
dependencies:
has-flag: 4.0.0
tree-kill@1.2.2: {}
tslib@2.8.1: {}
wrap-ansi@7.0.0:
dependencies:
ansi-styles: 4.3.0
string-width: 4.2.3
strip-ansi: 6.0.1
y18n@5.0.8: {}
yargs-parser@21.1.1: {}
yargs@17.7.2:
dependencies:
cliui: 8.0.1
escalade: 3.2.0
get-caller-file: 2.0.5
require-directory: 2.1.1
string-width: 4.2.3
y18n: 5.0.8
yargs-parser: 21.1.1