nsrb/jerapah-flow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(runner): queue workflow runs with BullMQ (phase A)

Browse Source 
Enqueue HTTP, cron, and manual runs via Redis/BullMQ; workers execute
asynchronously with configurable concurrency. Triggers return 202 and
clients poll GET /api/runs/:id for progress (queued → running → done).

Co-authored-by: Nasyarobby Putra <nasyarobby@gmail.com>
This commit is contained in:
Cursor Agent
2026-08-19 04:25:21 +00:00
co-authored by nsrb
parent 93362208a2
commit a95e6d7bc8
14 changed files with 716 additions and 201 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+8 -1
View File
@@ -64,18 +64,25 @@ Optional `script.meta.reads = "ctx"` documents expression hosts. `meta.input` /
| `JFLOW_JWT_SECRET` | `jflow-dev-secret` (dev only) | **Required in production**. |
| `JFLOW_SECRETS_KEY` | `jflow-dev-secrets-key` (dev only) | Master key for named secrets. **Required in production**. Changing it makes existing secrets unreadable. 64 hex chars are used as a raw AES-256 key; any other string is derived with scrypt. |
| `JFLOW_DB_PATH` | `packages/server/data/jerapah-flow.db` | SQLite file. |
| `REDIS_URL` | `redis://127.0.0.1:6379` | Redis for BullMQ workflow queue. **Required** — the server will not start if Redis is unreachable. |
| `JFLOW_QUEUE_NAME` | `jerapah-workflows` | BullMQ queue name. |
| `JFLOW_WORKER_CONCURRENCY` | `5` | Max parallel workflow jobs per worker process. |
| `JFLOW_ROLE` | `all` | `all` (API + cron producer + worker), `api` (HTTP/admin/cron enqueue only), or `worker` (consume queue only). |
| `JFLOW_LOG_LEVEL` | `debug` | Pino level |
| `JFLOW_RETENTION_DAYS` | `30` | Run history prune |
| `JFLOW_CORS_ORIGIN` | `http://localhost:5173` | Vite origin in dev |
| `PORT` | `9000` | HTTP port |
| `NODE_ENV` | — | Set `production` for secure cookies |
Workflow runs are **queued** via BullMQ. HTTP and manual triggers return `202 { runId, status: "queued" }` immediately; poll `GET /api/runs/:id` for progress (`queued` → `running` → `success` \| `failed`). Cron remains an in-process producer that enqueues jobs on each tick.
## Production
```bash
pnpm install
pnpm build
JFLOW_JWT_SECRET=... JFLOW_SECRETS_KEY=... NODE_ENV=production pnpm start
# Redis must be reachable at REDIS_URL
JFLOW_JWT_SECRET=... JFLOW_SECRETS_KEY=... REDIS_URL=redis://127.0.0.1:6379 NODE_ENV=production pnpm start
```
The server serves `packages/web/dist` when that folder exists.