Lobster Forge Pipeline
name: SPEC_LOBSTER_FORGE_PIPELINE
description: Lobster Autonomous Forge Pipeline — SPECIFIED ✓; corpus→GCS→Colab T4→DONE.json→adapter→GGUF→Ollama→smoke→promote→chain; GPU quota handling; abort gate; VELA α.13 2026-04-20
type: project
SPEC — Lobster Autonomous Forge Pipeline
Status: SPECIFIED ✓
Author: VELA #13 (⊹.VELA α.13)
Date: 2026-04-20
Authored under: ⊢.α.13.⊹.VELA
PURPOSE
The Lobster forges brains autonomously by chaining: upload corpus to GCS → launch Colab notebook → monitor for DONE.json → download adapter → convert LoRA to GGUF → create Ollama model → run smoke test → report results → chain next brain. Proven today with ANVIL → ORPHEUS → CHROMA → DR.LOGOS.
PIPELINE STEPS
- Pre-upload all corpora to GCS (
colab_jobs/[brain]_forge_input/) - Trigger Colab T4 notebook via
colab_dispatch.py - Monitor GCS for
DONE.json(poll every 60s) - On DONE.json: download adapter files from GCS
- Convert LoRA adapter to GGUF (using
llama.cpp convert-lora-to-gguf) - Create Ollama model:
ollama create [brain] -f ~/Modelfile.[brain] - Run smoke test:
python3 ~/smoke_[brain].py(5 tests, 3/5 minimum to promote) - Score:
- If 3/5+: promote to [brain]:latest.
- If <3/5: log failure, queue for reforge with corpus expansion.
- Chain: proceed to next brain in queue.
ABORT GATE
If val_loss hasn't improved by epoch 7, abort forge early. Don't waste GPU quota on a diverging run.
GPU QUOTA
Colab free tier has rolling quotas. After 2–3 forges, expect a quota wall (4–8 hour cooldown). Set wakeup alarm and resume automatically. Don't block the pipeline — move to non-GPU tasks while waiting.
CHAINING RULES
- Only one forge runs at a time (single Colab session)
- Failed smoke → log, expand corpus, queue reforge at END of chain (don't block others)
- Successful smoke → promote, evict old model from Ollama if RAM tight
- Each brain independently assessed — one failure doesn't block others
SMOKE TEST STANDARD
5 questions per brain:
- T1 — Identity
- T2 — Governance refusal
- T3 — Domain knowledge
- T4 — Complex reasoning
- T5 — Edge case
Minimum: 3/5. Captain rules on borderline cases.
CURRENT QUEUE
| Brain | Status |
|-------|--------|
| ANVIL | ✗ 3/5 — corpus v3 ready, queued for reforge |
| ORPHEUS | ✓ PROMOTED orpheus:v1 (5/5) |
| CHROMA | ⏸ Forging — GPU quota pause 18:28 ET, resume 19:30 ET |
| DR.LOGOS | ⬜ Queued after CHROMA |
INVARIANTS
- Never promote a brain that scores <3/5 on smoke test
- Never skip the smoke test
- Never run two forges simultaneously on Colab free tier
- Log every forge: brain name, corpus size, epochs, final loss, smoke score, promote/fail
- Captain approves all promotions — Lobster recommends, Captain decides
Jeremy Zlabis / Chronogeometer · Visionary · Disruptor · Chief / 42 Sisters AI · East York, Toronto / 🍁 Φ 0.042