Home Server

Crawl the web from your own machine.

Web Spider is the home server's polite crawler — robots.txt-respecting, rate-limited, JS-rendered when needed. OB1 uses it to research, monitor competitors, scrape public datasets, and refresh the contents of brain folders.

Choose Your Plan Back to overview

Polite web crawler on your home server — robots-aware, rate-limited, JS-rendered when needed.

What it does

Web Spider at a glance.

Robots-respecting

Reads robots.txt before every crawl. Honors crawl-delay. Backs off on rate limits. The kind of crawler you can actually point at production sites.

JS-rendered when needed

Headless Chromium spins up for sites that need JS to render content. Static fetch first; falls back to render only when the static HTML is empty.

Brain-folder destination

Set up a crawl that drops new articles into a brain folder. Folder agent fires Whisper-style summarization. OB1 reads the digest in your daily briefing.

Brave quota wiring

Pair with a Brave Search API key for query-then-crawl flows. Quota tracked + visible in admin so you don't burn through your monthly allotment.

Add Web Spider to your agent.

Every VP3 plan unlocks the full c:// drive catalog. Pick a plan, install the home server, flip the app on.

Choose Your Plan