Purl

Save anything. Ask questions. Get answers.

Live preview: https://purl.nublson.com

Purl is an AI-powered read-it-later app and personal knowledge base. You paste URLs (or upload files): web pages, PDFs, YouTube videos, and audio. Purl ingests the content, stores chunked text with vector embeddings, and answers questions by searching what you saved — optionally scoped with @ mentions to specific items.

Plans: Free, Pro, and BYOK are enforced server-side (see docs/commercial-model.md). New signups get a 7-day Pro trial (no card required). Stripe Checkout handles the one-time Pro payment; webhooks sync status to Postgres.

The product goal: one place to stash material you care about, then query it later with citations instead of digging through bookmarks.

Plans (summary)

Exact limits are in docs/commercial-model.md.

Implemented today

• Marketing site — Landing page with hero, features, supported content types, pricing section, and FAQ.
• Authentication — Email/password (and related flows) via Better Auth; optional email verification through Resend.
• Save & organize
  • Add items by URL with automatic content-type detection (web, PDF, YouTube, audio).
  • File upload for PDF and audio (size limits enforced server-side).
  • Links grouped by relative time (e.g. Today, This Week, Last Month).
  • Preview metadata (title, description, favicon, thumbnail where available).
• Ingestion pipeline — Fetches or extracts text (including transcripts for YouTube/audio), chunks it, embeds via Vercel AI Gateway (openai/text-embedding-3-small), stores in Postgres with pgvector; tracks per-link ingest status (pending, processing, completed, failed, skipped for edge cases like heavy SPAs).
• AI providers — Vercel AI Gateway for streaming chat (Claude) and embeddings (openai/text-embedding-3-small). OpenAI directly for Whisper transcription only (OPENAI_API_KEY). Keys live in server environment variables only.
• AI Gateway observability — Chat, ingest embeddings, and the chat tool’s semantic search send providerOptions.gateway with the signed-in user id and tags so the Vercel AI dashboard can filter spend and usage by person and surface (feature:chat, env:… from VERCEL_ENV / NODE_ENV; feature:ingest on save pipelines; feature:semantic-search when the model runs vector search over saved chunks).
• Hardened outbound fetch — Server-side safeFetch with optional proxy/DNS controls (see AGENTS.md). For reliable YouTube transcripts on Vercel, configure SAFE_OUTBOUND_HTTP_PROXY in production.
• Realtime list sync — Supabase Realtime so saves and updates propagate across tabs/devices quickly.
• AI chat
  • Streaming replies (Anthropic Claude) with tool use: list saved items (filters by date/type) and search over stored chunks.
  • @ mentions to focus the model on specific saved links; mentions persist on messages.
  • Multiple chats, titles, and message history stored in the database.
• Link actions — Open original, copy URL, edit metadata, re-ingest, delete, add to chat context from the list.
• Operational extras — Optional Upstash-backed API rate limiting, optional Sentry, Vitest coverage for critical paths.
• PWA (installable app) — Web App Manifest plus a Serwist service worker (src/app/sw.ts) that builds to public/sw.js (generated on pnpm build, gitignored). Enables Install in Chrome/Edge and similar where the platform supports it, with runtime caching via Serwist's Next.js defaults and a static offline shell at /~offline. Serwist is disabled in pnpm dev to avoid service-worker cache surprises during development — use pnpm build && pnpm start (or your production URL) to exercise installability and the SW.

Ingestion flow

Saving a link is synchronous through metadata resolution and the database row; heavy work runs afterward so the API can return quickly.

1. Input — POST /api/links with a URL, or POST /api/upload with a PDF/audio file (files go to Supabase Storage; the Link stores the public URL).
2. Classify & decorate — Server-side detectContentType (SSRF-safe HEAD / sniff) plus scrapeLinkMetadata (Open Graph HTML, PDF Content-Disposition / size, YouTube oEmbed). Duplicates of the same URL refresh metadata and reset ingestion.
3. Persist — A Link row is created (default PENDING) with title, favicon, thumbnail, domain, and contentType (WEB, PDF, YOUTUBE, or AUDIO).
4. Schedule — prepareIngestForLink enforces plan limits, then uses Next.js after() to run the right handler: ingestWeb, ingestPdf, ingestYoutube, or ingestAudio. Free accounts skip extraction (metadata-only; ingest SKIPPED).
5. Pipeline (each handler) — Set PROCESSING → fetch or extract plain text → split into chunks (with a synthetic metadata chunk first) → Vercel AI Gateway embeddings (openai/text-embedding-3-small) → replace LinkContent rows and attach pgvector values → COMPLETED. Failures set ingestFailureReason (SCRAPE_FAILED, LINK_NOT_FOUND, OTHER, etc.) alongside FAILED. Re-ingest reuses the same pipeline without re-scraping listing metadata.

Web pages (WEB). Article-style HTML is fetched with safeFetch, parsed in jsdom, and the main content is extracted with Mozilla's Readability (scrapeWebContent). That matches how Firefox's reader mode chooses "the article," but it is not universal: many SPAs and other client-rendered sites return a thin HTML shell to crawlers, so Readability finds little or nothing and ingest may FAIL. A small set of hosts that need a full browser are rejected early (UnsupportedSpaError → ingest SKIPPED).

Realtime subscribers get updates when ingestion finishes via notifyLinksAfterIngest (which calls broadcastLinksChanged).

flowchart TB
  subgraph save["Save path (responds to client)"]
    A(["URL or file upload"]) --> B["/api/links or /api/upload"]
    B --> C["detectContentType + scrapeLinkMetadata (safeFetch)"]
    C --> D["Insert Link — ingestStatus PENDING"]
    D --> E["after() → prepareIngestForLink by contentType"]
  end

  subgraph work["Background ingest"]
    E --> F["ingestStatus PROCESSING"]
    F --> G{"Extract text"}
    G --> W["WEB — jsdom + Mozilla Readability"]
    G --> P["PDF — page text"]
    G --> Y["YOUTUBE — transcript"]
    G --> A2["AUDIO — transcription"]
    W --> H["Chunk + metadata header"]
    P --> H
    Y --> H
    A2 --> H
    H --> I["Gateway embeddings"]
    I --> J["Write LinkContent + pgvector"]
    J --> K{"Outcome"}
    K --> K1["COMPLETED"]
    K --> K2["FAILED (+ ingestFailureReason)"]
    K --> K3["SKIPPED (known SPA hosts)"]
  end

  K1 --> R["Realtime: list refresh"]
  K2 --> R
  K3 --> R

Not implemented yet

These are called out explicitly because the repo is going public:

• Settings breadth — Settings include account deletion; broader account preferences (profile edits, password change, notification settings, etc.) are not implemented yet.

Marketing vs. product: The landing page copy mentions ideas such as collections and a weekly digest. Those are not built in the current schema or app — treat them as roadmap, not shipped features.

Tech stack

• Web: Next.js (App Router), React, TypeScript
• UI: Tailwind CSS, shadcn/ui
• Auth: Better Auth
• Database: PostgreSQL + Prisma (with vector column for embeddings)
• AI: Vercel AI Gateway (Claude chat + OpenAI embeddings through gateway) and OpenAI (Whisper transcription direct) via the Vercel AI SDK — server environment variables
• Email (optional in dev): Resend for verification emails
• Realtime: Supabase client (anon + service role on server)
• PWA: Serwist (@serwist/next), web manifest + precache / offline fallback

CI / GitHub Actions

Automation lives under .github/workflows/. Every PR and manual release is gated by these pipelines.

PR checks — pr-checks.yml