Batch AI Task Extraction From Email

Manual inbox triage does not scale on release weeks. AI extraction promises to read mail and surface action items — but one false positive per day trains users to ignore the feature. One missed email trains them to leave.

Our email extraction pipeline batches messages to the LLM, maps results back to real message IDs, dedupes on persist, and only advances mail sync cursors after the full run succeeds. This post is about the extraction layer specifically — not fetch, not cursor commit (see sync cursors and AI pipelines).

The triage tax

Action items hide in polite mid-paragraph asks, forwarded threads with no owner, and calendar invites with implicit prep. Engineers already have Jira for sprint work and Slack for urgent pings. Email is the leak.

Extraction should propose tasks with title, priority, optional due date, and a link back to the original thread — not silently create fifty rows while the user sleeps.

Why batch, not per-email

We chunk fetched mail into batches of ten and call extractTasksFromEmailsBatch once per chunk. Sequential per-email calls multiply latency, cold-start cost, and rate-limit surface area.

Batches log index and total (batch 2/5) so support can correlate timeouts with payload size. If one batch throws, the entire request fails and mail cursors do not advance — the user gets a retry, not silent loss.

Temp IDs and conservative extraction

Gmail and Outlook message IDs are long opaque strings. Models corrupt them. In the batch prompt we assign email_1, email_2, … and map back after parse:

typescript

// Prompt uses: ID: email_1, Subject: ..., Preview: ...
// Response: { "email_1": { tasks: [...], confidence: 0.82 } }
// Map email_1 → realMessages[0].id before create

The prompt instructs conservative extraction: ignore newsletters, promos, automated shipping notifications, and casual threads without clear asks. Confidence below 0.6 drops the email from results. False negatives beat invented tasks.

We extract from bodyPreview / snippet only today — not full MIME bodies. Tasks buried in attachments or trimmed HTML are a known gap.

Task rows with source links

Created tasks use source: EMAIL_AI, sourceId set to the email ID, and url pointing to Gmail htmlLink or Outlook webLink.sourceData JSON stores subject, from, provider, confidence, and extraction timestamp for audit.

Dedupe before insert: same user, same email ID, same title → skip. Retries after cursor deferral re-hit the same mail safely.

Consent and rate limits

Mail content leaves the VPC only after requireAIConsent passes. Without opt-in, the API returns AI_CONSENT_REQUIRED — not a silent skip.

Rate limit: six extractions per minute per user. Enough for manual triggers; blocks runaway loops from buggy clients. Return retryAfterSeconds on 429.