Core Services

Audience: Engineers extending or configuring the deadline engine, QC checks, document generation, or extraction pipeline.
What you will accomplish: Understand the four core services — their contracts, failure modes, and extension points — so you can configure, extend, or debug them.
Prerequisites: Read Workflow Engine first to understand how services are invoked from step handlers.
Estimated time: ~15 minutes.

Core services are the reusable engines shared by all workflows. They live in cam.core.services/ and are injected into workflow handlers through the ServiceContainer.

Deadline Engine

The deadline engine is a liability-critical subsystem. It is treated as a first-class, independently-monitored component with redundant scheduling and a dead-man's-switch.

Rule sets

Rules are data, not code — authored as YAML, versioned by content hash (rule_version = sha256(yaml)[:16]). The registry is append-only; old versions are never deleted.

rule: statute_of_limitations_personal_injury
jurisdiction: US
trigger: incident_date
offset:
  years: 2
adjust: next_business_day
reminders: [-90d, -30d, -7d, -1d]
escalation:
  after_due: notify_supervisor

Computation trace

Every deadline.compute result includes a ComputationTrace that records how the due date was derived:

class ComputationTrace(BaseModel):
    rule_id: str
    rule_version: str       # content hash of the rule definition
    jurisdiction: str
    trigger_date: date
    raw_due: date           # before business-day adjustment
    adjustments: list[str]  # e.g. "skipped Saturday", "skipped US Independence Day"
    final_due: date         # after all adjustments

This trace is persisted in the audit log, so any computed deadline can be independently verified against its rule version.

Key types

class RuleRef(BaseModel):
    rule_id: str
    rule_version: str       # content hash of the rule definition
    jurisdiction: str
 
class ScheduledDeadline(BaseModel):
    id: str
    matter_id: str
    name: str
    due_at: datetime
    rule: RuleRef
    computation: ComputationTrace
    status: Literal["pending", "reminded", "done", "missed", "past_due"]
    escalation_level: int

deadline.schedule returns a ScheduledDeadline — not a bare Deadline — so the caller receives both the rule provenance and the full computation trace in one response.

Business-day computation

All date math goes through the Calendar abstraction. No timedelta(days=n) is applied directly to business-day offsets.

from cam.core.services.deadline.calendar import Calendar
 
cal = Calendar("US")
due = cal.add_business_days(trigger_date, 90)  # skips weekends and US federal holidays

Missing holiday data for a year → loud failure, never a guess.

Reminder arming

Each reminder is armed via two independent scheduler paths sharing one idempotency key. This gives exactly-once delivery even if one scheduler path fails:

• Path A: Celery beat fires fire_reminder task
• Path B: Sweep task finds armed but un-fired reminders and re-arms

Both paths check cam:reminder_fired:{idem_key} in Redis before executing.

Escalation

escalation_level is monotonically non-decreasing. The recipient set only widens:

Past-due + unsatisfied → Deadline.status = missed + safety incident log.

Dead-man's-switch

Runs on an independent timer (not co-scheduled with the reminder scheduler). If the scheduler heartbeat age exceeds the staleness threshold (default 300s), a safety incident is paged immediately.

Past-due-on-create

When deadline.schedule computes a due_at that is already past, the engine:

• Raises PastDueOnCreateError — not silently persisted
• Flags the deadline as past_due but does not arm reminders
• The calling workflow logs the flag and continues (not parked)

This ensures a human sees the flag, but the workflow is not blocked by a deadline that was already tight at creation time.

Reconciliation

A periodic reconciliation sweep compares persisted Deadline records against the case management system's current state:

• Drift detection: changed due dates, closed matters, new deadlines from the case system
• Alert-only: reconciliation never auto-mutates the source of record; it surfaces discrepancies for human triage
• ConnectorError during reconciliation: parks the reconciliation run and alerts operations
• Absence of case data is NEVER treated as "no drift" — missing data is itself a signal

Scheduler abstraction

The deadline engine depends on a Scheduler protocol, not a specific backend. The current implementation uses Celery beat (confirmed in ASSUMPTIONS H-003), but the engine can swap to APScheduler or any backend that satisfies the protocol.

QC Verification

The QC service (cam.core.services.qc) is a pluggable check registry. It runs a set of composable checks against a VerificationPacket and returns a QCReport.

Verdict model

Key invariant: any single fail verdict → aggregate = block.

The seven built-in checks

Privilege check

The privilege check is fail-closed:

• document.privileged=True + external target → always fail
• No approval token can override this
• Unknown privilege + external → fail (default-deny)

This is proven by a Hypothesis property-based test (test_pbt_privilege_never_passes_external) that generates hundreds of (privileged, external) combinations.

Adding a new check

from cam.core.services.qc import register_check, Check, CheckResult
 
class MyCustomCheck(Check):
    id = "my_check"
    version = 1
    applies_to = ["email_send", "document_generate"]
    severity_policy = "warn_allowed"
 
    async def run(self, packet: VerificationPacket, cfg: QCConfig) -> CheckResult:
        # ... validation logic ...
        return CheckResult(status="pass", reason="All good")
 
register_check(MyCustomCheck())

No edits to registry core needed (FR-16). The registry discovers and runs it automatically.

Packet kinds

The VerificationPacket carries a kind field that determines which checks apply:

class VerificationPacket(BaseModel):
    kind: PacketKind
    template_bindings: list[TemplateBinding]   # template var → value bindings
    attachments: list[AttachmentRef]            # document references with checksums
    external_bound: bool                       # any recipient is external?
    now: datetime                              # reference timestamp
    config_snapshot: QCConfig                  # frozen config at verification time
    config_fingerprint: str                    # sha256 of config_snapshot JSON

The config_snapshot is frozen at verification time and its config_fingerprint is stored in the QCReport. This means any QC result can be traced back to the exact configuration that produced it — essential for audit and reproducibility.

Totality invariant

Every selected check either returns a CheckResult or a skipped entry with a SkipReason (not_applicable, field_absent, disabled, unsupported_packet_kind). No check can silently pass or be omitted. The QCReport lists both checks_selected and checks_skipped with reasons.

Severity policy enforcement

Each check declares which verdicts it may emit via severity_policy. A check configured as fail_only (like privilege) cannot downgrade to warn. The registry enforces this at runtime — a misconfigured check that attempts an invalid verdict is treated as a fail. This prevents accidental downgrading of safety-critical checks.

Configuration

Document Generation

The document generation pipeline (cam.core.workflows.document_gen) produces DOCX and PDF artefacts from Jinja2 templates and matter/contact data.

Pipeline

template (DOCX/HTML, Jinja2 vars)
   + matter/contact data (domain model)
        │
        ▼
   render (docxtpl / Jinja2) ──► DOCX
        │                          └─► WeasyPrint / LibreOffice ──► PDF
        ▼
   QC.verify (all vars resolved? names/dates consistent?)
        ▼
   store (object store / doc connector) ──► Document(version, checksum)
        ▼
   route (document.route) ──► correct matter/folder + ACL

Gap placeholders

Unresolved required variables → [[MISSING: label]] token in output. Never an empty string (FR-9). The human receives a concrete artefact with explicit gaps to fix.

Status derivation

• gaps present → incomplete
• QC fail → blocked
• QC skipped (unavailable) → incomplete
• QC pass/warn + no gaps → ready
• Render/store error → failed

Versioning

Versions are monotonic per (matter_id, template_name, doc_key):

doc = Document(
    id="doc_abc",
    matter_id="mat_123",
    name="Engagement Letter",
    version=3,
    checksum="sha256:abc...",
)

Old versions are immutable. Nothing overwrites silently.

Data Extraction

The extraction pipeline (cam.core.services.extraction) turns unstructured documents into structured domain-model fields.

Text layer

• Native PDFs: pdfplumber / PyMuPDF for text extraction
• Scans/images: Tesseract OCR fallback
• Structured forms: Direct field mapping where templates are known

Structuring

An LLM (via LLMStructuringClient) maps raw text → domain model using the extraction_schema prompt. Returns per-field confidence scores.

fields = await extractor.extract(
    document_bytes,
    profile=MappingProfile(
        fields=[
            ExtractedField(name="full_name", type="str", required=True),
            ExtractedField(name="date_of_birth", type="date", required=True),
            ExtractedField(name="a_number", type="str", pattern=r"A\d{8,9}"),
        ]
    )
)

Extraction source tracking

Each extracted field records its provenance via ExtractionSource:

class ExtractionSource(str, Enum):
    TEXT = "text"          # native PDF text extraction
    OCR = "ocr"            # Tesseract OCR fallback
    LLM = "llm"            # LLM structuring path
    FORM_MAP = "form_map"  # structured form field mapping

Every ExtractedField carries a source: ExtractionSource so downstream consumers know exactly how the value was obtained. This drives both the deterministic flag and QC extraction_confidence check behaviour.

Low-confidence gate

Fields with confidence < threshold (default 0.80) are flagged for human verification before they enter a system of record. This feeds into the QC extraction_confidence check.

Page-coverage invariant

Every extraction produces a PageCoverage report:

class PageCoverage(BaseModel):
    total_pages: int
    text_pages: list[int]        # native text extracted
    ocr_pages: list[int]         # OCR fallback applied
    skipped_pages: list[int]     # explicitly unprocessable, surfaced
    # INVARIANT: sorted(text ∪ ocr ∪ skipped) == [1..total_pages] (disjoint)

No page is silently dropped. If a page cannot be processed, it appears in skipped_pages with a reason. The caller knows exactly what was covered and what was not.

Determinism flag

The ExtractionProposal carries a deterministic: bool flag:

• True — only text/OCR paths were used; output is deterministic and idempotent
• False — the LLM structuring path was invoked; output may vary between runs

This flag enables downstream consumers (QC checks, workflows) to adjust their behaviour based on extraction reliability.

Residency guard in detail

The ResidencyGuard wraps the LLMStructuringClient and performs these checks before any inference call:

1. Is CAM_EXTRACTION_ALLOW_EXTERNAL_INFERENCE set to true?
2. Is the target endpoint in EXTRACTION_RESIDENCY_ALLOWLIST?
3. Does the request data satisfy the configured residency region?

If any check fails, the guard blocks the call and the pipeline falls back to rule-based / regex extraction. The guard logs the block reason but never crashes the pipeline.

Extraction configuration

Residency guard

If CAM_EXTRACTION_ALLOW_EXTERNAL_INFERENCE=false (default), the ResidencyGuard blocks any call that would send client data to an external LLM provider. The extraction pipeline falls back to rule-based / regex extraction.

Service container

All core services are accessed through a ServiceContainer that is injected into workflow handlers:

@dataclass
class ServiceContainer:
    audit: AuditService
    deadline: DeadlineService
    qc: QCService
    extraction: ExtractionService
    document_gen: DocumentGenerationService
    notification: NotificationService
    connectors: ConnectorRegistry

The container is created once at startup and passed to register_intake_workflow(services), register_status_update_workflow(services), etc. This replaces the deprecated module-level _services singleton pattern.

Common mistakes

Verification

After reading this page, verify your understanding:

1. Explain what happens when the deadline engine encounters missing holiday data for a year
2. Describe the QC verdict model: what single verdict causes an aggregate block?
3. Show how to add a custom QC check without editing registry core
4. Explain what [[MISSING: label]] means in a generated document

Next steps

• Document Generation Workflow — End-to-end template → DOCX/PDF
• Document Routing Workflow — Classification, filing, and privilege gates
• Client Intake Workflow — Extraction in practice: PDF → structured matter
• Connector Layer — How services interact with external systems

Last updated: 2026-06-01