Skip to content
All work

Case study. R / Shiny / RAG.

DevSight

Hybrid-RAG deviation investigation assistant

Internal enterprise application, built at Takeda Pharmaceutical

Role
Full-stack: indexing pipeline, two-stage RAG, app
Status
Decommissioned 2026final v2.7.0
Context
Takeda LA
Search-time reduction
93%

~30 min to ~2 min per query, measured in production

Self-reported satisfaction
4.8/5

Self-reported by users

Resolved cases indexed
2,213+

Four years of records, 2022 to 2026

Searchable chunks
31,577

Two-stage hybrid RAG index

DevSight — 2,213 records indexed as 31,577 chunks, metadata filter then semantic rank, ~30 minutes to ~2 minutes per search; decommissioned 2026, search carried into the Horizon proof of concept

The problem

Investigating a new deviation meant manually keyword-searching four years of Trackwise records — roughly 30 minutes per question, and brittle: the right precedent was easy to miss if it used different wording. Investigators needed semantic recall over 2,213+ resolved cases without leaving the 30-day closure window.

Architecture

  1. 1.Deviation source
    • 4 yrs deviations
    • 2,213+ cases
    • 10 fields
  2. 2.Index
    • Chunk → 31,577 units
    • Embed
    • Databricks Vector Search
  3. 3.Stage 1 — Filter
    • Date metadata
    • Equipment pre-filter
  4. 4.Stage 2 — Semantic
    • Vector rank
    • Post-filter client-side
  5. 5.Answer
    • Streamed via ellmer
    • Cites real cases
    • 10-turn memory
DevSight two-stage hybrid-RAG pipelineR computes any statistics; the LLM only interprets — no invented numbers.

How it is kept honest

Two-stage hybrid RAG: a date-metadata filter narrows the candidate set, then semantic vector search ranks across ten record fields. The assistant answers from retrieved records and streams the synthesis; it does not derive new figures.

Guardrails
  • Date filtering applied client-side after the vector search
  • Conversation memory limited to the last 10 turns
  • Document upload limited to Word and Excel on purpose, to preserve streaming
How correctness is checked
Search time was measured in production at roughly 30 minutes down to 2 per deviation query; satisfaction was self-reported at 4.8/5.
Where it can still be wrong
  • Recall depends on the index: 2,213 records as 31,577 chunks, 2022 to 2026; a precedent outside that window is not found
  • Decommissioned in 2026 during a global application consolidation; the search lives on in the Horizon proof of concept and in BioTrack

Reliability and scale

  • The final v2.7.0 release removed roughly 2,200 net lines while adding dark mode, an accessibility pass and a tokenized component library
  • The vector index is shared: BioTrack reuses it for a search scoped to bioburden investigations

What it did

DevSight (formerly TrackLA) was a hybrid-RAG assistant over four years of Trackwise deviations, 2022–2026: 2,213+ resolved cases chunked into 31,577 searchable units. It replaced ~30-minute manual keyword searches with ~2-minute semantic ones — a 93% time reduction — and surfaced the actual precedent cases behind every answer, so an investigator could check the source instead of trusting the summary.

Two-stage hybrid RAG

Search ran in two stages: a date-metadata filter narrowed the candidate set, then semantic vector search ranked across 10 Trackwise fields. Date filtering happened post-search, client-side — a deliberate workaround for Databricks Vector Search not handling DATE range operators cleanly.

  • Conversation memory across the last 10 turns
  • Real-time token streaming with voice input and read-aloud
  • Document upload limited to Word and Excel — image and PDF disabled on purpose, to preserve streaming
  • Mobile-optimized with a fixed input bar, for use away from a desk

The final release

v2.7.0 was a design-system rewrite that removed roughly 2,200 net lines while adding dark mode, an accessibility pass and a tokenized component library — the codebase got smaller as capability grew. It also introduced a single APP_VERSION constant as the source of truth for every version string, and self-hosted fonts because the plant floor blocks CDNs.

Why it was decommissioned

DevSight was decommissioned in 2026 during a global application consolidation: the site moved onto standardized global applications, and locally built tools in that space were wound down regardless of how they were performing. That was a portfolio decision, not a verdict on this application. At the time it held a self-reported 4.8/5 satisfaction score, was still delivering the search-time reduction it launched with, and had shipped its largest release — v2.7.0, the final one — in its last month of operation.

What survived

The capability outlived the app. On 2026-08-14 the general-purpose search was merged into the Horizon v1.2.0 proof of concept as pure R utilities plus one Shiny module with its own tests, and BioTrack uses the same vector index for a search scoped to bioburden investigations. Retiring an application and retiring a capability are different decisions, and only the first one was made.

Stack

  • R Shiny
  • Databricks Vector Search
  • ellmer
  • Hybrid RAG

Related work