New Relic Data Dictionary System
Led development of New Relic's data dictionary system across three major phases, evolving from initial cross-team contributor to lead engineer. Built multi-component infrastructure delivering event and attribute definitions through NerdGraph API, public docs catalog, and in-product UI tooltips—solving critical customer knowledge gaps as the platform scaled from 7 products to 500+ integrations.
The Challenge
As New Relic expanded from 6 language agents and 7 products to 500+ integrations and 30+ capabilities, customers increasingly needed to query telemetry data using NRQL (New Relic Query Language). However, with exponentially more event types, metrics, logs, and traces flowing through the system, customers lacked critical context about what their data represented—its measurements, units, calculations, and meaning. This knowledge gap prevented effective use of core features like dashboards, charts, and query interfaces, directly impacting product adoption and customer success.
Approach
**Phase 1: Initial System (Cross-Team Contributor)** - Collaborated with engineers from multiple teams to design and build initial data dictionary service - Developed API from public docs system that fed product UI tooltips and contextual help - Established foundational data model for storing and delivering definitions across the platform **Phase 2: Robust Rearchitecture (Engineer Co-Leading Work)** - Led technical planning and development after re-org to Developer Experience team - Architected scalable redesign: GitHub Actions syncing content to Postgres, exposed via NerdGraph (New Relic's public GraphQL API) - Rebuilt public data catalog with modern UI, advanced filtering, and improved search using Gatsby and React - Enabled integration across 10+ product surfaces including Query Explorer, Dashboards, and Browser Errors UI **Phase 3: Optimization & Simplification (Manager of Engineering Team overseeing project)** - Led migration from Elixir to Node.js, modernizing tech stack for better team maintainability - Redesigned content architecture: moved definitions from Postgres to MDX files in enterprise GitHub - Implemented API-driven delivery where docs site consumed definitions via NerdGraph - Simplified content management, enabling GitHub PR-based updates instead of database operations
Outcome
- Delivered critical infrastructure serving definitions to 10+ product teams across core user workflows - Reduced operational costs by eliminating expensive Postgres dependency while improving reliability - Enabled self-service data discovery for customers querying complex, multi-source telemetry data - Created scalable content model supporting thousands of definitions with streamlined contribution workflows - Received consistent positive feedback from internal teams and customers as essential for effective data querying