#Data Quality

Articles about Data Quality — exploring patterns, best practices, and real-world implementations in production systems.

2 posts tagged with data quality. ← All posts

#A/B Testing (3)#A2A (14)#ADK (8)#AG-UI (6)#AI (9)#AI Agents (335)#AI Architecture (22)#AI Cost (10)#AI Cost Optimization (8)#AI Engineering (282)#AI Evaluation (9)#AI Financial Engineering (14)#AI Gateway (8)#AI Governance (40)#AI Red Teaming (9)#AI Research (9)#AI Safety (12)#AI Security (29)#AI in Production (12)#AML (3)#API Design (10)#API Security (8)#APIs (59)#AWS (17)#Accountability (8)#Accounting (9)#Agent Skills (11)#Agentic AI (25)#Agentic Commerce (12)#Agentic RAG (8)#Agents (4)#Amazon Bedrock (8)#Analytics (3)#Architecture (42)#Attention (3)#Audit (4)#Authentication (11)#Authorization (3)#Automation (11)#Azure (11)#Azure AI Foundry (9)#Backend Engineering (310)#Benchmarks (3)#Best Practices (4)#BigQuery (6)#Business Finance (8)#Business Strategy (55)#C (8)#CI/CD (16)#Caching (11)#Capital Markets (14)#Card Payments (12)#Cards (13)#Career (25)#Causal Inference (8)#Cellular Automata (8)#Checkpointing (4)#Claude Code (8)#Cloud (5)#Cloud Architecture (3)#Cloud Native (10)#Code Review (9)#Coding Agents (8)#Collaboration (5)#Communication (9)#Complexity (8)#Compliance (54)#Computer Networking (9)#Computer Science (40)#Computer Vision (15)#Computer-Use Agents (8)#Concurrency (40)#Consulting (5)#Containers (10)#Context Engineering (14)#Conversational AI (8)#Cost Optimisation (5)#Cost Optimization (3)#Credit (16)#Credit Risk (16)#CrewAI (8)#Crypto (12)#Cryptocurrency (12)#Cryptography (9)#Custody (9)#DSPy (8)#Data (14)#Data Engineering (24)#Data Quality (2)#Data Science (8)#Data Structures (10)#Databases (41)#Deep Learning (20)#Deployment (5)#Design (3)#Design Patterns (10)#DevOps (24)#DevSecOps (22)#Developer Experience (5)#Developer Tools (5)#Differential Privacy (3)#Distributed Systems (100)#Documentation (3)#Edge AI (8)#Embeddings (21)#Emotional Intelligence (8)#Energy (8)#Engineering (14)#Engineering Culture (3)#Engineering Judgment (7)#Engineering Practices (16)#Error Handling (4)#Evaluation (68)#Event-Driven Architecture (8)#Explainable AI (8)#FREE-AI (8)#FX (5)#Feature Engineering (3)#Feedback (4)#FinOps (23)#FinTech (6)#Financial AI (14)#Financial Engineering (8)#Financial Systems (129)#Fine-Tuning (12)#Fintech (131)#Flutter (8)#Forecasting (10)#Foreign Exchange (5)#Forward Deployed Engineer (22)#Forward Deployed Engineering (8)#Fraud (10)#Function Tools (5)#Functional Programming (3)#Fundamentals (23)#Fundraising (8)#GCP (5)#GPU (8)#GUI Automation (8)#Gemma (4)#Generative AI (3)#Git (8)#Go (220)#Go-to-Market (13)#Google ADK (36)#Governance (61)#Granite (6)#Graph Neural Networks (8)#GraphQL (3)#Growth (3)#Guardrails (34)#HIPAA (3)#HTTP (3)#Harness Engineering (8)#Hiring (8)#Hugging Face (8)#Human-in-the-Loop (9)#IBM watsonx (8)#Identity (11)#Inference (8)#Information Retrieval (8)#Integration (3)#Intellectual Property (8)#Interfaces (3)#Interpretability (8)#JavaScript (8)#KYC (11)#KYC and AML (12)#Kafka (11)#Knowledge Graphs (10)#Kubernetes (17)#LLM (5)#LLM Inference (8)#LLM Infrastructure (8)#LLM-as-Judge (3)#LLMOps (8)#LLMs (192)#LangChain (8)#LangGraph (11)#Leadership (29)#Ledger (12)#Legal (8)#Lending (14)#Linux (9)#LlamaIndex (8)#Load Balancing (3)#MCP (22)#MLOps (36)#Machine Learning (109)#Marketing (16)#Markets (4)#Mathematical Finance (8)#Memory (15)#Memory Management (5)#Metrics (7)#Microservices (3)#Microsoft Agent Framework (150)#Middleware (6)#Migration (9)#Mixture of Experts (5)#Model Risk (3)#Monitoring (5)#Multi-Agent (10)#Multi-Agent AI (14)#Multi-Agent Systems (73)#Multimodal (3)#Multimodal AI (11)#NIM (5)#NVIDIA (8)#Networking (3)#OAuth (3)#OWASP (7)#Observability (50)#On-Device AI (8)#Ontology (8)#Open Source (7)#OpenTelemetry (5)#Operating Systems (9)#Operations (13)#Opinion (6)#Orchestration (10)#Organizational Design (8)#Payment Rails (16)#Payments (54)#People (8)#Performance (56)#Personalization (9)#Platform Engineering (9)#Policy as Code (3)#PreSales (8)#Privacy (13)#Privacy Engineering (3)#Process (4)#Product (30)#Product Management (8)#Production (22)#Programming (10)#Programming Languages (48)#Prompt Engineering (80)#Prompt Injection (15)#Protocol Buffers (3)#Protocols (9)#Providers (4)#Pydantic AI (8)#Python (142)#Quality (3)#Quantitative Finance (22)#RAG (70)#RBI (3)#REST (5)#Rails (16)#Real-Time (8)#Reasoning Models (8)#Recommender Systems (8)#Reconciliation (3)#RegTech (8)#Regulation (10)#Reinforcement Learning (9)#Reliability (56)#Resilience (4)#Responsible AI (5)#Retrieval (3)#Risk (14)#Risk Management (11)#Rust (32)#SLSA (3)#SRE (23)#Sales (9)#Scalability (6)#Search (10)#Security (102)#Security Engineering (8)#Self-Evolving Agents (16)#Semantic Web (8)#Sessions (3)#Settlement (9)#Soft Skills (8)#Software (3)#Software Architecture (36)#Software Delivery (9)#Software Engineering (152)#Spanner (4)#Speech (8)#Startups (30)#Statistics (8)#Strands (8)#Strategy (6)#Stream Processing (8)#Streaming (36)#Structured Output (4)#Supply Chain Security (9)#Sustainability (8)#System Design (32)#Systems Programming (56)#Testing (66)#Threat Modeling (3)#Time Series (9)#Tool Use (23)#Tooling (5)#Tools (4)#Trading (8)#Treasury (6)#Trust (3)#Type System (3)#Type Systems (11)#TypeScript (8)#Validation (3)#Valuation (3)#Vector Databases (22)#Vector Search (11)#Venture Capital (8)#Version Control (9)#Voice AI (9)#Web Development (6)#Workflow (4)#Workflows (14)#eBPF (8)#gRPC (13)#smolagents (8)
Pratik Dhanave · ·7 min read

Building a Knowledge Graph — Construction and Entity Resolution

The hardest part of a knowledge graph isn't querying it or designing the ontology — it's building it from messy, real-world data. Most of your knowledge lives in databases that don't connect, documents that don't parse, and records that refer to the same thing by different names. Construction is the engineering of turning that mess into a clean, connected graph, and its central, unglamorous challenge is entity resolution: deciding when two records are actually the same thing. This post is about the work that makes or breaks a knowledge graph.

The hard part isn't the ontology or the queries — it's construction. The pipeline from heterogeneous sources, and its make-or-break challenge: entity resolution (deciding when two records are the same thing). Under-merging fragments; over-merging asserts false connections; connectivity amplifies every error.

#Knowledge Graphs#Entity Resolution#Data Engineering#Data Quality#Ontology#Semantic Web
Pratik Dhanave · ·8 min read

Data Quality and Governance

The most dangerous failure in data engineering is the one that doesn't announce itself. A crashed pipeline gets noticed; silently wrong data flows straight into dashboards and models, where people trust it and make decisions on it — wrong decisions, confidently made. Data quality and governance are the disciplines that guard against this: ensuring data is correct, trustworthy, and well-managed. They're the least glamorous and most important part of production data engineering, because without them, all the pipelines and warehouses just deliver garbage efficiently.

The most dangerous failure in data engineering is the one that doesn't announce itself. A crashed pipeline gets noticed; silently wrong data flows straight into dashboards and models, where people trust it and make wrong decisions confidently. Data quality and governance are the disciplines that guard against this.

#Data Engineering#Data Governance#Data Quality#Backend Engineering#Data

All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse all posts or explore related topics using the tag cloud above.