#Pub/Sub

Posts about pub/sub. ← All posts

A2AADKAI GovernanceAIGPAMLAPI DesignAWSAadhaarAccountingAgentsAnomaly DetectionArchitectureAuditAudit LogAzureBCPBankingBedrockBenchmarksBhashiniBigQueryCRAGCareerCase StudyClinical Decision SupportCloud ArchitectureCloud KMSCloud RunComplianceConcurrencyConfigCost OptimisationCryptographyCultureCures ActDSLData ResidencyDatabase DesignDatabase MigrationDatabase SecurityDataflowDatastreamDeploymentDesign PatternDevOpsDevice FlowDistributed SystemsElevenLabsEngineeringEntity ResolutionEnvoyEvaluationFHIRFREE-AIFinOpsFinTechFraudGCPGDPRGKEGOMEMLIMITGSoCGeminiGenieGitHubGoGo 1.23Google CloudGoogle Cloud NextGovernanceGraphQLGraphRAGHIPAAHITLHL7 v2Healthcare ITHyDEIAPPISO 27001IdempotencyIdentity FederationIncident ResponseIndic LanguagesIntegrationJWTKMSKYCKafkaKnowledge GraphKubernetesLLMLLM OpsLatencyLendingLessons LearnedLoggingMARAML EngineeringMemoryMentorshipMicroservicesMiddlewareMigrationMulti-AgentMulti-Agent AIMulti-CloudMulti-LanguageMultilingualNPCINetworkingOAuthOPAOTelObservabilityOpen BankingOpen SourceOpenTelemetryOperationsOperatorsOpinionOrchestrationPAMPCSEPKCEPasskeysPatternsPaymentsPerformancePolicyPolicy as CodePostgreSQLPrivacy EngineeringProductionPrometheusProtocolsProvider AbstractionPub/SubPythonRAGRBACRBIRFC 8693RedisRegulationReliabilityReservationsResilienceRetrievalRetrospectiveSAMLSLOSOC 2SPIFFESPIRESQLSRESagaSaudi ArabiaSchemaSecuritySecurity Command CenterSelf-RAGService MeshSoftware ArchitectureSpannerSpeakingState ManagementStdlibStorageTata GroupTerraformTestingTier PromotionToken BudgetingToolsUAEUPIVertex AIVoice AIVotingWebAuthnWorkflowWorkload IdentityWorkload Identity FederationWritingZero-Trustembed.FSerrgroupgRPCiter.SeqmTLSslog
· Engineering ·9 min read

CDC for minimal-downtime Spanner migration — Datastream + Pub/Sub + Dataflow

A bulk migration takes hours; the application can't be offline that long. CDC keeps the source and destination in sync while the bulk runs, and a quick cutover swaps traffic. The handoff between bulk and CDC is where most migrations go wrong.