What I learned from a month with WalShadow on ClickHouse Managed Postgres
Hidehiko ASAHI shares a month of WalShadow private-preview testing, comparing replication latency with ClickPipes and checking data correctness, schema changes, and recovery.
Search headlines, summaries and articles to connect the dots.
Hidehiko ASAHI shares a month of WalShadow private-preview testing, comparing replication latency with ClickPipes and checking data correctness, schema changes, and recovery.
Infinitas Learning runs transactions for 1.6 million learners on ClickHouse Managed Postgres and serves real-time analytics to teachers from ClickHouse Cloud, where distributed cache cut P99 latency from about 13 seconds to 410 milliseconds.
Promoting a standby doesn't stop the old Postgres primary from taking writes. Here's how the control plane behind ClickHouse Managed Postgres fences it on bare metal, AWS and GCP, and why the design is fail-open.
Frontier AI models that uncover software flaws by the hundreds are forcing database providers into larger, faster patch cycles, and real application testing is becoming the safeguard that keeps those fixes from breaking production systems. The same AI wave is also undermining security rules written into applications. Oracle Corp. has responded on two fronts, hardening […] The post Oracle connects AI-driven testing with a new approach to database security appeared first on SiliconANGLE.
arXiv:2611.04092v1 Announce Type: new Abstract: Capturing an image in a low-light environment produces a weak, noisy sensor signal. Low-light image enhancement (LLIE) aims to restore it. Auto white balance (AWB) is also required for every image, and the same noisy signal degrades its ability to infer the scene illuminant. LLIE is typically performed first, followed by AWB, but this strategy remains largely unexamined because existing low-light datasets span a narrow range of illuminants and lack illuminant ground truth for evaluating AWB. We address this with CHROMA-MILL, a RAW dataset of 50 scenes captured with a DSLR and smartphone under four calibrated illuminants at eleven intensity levels. Because light is additive, we blend these captures in the linear RAW domain to synthesize physically valid illuminant mixtures with exact chromaticity ground truth. Benchmarking on this data, we find joint correction is best, and introduce CTI-Retinexformer, which supervises an illumination map, a chromaticity prediction, and an illumination-invariant scene embedding, outperforming the best cascade by 0.7 dB PSNR$_L$ and 1.1 $\Delta E_{00}$.
arXiv:2611.03734v1 Announce Type: new Abstract: Training a classifier means letting a validation metric decide which model is kept and when training stops. A natural requirement on that metric is superiority: every correctly classified forecast should score better than every incorrectly classified one, however close the underlying probabilities are. Neither the Brier score nor the logarithmic score satisfies this on its own. We show that superiority is governed by the separation between two endpoints -- the supremum of the score on correct predictions and the infimum on incorrect predictions -- and derive both in closed form for each score. Their difference is the infimal penalty, the greatest lower bound on the penalty needed to restore superiority. Prior work restores superiority with a penalty calibrated from the first endpoint alone; for the Brier base, our infimal penalty is smaller than that published penalty by a factor ranging from 4.00 at three classes to 2.11 at twenty. We further embed the published and infimal penalties in a one-parameter family and show that, under a single-valued tie convention, every member of the family is strictly proper. In a pre-specified two-dataset case study, mean selected-model macro-F1 rises with the penalty in the reported ten-class regimes, though not distinguishably from noise once seed/selection-mode nesting is accounted for; a fifth regime at forty-seven classes does not reproduce even this descriptive pattern. The intended no-superiority baseline is algebraically a rescaling of the family under test, so the experiment cannot attribute the observed pattern specifically to superiority. When superiority itself is the objective, any penalty strictly above the infimal value suffices; choosing a larger penalty is a separate calibration trade-off rather than a requirement for the guarantee.
arXiv:2611.03737v1 Announce Type: new Abstract: Extracting semantically meaningful staypoints from raw trajectory streams is a foundational prerequisite for mobility analytics. However, real-world trajectory data is frequently degraded by severe spatial noise and protracted temporal dropouts. Classic threshold-based extraction methods, such as the widely adopted sliding-window algorithm, are highly sensitive to these anomalies, routinely fracturing continuous stationary episodes into disjointed fragments. To overcome this limitation, we propose the Sequential Staypoint Extraction (SSPE) algorithm. SSPE employs a resilient, state-driven architecture that replaces static spatial anchors with a dynamic centroid and introduces an $n$-step lookahead heuristic to structurally tolerate transient coordinate drift. Concurrently, it utilizes temporal bridging mechanisms to prevent artificial fragmentation during prolonged signal loss. Since optimal extraction parameters vary significantly across datasets, we further introduce a fully unsupervised, consensus-based parameter estimation framework. This framework derives baseline statistical scales directly from the raw trajectory data and identifies the optimal parameter configuration by maximizing structural agreement, thereby eliminating the reliance on annotated ground truth. We evaluate SSPE on the ExtraSensory dataset, a real-world mobility corpus with manually verified ground-truth staypoints, against the sliding-window implementation of Trackintel, one of the most widely adopted extraction baselines in the field. Empirical results demonstrate that SSPE achieves superior accuracy, yielding higher F1-scores while simultaneously improving computational efficiency over this standard.
ClickHouse’s new exhaustive cost-based join optimizer safely reorders inner, outer, semi, and anti joins to find the cheapest valid plan.
Checkr gave its AI agent a structured dbt context layer, cutting a support-metric investigation from hours to minutes.
dbt Wizard's designer and engineering lead show how the agent validates its own work before code ever reaches production.
dbt v2 bundles adapters, speeds up parsing, and replaces manifest.json with smaller, queryable files for people and AI.
We ran one AI skill 80 times against four context sources. Engineered summaries cut cost 48.5% and run time 43.2%.
Ramp shares the agents, sandboxes, and dashboards it built by treating internal AI as analytics engineering.
Verisk modernized decades of insurance data, from mainframe to dbt, to get AI-ready.
Careem's AI agent writes and validates dbt models end to end before a human ever reviews the PR.
How a data scientist built an AI context layer for organizational knowledge a dbt project alone can't hold.
Benn Stancil uses the Hugging Face hack to explain why the most useful ideas come from the weirdest connections.
ClickHouse’s new native TCP client for .NET brings faster reads and inserts with fewer allocations, columnar APIs, POCO support, and a path to migrate from HTTP.
Offload settled Postgres data to ClickHouse to reduce storage and query costs while keeping active transactions in Postgres.
Ridgeline rebuilt its performance analytics on ClickHouse Cloud, querying decades of Iceberg history alongside recent data so it can serve investment managers 10x larger than before and more than double the customers live on its platform.