惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

V
V2EX - 技术
D
DataBreaches.Net
阮一峰的网络日志
阮一峰的网络日志
Recent Announcements
Recent Announcements
V
V2EX
Hugging Face - Blog
Hugging Face - Blog
T
The Exploit Database - CXSecurity.com
Simon Willison's Weblog
Simon Willison's Weblog
Cisco Talos Blog
Cisco Talos Blog
Microsoft Security Blog
Microsoft Security Blog
C
Cyber Attacks, Cyber Crime and Cyber Security
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
K
Kaspersky official blog
F
Fortinet All Blogs
GbyAI
GbyAI
Forbes - Security
Forbes - Security
The Cloudflare Blog
博客园 - 司徒正美
博客园_首页
量子位
Schneier on Security
Schneier on Security
G
GRAHAM CLULEY
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
P
Proofpoint News Feed
N
News | PayPal Newsroom
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
博客园 - 聂微东
T
Tor Project blog
V
Vulnerabilities – Threatpost
Y
Y Combinator Blog
Jina AI
Jina AI
Help Net Security
Help Net Security
T
Threat Research - Cisco Blogs
Recent Commits to openclaw:main
Recent Commits to openclaw:main
C
Cybersecurity and Infrastructure Security Agency CISA
Project Zero
Project Zero
N
News and Events Feed by Topic
I
Intezer
B
Blog
美团技术团队
C
CERT Recently Published Vulnerability Notes
NISL@THU
NISL@THU
L
LINUX DO - 最新话题
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
Blog — PlanetScale
Blog — PlanetScale
AWS News Blog
AWS News Blog
T
Tailwind CSS Blog
The Last Watchdog
The Last Watchdog
雷峰网
雷峰网
有赞技术团队
有赞技术团队

The Practical Developer

The Libuv Thread Pool Trap: Why Node.js Async APIs Stall Under Load Postgres Covering Indexes with INCLUDE: Eliminate Heap Fetches on Read-Heavy Workloads Postgres DISTINCT ON: The Fastest Way to Get the Latest Row Per Group Postgres Transaction Isolation: The Anomalies Your App Actually Faces in Production Linux TCP Tuning for Node.js Microservices: The Kernel Settings That Stop Silent Connection Drops Under Load Postgres HOT Updates and Fillfactor: Why Not All Writes Are Created Equal Database Connection Pool Leaks: Finding the Promise That Never Returns Its Seat Linux OOM Killer in Production: Why Your Node.js Containers Die Without a Stack Trace API Dependency Health Checks: Why /health Is Not Enough Authorization with Zanzibar Tuples: How Google Manages Permissions and How To Build the Same Check in Node.js Postgres Advisory Locks: The 20-Character Primitive That Replaces Redis for Coordination Dead Letter Queues: The Message Queue Pattern That Saves You at 2 a.m. File Descriptor Exhaustion: The Kernel Limit That Silently Drops Node.js Connections Graceful Degradation: The Pattern That Turns Total Outages into Partial Success PostgreSQL Full-Text Search: Dropping Elasticsearch for 90% of Use Cases S3 Presigned Multipart Uploads: Stop Your API Server from Being a File Upload Bottleneck MessagePack vs JSON: The Binary Serialization Switch That Cut Our Internal RPC Overhead by 40% DNS Caching in Node.js: The Silent Cause of Production Latency Spikes Reliable Cron Jobs: The Pattern That Stops Double Runs, Missed Executions, And The 2 AM Page GraphQL Query Complexity: Stop the OOM Query Before It Reaches Your Resolver Node.js Event Loop Lag: The Hidden Metric Behind Random Latency Spikes API Request Validation with Zod: The Schema That Catches Bad Input Before It Corrupts Your Database Load Shedding in Node.js: How to Reject Traffic Before You Drown Request Hedging: Cut Tail Latency In Half Without Overprovisioning Git Bisect: The Automated Binary Search That Finds Breaking Commits in Minutes Node.js Garbage Collection Tuning: Stop Letting V8 Pause Your Event Loop Node.js Server Timeouts: The Settings That Stop Slow Clients from Holding Sockets Hostage Postgres BRIN Indexes: The Time-Series Secret That Shrinks Indexes by 99% Event Sourcing with PostgreSQL: The Pragmatic 80% Solution Node.js Cluster Mode: Scaling the Event Loop Across CPU Cores Postgres Partial Indexes: Stopping Soft Deletes from Ruining Your Query Performance Request Coalescing with the Singleflight Pattern: Stop Drowning Your Database on Every Cache Miss The Bulkhead Pattern: Why One Slow Endpoint Should Not Drown Your Whole Service Node.js AsyncLocalStorage: End-to-End Request Context Without the Propagation Hell Postgres Deadlocks: Logging the Victim, Reproducing the Race, and Fixing the Lock Order Your Node.js HTTP Client Is the Bottleneck: Connection Pool Tuning That Works Optimistic Locking in Postgres: Stop Losing Data to Race Conditions Postgres Read Replicas: Stop Serving Stale Data to Your Users Cursor Pagination: Why Offset Queries Explode at Scale and How to Fix Them Node.js Worker Threads: 60 Lines That Stop a CSV Upload from Timing Out Every Other Request Reliable Webhook Delivery: Architecture for Outbound HTTP You Can Trust Request Timeouts and Deadline Propagation: Stop the Chain of Slowness Advanced Security Practices in Node.js Graceful Shutdown in Node.js: The 40 Lines That Stop 502s During Deploys Finding Node.js Memory Leaks with Heap Snapshots Idempotency Keys in 30 Lines: Stop Your Webhook From Charging Customers Twice Backpressure In Node.js: The Fix For Slow-Motion Queue Meltdowns Retries Done Right: Jitter, Budgets, and the Stampede You Did Not See Coming The Cache Stampede: Why Your "Just Add Redis" Layer Crashes Postgres at 3 a.m. Postgres SKIP LOCKED: An 80-Line Job Queue You Can Run Without Redis Stop Doing Work Nobody Wants: AbortController in Node.js, Done Right The N+1 Query Problem: We Found 23 In One Codebase And Killed Every One I Tried 5 AI Coding Tools for a Month. Here Is What I Actually Use CI/CD From Zero to Production in 30 Minutes With GitHub Actions Node.js vs Bun vs Deno: Which Runtime Should You Pick in 2025? Kubernetes Resource Requests And Limits: The Numbers That Decide If Your Cluster Is Stable The Three Pillars of Observability Are A Myth: What Actually Matters In Production pnpm Vs npm Vs yarn Vs Bun For Monorepos: Which One Earns The Migration In 2024 JSONB Indexing In Postgres: GIN Vs Expression Indexes, And When Each Is The Right Choice A Code Review Checklist That Ends The Same Three Arguments Every Sprint gRPC Vs REST In 2024: When The Switch Pays For Itself React Suspense For Data Fetching: The Pattern That Replaces Half Your Loading State Code The Five-Stage Rollout: How To Ship A Risky Change Without Holding Your Breath GitHub Actions In A Monorepo: Caching, Path Filters, And Secret Boundaries That Actually Work The Blameless Postmortem That Actually Improves Things: A Template And Six Hard-Won Rules Recursive CTEs In Postgres: How To Query A Tree Without N Round Trips Node.js Streams: When They Actually Help, And When They Just Add Complexity Playwright Vs Cypress In 2024: The Honest Comparison Of Which One Earns The Test Time React Server Components: The Mental Model That Makes The "use client" Boundary Obvious Pod Disruption Budgets: The K8s Object That Keeps Your Service Up During Cluster Maintenance Postgres LISTEN/NOTIFY: The Pub/Sub You Already Have And Are Not Using Chaos Engineering Starter Kit: The Five Drills That Don't Need Netflix-Scale Spec-Driven API Development With OpenAPI: How To Stop Drifting From Your Docs Saga Pattern vs Two-Phase Commit: Distributed Transactions Without The Lies Kubernetes Autoscaling Beyond CPU: The Custom-Metric HPA Pattern That Actually Works Postgres Partitioning For Time-Series: The Boring Setup That Saves Your Database Distributed Locks With Redis: An Honest Look At Redlock And When You Don't Need It HTTP/2 vs HTTP/3: What Actually Changes For Your App, And What Doesn't Image Optimization For The Web In 2023: srcset, AVIF, And The Lighthouse Score You Actually Want Kafka vs RabbitMQ: A Decision Tree That Doesn't Hate You UUID vs Bigint Primary Keys In Postgres: The Index Math That Decides For You Flame Graphs: How To Find The Slow Function In 30 Seconds Without Profiling Theatre Postgres Streaming Vs. Logical Replication: Which One Solves Your Actual Problem ESLint Rules That Earn Their Keep: The Twelve I Enable On Every Project Pre-Commit Hooks That Pay For Themselves: Husky, lint-staged, And The Five Rules That Stick Zero-Downtime Database Migrations: The Six-Step Pattern That Rules Them All Circuit Breakers In Node.js: 50 Lines That Stop A Failing Dependency From Taking Down Your Service Postgres VACUUM Is Not Magic: How Your Hot Table Bloats To 80GB And How To Fix It Kubernetes Liveness And Readiness Probes: The Difference That Causes Half Your Outages Rate Limiting In Production: A Token Bucket In 30 Lines Of Redis The Outbox Pattern: How To Stop Losing Events When Postgres And Kafka Disagree Load Testing With k6: The Three Scenarios That Find Real Bugs (Not Synthetic Numbers) Postgres Row-Level Security For Multi-Tenant Apps: The Pattern That Stops You From Leaking Data Rebase vs. Merge: The Team Policy That Ends The Argument Forever OpenTelemetry in Node.js: Distributed Tracing That Actually Helps During an Incident Feature Flags That Pay Rent: The 4 Flag Types And When To Delete Each ETag, Last-Modified, and the Caching Headers Most APIs Get Wrong Connection Pooling Without the Cargo Cult: pgbouncer in 100 Lines of Config JSONB Is Not a Schema: When To Reach For It in Postgres, And When To Stop Bash Strict Mode: The Three Lines That Stop Your Deploy Script From Lying To You
Postgres Materialized Views: Refresh Strategies That Do Not Lock Your Dashboards
The Practica · 2026-05-25 · via The Practical Developer

The dashboard was unusable by 9 a.m. every Monday. A single page loaded six charts, each running a COUNT with GROUP BY over three months of order data. The queries were not badly written. They had indexes. But indexes cannot precompute an aggregation, and scanning 12 million rows to build a bar chart is slow no matter how you index it.

We added Redis caching. That worked until the first support ticket arrived: “I just processed a refund and the dashboard still shows the old total.” The cache TTL was five minutes. Users expected five milliseconds. We lowered the TTL to 30 seconds and now the database was melting under the refresh load.

The real fix was a materialized view: a snapshot of the aggregated result, stored on disk, refreshed on a schedule. The read time dropped from 28 seconds to 12 milliseconds. Then we discovered the catch. REFRESH MATERIALIZED VIEW takes an access-exclusive lock. While it runs, every dashboard query blocks. Our 45-second refresh window became a 45-second outage, six times per hour.

This post covers three refresh strategies that keep the data current without locking out readers: REFRESH CONCURRENTLY, the transactional swap pattern, and trigger-based incremental updates. Each has different trade-offs, and most teams need more than one.

What a materialized view actually buys you

A regular view is just a stored query. Every time you select from it, Postgres runs the underlying SQL. A materialized view is the result of that query, written to a physical table. The first read is expensive (during creation), but every subsequent read is a simple table scan of the precomputed result.

CREATE MATERIALIZED VIEW daily_revenue AS
SELECT
  date_trunc('day', created_at) AS day,
  SUM(amount) AS revenue,
  COUNT(*) AS order_count
FROM orders
WHERE created_at > now() - interval '90 days'
GROUP BY 1;

CREATE INDEX idx_daily_revenue_day ON daily_revenue(day);

Querying daily_revenue is now a single index scan over 90 rows instead of an aggregation over 12 million. The speedup is not 2x or 5x. It is three orders of magnitude. But the data is frozen at the moment of creation. To update it, you must refresh.

The locking problem

The default refresh is brutal:

REFRESH MATERIALIZED VIEW daily_revenue;

This runs the underlying query, writes the result to a new storage chunk, and swaps it in. While the swap happens, Postgres acquires an ACCESS EXCLUSIVE lock on the materialized view. That lock blocks SELECT, INSERT, UPDATE, DELETE, and even REFRESH CONCURRENTLY from any other session. If your refresh takes 40 seconds, your dashboard is down for 40 seconds.

In our case, the refresh was fast in staging (2 million rows) and catastrophic in production (12 million rows plus a JOIN to customers for segmentation). The lock duration scaled with data size, which is exactly the wrong direction for a performance optimization.

Strategy 1: REFRESH CONCURRENTLY

Postgres 9.4 added the right tool for most cases:

REFRESH MATERIALIZED VIEW CONCURRENTLY daily_revenue;

Instead of swapping the storage in place, CONCURRENTLY builds the new result in a hidden temporary table, then swaps the underlying filenode pointers at the end. The swap is fast (metadata-only), so the ACCESS EXCLUSIVE lock is held for milliseconds instead of seconds. Readers continue to see the old data during the refresh, then atomically switch to the new data when it is ready.

There is one hard requirement: the materialized view must have at least one unique index.

CREATE UNIQUE INDEX idx_daily_revenue_day_unique
  ON daily_revenue(day);

Without this, Postgres cannot identify rows for the internal diff operation that merges the new snapshot. The refresh will fail with a clear error, which is better than silently locking, but you need to plan for it at creation time.

When CONCURRENTLY is not enough

REFRESH CONCURRENTLY still has to run the full underlying query. If that query takes 60 seconds, the refresh takes 60 seconds plus swap time. During those 60 seconds, the old data is still visible, which is correct for dashboards, but the refresh itself consumes I/O and CPU. If you refresh six views every minute, you can saturate a read replica.

It also does not work if the underlying query contains:

  • ORDER BY (sorting a materialized view is pointless; add an index instead)
  • LIMIT or OFFSET (the result set is not deterministic enough for diffing)
  • Mutable functions like random(), now(), or current_timestamp inside the view definition (the unique index cannot stabilize across refreshes)

For most analytics dashboards, these restrictions are fine. For real-time leaderboards or time-series rollups that refresh every 10 seconds, the full-query cost is too high.

Strategy 2: The transactional swap pattern

When CONCURRENTLY is unavailable (Postgres < 9.4, or the view has LIMIT, or you need more control), you can fake it with two tables and a view wrapper.

The idea is simple. Maintain two identical tables: daily_revenue_a and daily_revenue_b. A regular view called daily_revenue points to whichever table is currently “live.” To refresh, you populate the inactive table, then swap the view definition in a transaction.

-- Two backing tables
CREATE TABLE daily_revenue_a (LIKE daily_revenue INCLUDING ALL);
CREATE TABLE daily_revenue_b (LIKE daily_revenue INCLUDING ALL);

-- The swapable view
CREATE OR REPLACE VIEW daily_revenue AS
SELECT * FROM daily_revenue_a;

-- Populate the inactive table
INSERT INTO daily_revenue_b
SELECT date_trunc('day', created_at) AS day,
       SUM(amount) AS revenue,
       COUNT(*) AS order_count
FROM orders
WHERE created_at > now() - interval '90 days'
GROUP BY 1;

-- Atomic swap
BEGIN;
CREATE OR REPLACE VIEW daily_revenue AS
SELECT * FROM daily_revenue_b;
COMMIT;

The CREATE OR REPLACE VIEW inside a transaction acquires a brief lock on the view, but because it is only metadata, the lock duration is negligible. Readers see either the old data or the new data; there is no intermediate state.

Automating the swap

In practice, you do not want to write this by hand every refresh. A PL/pgSQL function handles the A/B toggle:

CREATE OR REPLACE FUNCTION refresh_daily_revenue_swap()
RETURNS void AS $$
DECLARE
  current_live text;
  next_table text;
BEGIN
  -- Discover which table is currently live
  SELECT CASE WHEN definition LIKE '%daily_revenue_a%'
              THEN 'a' ELSE 'b' END
  INTO current_live
  FROM pg_views
  WHERE viewname = 'daily_revenue';

  next_table := CASE WHEN current_live = 'a' THEN 'b' ELSE 'a' END;

  -- Truncate and populate the inactive table
  EXECUTE format('TRUNCATE TABLE daily_revenue_%s', next_table);
  EXECUTE format(
    'INSERT INTO daily_revenue_%s
     SELECT date_trunc(''day'', created_at) AS day,
            SUM(amount) AS revenue,
            COUNT(*) AS order_count
     FROM orders
     WHERE created_at > now() - interval ''90 days''
     GROUP BY 1',
    next_table
  );

  -- Swap the view
  EXECUTE format(
    'CREATE OR REPLACE VIEW daily_revenue AS SELECT * FROM daily_revenue_%s',
    next_table
  );
END;
$$ LANGUAGE plpgsql;

Call it from a cron job, pg_cron, or your application scheduler:

SELECT refresh_daily_revenue_swap();

The downside is double storage. You need enough disk for two copies of the result set. For a 90-day rollup that is 10 MB, this is free. For a 10-year rollup that is 200 GB, it is not. Measure before you deploy.

Strategy 3: Incremental refresh with triggers

If your dashboard needs near-real-time data (sub-5-second lag), full refreshes are too expensive. The solution is to stop recomputing the whole view and start updating only the changed rows.

Postgres does not have built-in incremental materialized views (unless you install pg_ivm, which is promising but not production-ready for most teams). You can build the same behavior with a summary table and triggers.

CREATE TABLE daily_revenue_summary (
  day date PRIMARY KEY,
  revenue numeric(12,2) NOT NULL DEFAULT 0,
  order_count int NOT NULL DEFAULT 0
);

-- Seed it once
INSERT INTO daily_revenue_summary (day, revenue, order_count)
SELECT date_trunc('day', created_at)::date,
       SUM(amount),
       COUNT(*)
FROM orders
GROUP BY 1
ON CONFLICT (day) DO UPDATE SET
  revenue = EXCLUDED.revenue,
  order_count = EXCLUDED.order_count;

Then maintain it with a trigger on the source table:

CREATE OR REPLACE FUNCTION maintain_daily_revenue_summary()
RETURNS trigger AS $$
DECLARE
  target_day date;
  delta_amount numeric(12,2);
BEGIN
  IF TG_OP = 'INSERT' THEN
    target_day := date_trunc('day', NEW.created_at)::date;
    delta_amount := NEW.amount;
  ELSIF TG_OP = 'UPDATE' THEN
    -- Handle both day change and amount change
    IF date_trunc('day', OLD.created_at)::date
       IS DISTINCT FROM date_trunc('day', NEW.created_at)::date THEN
      -- Decrement old day
      INSERT INTO daily_revenue_summary (day, revenue, order_count)
      VALUES (date_trunc('day', OLD.created_at)::date, 0, 0)
      ON CONFLICT (day) DO UPDATE SET
        revenue = daily_revenue_summary.revenue - OLD.amount,
        order_count = daily_revenue_summary.order_count - 1;
      -- Increment new day
      target_day := date_trunc('day', NEW.created_at)::date;
      delta_amount := NEW.amount;
    ELSE
      target_day := date_trunc('day', NEW.created_at)::date;
      delta_amount := NEW.amount - OLD.amount;
    END IF;
  ELSIF TG_OP = 'DELETE' THEN
    target_day := date_trunc('day', OLD.created_at)::date;
    delta_amount := -OLD.amount;
  END IF;

  IF TG_OP = 'DELETE' OR
     (TG_OP = 'UPDATE' AND date_trunc('day', OLD.created_at)::date
      IS DISTINCT FROM date_trunc('day', NEW.created_at)::date) THEN
    -- Already handled above for the delete case
    IF TG_OP = 'DELETE' THEN
      UPDATE daily_revenue_summary
      SET revenue = revenue + delta_amount,
          order_count = order_count - 1
      WHERE day = target_day;
    END IF;
  ELSE
    INSERT INTO daily_revenue_summary (day, revenue, order_count)
    VALUES (target_day, delta_amount,
            CASE WHEN TG_OP = 'INSERT' THEN 1 ELSE 0 END)
    ON CONFLICT (day) DO UPDATE SET
      revenue = daily_revenue_summary.revenue + EXCLUDED.revenue,
      order_count = daily_revenue_summary.order_count +
        CASE WHEN TG_OP = 'INSERT' THEN 1 ELSE 0 END;
  END IF;

  RETURN COALESCE(NEW, OLD);
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER trg_maintain_daily_revenue
AFTER INSERT OR UPDATE OR DELETE ON orders
FOR EACH ROW
EXECUTE FUNCTION maintain_daily_revenue_summary();

This trigger is verbose because it handles day boundaries correctly. An UPDATE that moves an order from Monday to Tuesday must decrement Monday and increment Tuesday. The ON CONFLICT clause ensures the summary row exists before we try to update it.

The read query is trivial:

SELECT * FROM daily_revenue_summary
WHERE day > now() - interval '90 days'
ORDER BY day;

The cost of triggers

Every INSERT into orders now runs trigger logic. If you bulk-load 100,000 rows, the trigger fires 100,000 times. For bulk loads, disable the trigger first:

ALTER TABLE orders DISABLE TRIGGER trg_maintain_daily_revenue;
-- bulk insert
ALTER TABLE orders ENABLE TRIGGER trg_maintain_daily_revenue;
-- Then reconcile the summary table with a single upsert

Triggers also add latency to the write transaction. If your dashboard refresh interval is 30 seconds, a full REFRESH CONCURRENTLY is probably cheaper than paying the trigger tax on every write. Use incremental refresh only when stale data is actually a business problem, not just a preference.

Choosing a strategy

ScenarioRecommended approach
Dashboard refreshes every 5-15 minutes, query takes < 30 sREFRESH CONCURRENTLY with a unique index
Dashboard refreshes every minute, CONCURRENTLY not availableTransactional swap pattern
Dashboard must reflect writes within secondsTrigger-based incremental summary
Query takes > 2 minutes, refresh frequency > 10 minutesTransactional swap or accept the lock window
Source table has very high write volume, few readsDo not use a materialized view; index the source

Most production dashboards I have worked with fall into the first bucket. REFRESH CONCURRENTLY is the default for a reason. It is simple, safe, and handles the lock problem without extra tables or trigger complexity.

Monitoring and maintenance

Materialized views are tables. They bloat. They need VACUUM. They need ANALYZE so the query planner knows how big they are.

Add a dedicated autovacuum configuration if the view refreshes frequently:

ALTER MATERIALIZED VIEW daily_revenue SET (
  autovacuum_vacuum_scale_factor = 0.1,
  autovacuum_analyze_scale_factor = 0.05
);

Monitor the refresh duration in your metrics pipeline. A sudden jump from 8 seconds to 45 seconds usually means the underlying query lost a good plan (check for stale statistics) or the source table grew past an index threshold.

Also watch disk space. REFRESH CONCURRENTLY keeps the old snapshot until the transaction ends, so a long-running SELECT on the view can pin the old data and prevent cleanup. If you have analysts running ad-hoc queries against the dashboard schema, set idle_in_transaction_session_timeout to something reasonable (5 minutes) so they do not hold dead tuples forever.

The takeaway

Materialized views are not a magic performance button. They are a trade-off: you accept stale data in exchange for read speed. The key is managing the refresh so the staleness window is predictable and the refresh itself does not become a new outage source.

Start with REFRESH CONCURRENTLY and a unique index. If that is not available, use the transactional swap pattern. Only reach for triggers when real-time is a hard requirement. And always measure the refresh duration in production, because the query planner loves to surprise you at scale.

A note from Yojji

The difference between a dashboard that loads in 12 milliseconds and one that times out after 30 seconds is often not a bigger server. It is understanding which Postgres primitives actually precompute work and which ones just hide the cost. Yojji’s teams have been building read-heavy analytics features and the data pipelines behind them since 2016, using exactly the kind of incremental refresh and swap patterns that keep dashboards fast without locking out users.

Yojji is an international custom software development company founded in 2016, with offices in Europe, the US, and the UK. They specialize in the JavaScript ecosystem (React, Node.js, TypeScript), cloud platforms (AWS, Azure, Google Cloud), and the database architecture decisions that keep production systems responsive when the dataset grows past the point where naive queries still work.