惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Apple Machine Learning Research
Apple Machine Learning Research
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
G
Google Developers Blog
博客园 - 司徒正美
J
Java Code Geeks
aimingoo的专栏
aimingoo的专栏
A
About on SuperTechFans
博客园 - 三生石上(FineUI控件)
WordPress大学
WordPress大学
T
The Blog of Author Tim Ferriss
D
Docker
大猫的无限游戏
大猫的无限游戏
D
DataBreaches.Net
腾讯CDC
V
Visual Studio Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
C
Check Point Blog
M
MIT News - Artificial intelligence
Jina AI
Jina AI
I
InfoQ
雷峰网
雷峰网
The Cloudflare Blog
美团技术团队
Engineering at Meta
Engineering at Meta

Stonecharioteer on Tech

I Traced My Traffic Through a Home Tailscale Exit Node What Was I Reading Last? In Three Not-So-Easy Pieces Dogfooding Is Hard Code blocks in your books, finally GoForGo v0.9.0 Merrilin - We built an app to read books I use a Macbook now Data Structures & Algorithms - Preparing for Interviews Using a local DNS namespace for local service discovery Direction KOllector - Publishing KOReader Highlights gbt: branches touched in the last 24 hours A Soiree into Symbols in Ruby Some Smalltalk about Ruby Loops Ruby Blocks Returning from Ruby Blocks, Procs and Lambdas My Linux Laptop Finally Works: How Claude Helped Me Fix Years of Annoyances TIL: Watchexec - Modern File Watching for Development Workflows A Less Busy Mind GoForGo - Learn Go through live examples Migrating My Old Blog to Hugo with Claude The Qtile Window Manager: A Python-Powered Tiling Experience Read the RFCs that Built the Internet Py-x-Protobuf - Or How I Learned to Stop Worrying and Love Protocol Buffers Python Reverse a List New Beginnings Leaving ChainSafe Systems Screen Lock for Cinnamon Desktop using Zenity and Terminal Commands Crews Not Teams A System for Getting Better at LeetCode
TIL: PostgreSQL Customization and Advanced Database Manag...
2020-11-08 · via Stonecharioteer on Tech

Today I discovered comprehensive techniques for customizing PostgreSQL environments and learned advanced database management practices that significantly improve developer and DBA productivity.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
-- ~/.psqlrc - PostgreSQL shell customization

-- Set better default formatting
\set QUIET 1
\pset null '[NULL]'
\pset linestyle unicode
\pset border 2
\pset format wrapped
\set COMP_KEYWORD_CASE upper
\set HISTSIZE 10000
\set PROMPT1 '%[%033[1m%]%M %n@%/%R%[%033[0m%]%# '
\set PROMPT2 '[more] %R > '
\unset QUIET

-- Timing for performance monitoring
\timing

-- Show expanded output for wide tables
\x auto

-- Better error handling
\set ON_ERROR_ROLLBACK interactive
\set ON_ERROR_STOP on

-- Helpful shortcuts and aliases
\set version 'SELECT version();'
\set extensions 'SELECT * FROM pg_available_extensions ORDER BY name;'
\set settings 'SELECT name, setting, unit, context FROM pg_settings ORDER BY name;'
\set locks 'SELECT * FROM pg_locks ORDER BY pid;'
\set activity 'SELECT pid, usename, application_name, client_addr, state, query_start, query FROM pg_stat_activity ORDER BY query_start DESC;'
\set blocking 'SELECT blocked_locks.pid AS blocked_pid, blocked_activity.usename AS blocked_user, blocking_locks.pid AS blocking_pid, blocking_activity.usename AS blocking_user, blocked_activity.query AS blocked_statement, blocking_activity.query AS current_statement_in_blocking_process FROM pg_catalog.pg_locks blocked_locks JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid JOIN pg_catalog.pg_locks blocking_locks ON blocking_locks.locktype = blocked_locks.locktype AND blocking_locks.DATABASE IS NOT DISTINCT FROM blocked_locks.DATABASE AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid AND blocking_locks.pid != blocked_locks.pid JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid WHERE NOT blocked_locks.GRANTED;'

-- Database size information
\set dbsize 'SELECT datname, pg_size_pretty(pg_database_size(datname)) as size FROM pg_database ORDER BY pg_database_size(datname) DESC;'
\set tablesize 'SELECT schemaname, tablename, pg_size_pretty(pg_total_relation_size(schemaname||''.''||tablename)) as size FROM pg_tables ORDER BY pg_total_relation_size(schemaname||''.''||tablename) DESC;'

-- Index usage statistics
\set indexusage 'SELECT schemaname, tablename, attname, n_distinct, correlation FROM pg_stats WHERE schemaname = ''public'' ORDER BY n_distinct DESC;'
\set indexsize 'SELECT schemaname, tablename, indexname, pg_size_pretty(pg_relation_size(indexname::regclass)) as size FROM pg_indexes WHERE schemaname = ''public'' ORDER BY pg_relation_size(indexname::regclass) DESC;'

-- Connection information
\set conninfo 'SELECT usename, count(*) FROM pg_stat_activity GROUP BY usename ORDER BY count DESC;'

-- Slow queries (requires pg_stat_statements)
\set slowqueries 'SELECT query, calls, total_time, mean_time, rows FROM pg_stat_statements ORDER BY total_time DESC LIMIT 10;'

-- Replication status
\set replication 'SELECT * FROM pg_stat_replication;'

-- Vacuum and analyze status
\set vacuum_stats 'SELECT schemaname, tablename, last_vacuum, last_autovacuum, last_analyze, last_autoanalyze FROM pg_stat_user_tables ORDER BY last_vacuum DESC NULLS LAST;'
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
-- Query optimization and performance tuning

-- 1. Identify slow queries with detailed analysis
WITH slow_queries AS (
    SELECT
        query,
        calls,
        total_time,
        mean_time,
        max_time,
        stddev_time,
        rows,
        100.0 * shared_blks_hit / NULLIF(shared_blks_hit + shared_blks_read, 0) AS cache_hit_ratio,
        shared_blks_read,
        shared_blks_written,
        temp_blks_read,
        temp_blks_written
    FROM pg_stat_statements
    WHERE mean_time > 100  -- Queries averaging more than 100ms
    OR total_time > 10000  -- Queries with high total time
    ORDER BY total_time DESC
)
SELECT
    LEFT(query, 100) as query_snippet,
    calls,
    ROUND(total_time::numeric, 2) as total_time_ms,
    ROUND(mean_time::numeric, 2) as mean_time_ms,
    ROUND(max_time::numeric, 2) as max_time_ms,
    rows,
    ROUND(cache_hit_ratio::numeric, 2) as cache_hit_pct,
    CASE
        WHEN temp_blks_written > 0 THEN 'Uses temp files'
        WHEN cache_hit_ratio < 95 THEN 'Poor cache utilization'
        WHEN mean_time > 1000 THEN 'Very slow average'
        ELSE 'Review for optimization'
    END as optimization_hint
FROM slow_queries
LIMIT 20;

-- 2. Index recommendations based on query patterns
WITH table_scans AS (
    SELECT
        schemaname,
        tablename,
        seq_scan,
        seq_tup_read,
        idx_scan,
        idx_tup_fetch,
        n_tup_ins + n_tup_upd + n_tup_del as modifications
    FROM pg_stat_user_tables
    WHERE schemaname = 'public'
),
index_candidates AS (
    SELECT
        schemaname,
        tablename,
        seq_scan,
        seq_tup_read,
        CASE
            WHEN seq_scan > 0 THEN seq_tup_read / seq_scan
            ELSE 0
        END as avg_seq_read,
        idx_scan,
        modifications,
        CASE
            WHEN seq_scan > idx_scan AND seq_tup_read > 10000
            THEN 'High sequential scan activity - consider indexing'
            WHEN modifications > (idx_scan + seq_scan) * 10
            THEN 'High modification rate - index overhead may be significant'
            ELSE 'Normal access pattern'
        END as recommendation
    FROM table_scans
)
SELECT * FROM index_candidates
WHERE recommendation LIKE 'High%'
ORDER BY avg_seq_read DESC;

-- 3. Connection and lock analysis
WITH connection_analysis AS (
    SELECT
        datname,
        usename,
        application_name,
        state,
        COUNT(*) as connection_count,
        AVG(EXTRACT(EPOCH FROM (now() - query_start))) as avg_query_duration,
        MAX(EXTRACT(EPOCH FROM (now() - query_start))) as max_query_duration,
        SUM(CASE WHEN state = 'idle in transaction' THEN 1 ELSE 0 END) as idle_in_transaction
    FROM pg_stat_activity
    WHERE pid != pg_backend_pid()
    GROUP BY datname, usename, application_name, state
)
SELECT
    *,
    CASE
        WHEN idle_in_transaction > 0 THEN 'Idle transactions detected - may cause blocking'
        WHEN max_query_duration > 300 THEN 'Long-running queries detected'
        WHEN connection_count > 100 THEN 'High connection count'
        ELSE 'Normal'
    END as alert
FROM connection_analysis
ORDER BY connection_count DESC;

This comprehensive exploration of PostgreSQL customization and management demonstrates how proper configuration and monitoring can transform database administration from reactive troubleshooting to proactive optimization.

These PostgreSQL insights from my archive showcase the evolution from basic database usage to advanced administration practices, emphasizing the importance of proper tooling and systematic monitoring for database reliability and performance.