惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

月光博客
月光博客
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
N
Netflix TechBlog - Medium
大猫的无限游戏
大猫的无限游戏
爱范儿
爱范儿
Martin Fowler
Martin Fowler
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
The Register - Security
The Register - Security
IT之家
IT之家
博客园_首页
Microsoft Security Blog
Microsoft Security Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
博客园 - 三生石上(FineUI控件)
I
InfoQ
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
Jina AI
Jina AI
Apple Machine Learning Research
Apple Machine Learning Research
M
MIT News - Artificial intelligence
博客园 - Franky
C
Check Point Blog
T
The Blog of Author Tim Ferriss
V
Visual Studio Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
T
Tailwind CSS Blog
Recent Announcements
Recent Announcements
云风的 BLOG
云风的 BLOG
美团技术团队
The Cloudflare Blog
Y
Y Combinator Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
MyScale Blog
MyScale Blog
The GitHub Blog
The GitHub Blog
D
DataBreaches.Net
Google DeepMind News
Google DeepMind News
V
V2EX
aimingoo的专栏
aimingoo的专栏
GbyAI
GbyAI
G
Google Developers Blog
S
SegmentFault 最新的问题
Hugging Face - Blog
Hugging Face - Blog
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
U
Unit 42
罗磊的独立博客
量子位
MongoDB | Blog
MongoDB | Blog
Last Week in AI
Last Week in AI
Stack Overflow Blog
Stack Overflow Blog
小众软件
小众软件
D
Docker
人人都是产品经理
人人都是产品经理

Stonecharioteer on Tech

I Traced My Traffic Through a Home Tailscale Exit Node What Was I Reading Last? In Three Not-So-Easy Pieces Dogfooding Is Hard Code blocks in your books, finally GoForGo v0.9.0 Merrilin - We built an app to read books I use a Macbook now Data Structures & Algorithms - Preparing for Interviews Using a local DNS namespace for local service discovery Direction KOllector - Publishing KOReader Highlights gbt: branches touched in the last 24 hours A Soiree into Symbols in Ruby Some Smalltalk about Ruby Loops Ruby Blocks Returning from Ruby Blocks, Procs and Lambdas My Linux Laptop Finally Works: How Claude Helped Me Fix Years of Annoyances TIL: Watchexec - Modern File Watching for Development Workflows A Less Busy Mind GoForGo - Learn Go through live examples Migrating My Old Blog to Hugo with Claude The Qtile Window Manager: A Python-Powered Tiling Experience Read the RFCs that Built the Internet Py-x-Protobuf - Or How I Learned to Stop Worrying and Love Protocol Buffers Python Reverse a List New Beginnings Leaving ChainSafe Systems Screen Lock for Cinnamon Desktop using Zenity and Terminal Commands Crews Not Teams A System for Getting Better at LeetCode So Far So Rust Retrying HTTP Requests with Rust A Primer on Control Charts Learning Rust Explicit is Better than Implicit: Rust for Pythonistas Using Custom Delimiters in Jinja Templates TIL: Creating Fixed Length Iterables in Python Documentation Without Assumption Vagrant Python - A Reflection in 2022 Learning Golang No, A Virtual Machine Is Not Enough: Why Developers Need Native Linux Empathy in Tech For Those Who Came in Late A Weekend With PostgreSQL TIL: Gooey and Python Fire for Quick GUIs and CLIs TIL: 2ality - Dr. Axel Rauschmayer's JavaScript Blog TIL: MassDNS - High-Performance Bulk DNS Lookups TIL: Matomo Analytics, Google Tech Writing, Memory Programming, and NES TV Signals TIL: MontyDB - MongoDB Implemented in Python Returning to the Craft of Programming TIL: CPUFetch, OneFetch, and Learn CSS TIL: DNS Performance Testing and Pi-hole with Unbound TIL: Eli Bendersky's Blog, Awesome By Example, NoCoDB, and Martin Kleppmann TIL: CRDTs, Extreme HTTP Performance, and BYTEPATH Game TIL: AutoInvent, ASGI, Python Packaging, RAPIDS GPU Computing, and FlaskCon TIL: MangaDesk - Terminal Client for MangaDex TIL: McFly - Smart Shell History Search TIL: Siege Load Testing and Awesome FastAPI Resources TIL: Ventoy Bootable USB and Justniffer Network Analysis TIL: CLI Code Review, Git Split Diffs, and Internal Combustion Engine TIL: Benford's Law, Web Security Headers, Event Sourcing, and Mozilla Security Guidelines How to Write Documentation - The README.md File The Importance of Documentation TIL: NNgroup UX Research, SponsorBlock, and Labella Python Library TIL: The Little Book of Rust Macros and Rust Performance Book TIL: Git-Bug Distributed Issue Tracker and Omni Kubernetes Monitoring TIL: Zellij - Modern Terminal Multiplexer TIL: How Discord Handles 2.5 Million Concurrent Voice Users TIL: Volumio - The Audiophile Music Player TIL: Areopagitica - Milton's Defense of Free Speech TIL: Fast Node Manager, Zoxide Smart CD, Technical Writing, PyO3, and Qubes OS TIL: Slurm Workload Manager for HPC Clusters TIL: Data Visualization Guide and Oso Authorization Academy TIL: CORS Deep Dive, Piku Tiny PaaS, Rust Strings, and Deno Standard Library TIL: Raspberry Pi OS Development, Vim Beginner Guide, Password Management, and QueryBook TIL: uBlock Origin Performance Optimization on Firefox TIL: Breaking PostgreSQL at Scale and LeetCode Problem Patterns TIL: Awesome Tmux Resources for Terminal Multiplexing TIL: Grit - A Multitree-Based Personal Task Manager TIL: Lens 4.2 Kubernetes IDE, Shell Scripting Guide, and Dark HTTP Server Do The Job You Hate So You Won't Hate The Job You Love TIL: Innernet VPN Solution and NoteCalc Calculator App TIL: Argo CD for GitOps and Lens Kubernetes IDE TIL: Modern Rust CLI Tools - System Monitoring, HTTP Requests, and DNS TIL: tz - A Time Zone Helper Tool TIL: Distributed Systems Education, Fallacies, and Self-Hosted Internet Archiving TIL: Real-Time Voice Cloning Technology TIL: ChartMuseum for Helm, AMD's Corporate Journey, and Kubernetes Pod Scaling TIL: Docker and Kubernetes Tools - Whaler, Descheduler, and Dive TIL: Post-Mortem Collection, Terminal Plotting, and Technical Twitter TIL: Dark Mode Toggle Web Component by Google Chrome Labs TIL: Python eval(), exec(), and compile() Functions TIL: Camelot PDF Tables, PostgreSQL Row Level Security, Zerodha Varsity, and Write Yourself a Git TIL: fuser Command for Process and File Investigation TIL: i Hate Regex - The Ultimate Regex Cheat Sheet TIL: Dolt - Git for Data and Database Version Control TIL: x86 Assembly Programming and SafeEyes Break Reminder TIL: Comprehensive Distributed Systems Reading List TIL: Cosmopolitan C Library, Distributed Systems Book, High Performance Browser Networking, and Rust Roguelike Tutorial
TIL: Docker Networking Deep Dive, Julia Evans' Systems Knowledge, and Kubernetes from the Ground Up
2020-07-30 · via Stonecharioteer on Tech

Today’s learning focused on container technologies, systems programming concepts, and distributed systems architecture.

Docker Networking and Graph Drivers

Brutally Honest Guide to Docker Graph Drivers and Container Networking provide deep insights into Docker’s internal mechanisms:

Docker Graph Drivers:

Storage Driver Comparison:

1
2
3
4
5
6
7
8
9
# Check current storage driver
docker info | grep "Storage Driver"

# Available drivers and characteristics:
# overlay2 (recommended): Copy-on-write, good performance
# aufs: Legacy, being phased out
# devicemapper: Block-level storage, complex configuration
# btrfs: Advanced filesystem features, snapshot support
# zfs: Enterprise features, high memory usage

Overlay2 Driver Deep Dive:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Inspect layer structure
docker pull ubuntu:20.04
docker history ubuntu:20.04

# Examine overlay2 directory structure
sudo ls -la /var/lib/docker/overlay2/

# Layer composition
# Lower layers: read-only base layers
# Upper layer: read-write container changes
# Merged layer: unified view presented to container

# Example layer structure:
# /var/lib/docker/overlay2/
# ├── abc123...def/
# │   ├── diff/          # Layer changes
# │   ├── link           # Short identifier
# │   ├── lower          # Parent layers
# │   └── work/          # Temporary work directory
# └── l/                 # Symbolic links to layers

Performance Implications:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Inefficient: creates many layers
FROM ubuntu:20.04
RUN apt-get update
RUN apt-get install -y python3
RUN apt-get install -y python3-pip
RUN apt-get install -y git
RUN apt-get clean

# Efficient: fewer layers, better caching
FROM ubuntu:20.04
RUN apt-get update && \
    apt-get install -y \
        python3 \
        python3-pip \
        git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Docker Container Networking:

Network Types:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# List networks
docker network ls

# Default networks:
# bridge: Default network for containers
# host: Container uses host networking directly
# none: No networking
# Custom networks: User-defined bridge networks

# Create custom network
docker network create --driver bridge my-network
docker network create --driver bridge --subnet=172.20.0.0/16 my-subnet

# Inspect network configuration
docker network inspect bridge
docker network inspect my-network

Bridge Network Deep Dive:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Default bridge network (docker0)
ip addr show docker0
brctl show docker0

# Container networking internals
docker run -d --name web nginx
docker exec web ip addr show
docker exec web ip route show

# Network namespace inspection
docker inspect web | grep NetworkMode
sudo nsenter -t $(docker inspect -f '{{.State.Pid}}' web) -n ip addr

Custom Networking:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# docker-compose.yml with custom networks
version: "3.8"
services:
  web:
    image: nginx
    networks:
      - frontend
      - backend
    ports:
      - "80:80"

  app:
    image: python:3.9
    networks:
      - backend
      - database
    depends_on:
      - db

  db:
    image: postgres:13
    networks:
      - database
    environment:
      POSTGRES_PASSWORD: secret

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
  database:
    driver: bridge
    internal: true # No external access

Container Communication:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Service discovery within custom networks
docker network create app-network
docker run -d --name db --network app-network postgres:13
docker run -d --name app --network app-network my-app:latest

# Containers can reach each other by name
# app container can connect to: db:5432
# No need for --link (deprecated)

# Port publishing vs exposure
docker run -p 8080:80 nginx    # Publish to host
docker run --expose 80 nginx   # Only expose to other containers

Julia Evans’ Systems Programming Insights

Julia Evans provides exceptional explanations of complex systems concepts:

Networking Fundamentals:

Computer Networking Basics:

  • TCP vs UDP: Reliability vs speed trade-offs
  • DNS resolution: How domain names become IP addresses
  • HTTP request flow: Complete journey from browser to server
  • Load balancing: Distributing traffic across multiple servers

HTTP Request Routing:

Client Request Flow:
1. DNS lookup: domain.com → IP address
2. TCP connection: Three-way handshake
3. TLS handshake: Certificate validation, encryption setup
4. HTTP request: Headers, body, method
5. Server processing: Route matching, business logic
6. HTTP response: Status code, headers, body
7. Connection handling: Keep-alive or close

SNI (Server Name Indication):
- Allows multiple HTTPS sites on same IP
- Client sends domain name during TLS handshake
- Server selects appropriate certificate
- Critical for shared hosting and CDNs

Kubernetes Architecture Understanding:

Kubernetes Learning Journey:

  • Pods: Basic deployment units, shared networking/storage
  • Services: Stable network endpoints for dynamic pods
  • Deployments: Declarative pod management and rolling updates
  • ConfigMaps/Secrets: Configuration and sensitive data management

Kamal Marhubi’s series provides deep architectural understanding:

Core Kubernetes Components:

The API Server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# API Server responsibilities:
# 1. RESTful API for all Kubernetes resources
# 2. Authentication and authorization
# 3. Admission control and validation
# 4. etcd storage interface
# 5. Resource change notifications

# Example API interaction
apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  namespace: default
spec:
  containers:
    - name: web
      image: nginx:1.20
      ports:
        - containerPort: 80
      resources:
        limits:
          memory: "128Mi"
          cpu: "500m"

The Kubelet:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Kubelet responsibilities:
# 1. Watch API server for pod assignments
# 2. Manage container lifecycle (start, stop, restart)
# 3. Health checking and reporting
# 4. Resource monitoring and management
# 5. Volume mounting and networking setup

# Kubelet configuration
sudo systemctl status kubelet
sudo journalctl -u kubelet -f

# Pod lifecycle management
# kubelet receives pod spec from API server
# Downloads container images
# Creates container runtime (containerd/docker)
# Sets up networking (CNI plugins)
# Mounts volumes
# Starts containers
# Monitors health and reports status

The Scheduler:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
// Simplified scheduler algorithm
func Schedule(pod *Pod, nodes []Node) *Node {
    // 1. Filter nodes (predicates)
    feasibleNodes := []Node{}
    for _, node := range nodes {
        if canSchedule(pod, node) {
            feasibleNodes = append(feasibleNodes, node)
        }
    }

    // 2. Score nodes (priorities)
    scoredNodes := make(map[Node]int)
    for _, node := range feasibleNodes {
        score := calculateScore(pod, node)
        scoredNodes[node] = score
    }

    // 3. Select highest scoring node
    bestNode := selectBestNode(scoredNodes)
    return bestNode
}

func canSchedule(pod *Pod, node *Node) bool {
    // Resource constraints
    if node.AvailableCPU < pod.RequestedCPU {
        return false
    }
    if node.AvailableMemory < pod.RequestedMemory {
        return false
    }

    // Node selectors
    if !nodeMatchesSelectors(node, pod.NodeSelector) {
        return false
    }

    // Taints and tolerations
    if !podToleratesNodeTaints(pod, node.Taints) {
        return false
    }

    return true
}

Advanced Kubernetes Concepts:

Networking Model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# CNI (Container Network Interface) plugins
# Provide pod-to-pod networking across nodes

# Example: Calico network policy
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: web-netpol
spec:
  podSelector:
    matchLabels:
      app: web
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 80
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: database
      ports:
        - protocol: TCP
          port: 5432

Storage and Persistence:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Persistent Volume and Claims
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-example
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: /data/pv-example

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-example
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Everything is a File Philosophy:

The Unix principle that everything is a file extends to container technologies:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Container processes visible in host /proc
docker run -d --name test nginx
docker exec test ps aux

# Container networking through Linux primitives
ip netns list
docker exec test cat /proc/net/tcp

# Container storage as filesystem layers
ls -la /var/lib/docker/overlay2/
mount | grep overlay

# Socket communication
stat /var/run/docker.sock  # Docker daemon socket
file /var/run/docker.sock  # Shows socket type

These concepts form the foundation for understanding modern containerized and orchestrated systems, from basic Docker networking to sophisticated Kubernetes cluster management.