惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

D
DataBreaches.Net
T
Threatpost
N
News and Events Feed by Topic
PCI Perspectives
PCI Perspectives
V2EX - 技术
V2EX - 技术
D
Docker
G
Google Developers Blog
Microsoft Security Blog
Microsoft Security Blog
N
News and Events Feed by Topic
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Google Online Security Blog
Google Online Security Blog
The GitHub Blog
The GitHub Blog
Hacker News - Newest:
Hacker News - Newest: "LLM"
Y
Y Combinator Blog
M
MIT News - Artificial intelligence
Blog — PlanetScale
Blog — PlanetScale
博客园 - 司徒正美
T
Troy Hunt's Blog
Webroot Blog
Webroot Blog
Security Archives - TechRepublic
Security Archives - TechRepublic
量子位
Apple Machine Learning Research
Apple Machine Learning Research
H
Help Net Security
F
Full Disclosure
B
Blog
O
OpenAI News
H
Hackread – Cybersecurity News, Data Breaches, AI and More
博客园_首页
Google DeepMind News
Google DeepMind News
Exploit-DB.com RSS Feed
Exploit-DB.com RSS Feed
Engineering at Meta
Engineering at Meta
大猫的无限游戏
大猫的无限游戏
Forbes - Security
Forbes - Security
Know Your Adversary
Know Your Adversary
B
Blog RSS Feed
MongoDB | Blog
MongoDB | Blog
Scott Helme
Scott Helme
T
The Exploit Database - CXSecurity.com
博客园 - 聂微东
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
The Last Watchdog
The Last Watchdog
Recorded Future
Recorded Future
IT之家
IT之家
Project Zero
Project Zero
Stack Overflow Blog
Stack Overflow Blog
小众软件
小众软件
Attack and Defense Labs
Attack and Defense Labs
L
Lohrmann on Cybersecurity
SecWiki News
SecWiki News
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com

Stonecharioteer on Tech

I Traced My Traffic Through a Home Tailscale Exit Node What Was I Reading Last? In Three Not-So-Easy Pieces Dogfooding Is Hard Code blocks in your books, finally GoForGo v0.9.0 Merrilin - We built an app to read books I use a Macbook now Data Structures & Algorithms - Preparing for Interviews Using a local DNS namespace for local service discovery Direction KOllector - Publishing KOReader Highlights gbt: branches touched in the last 24 hours A Soiree into Symbols in Ruby Some Smalltalk about Ruby Loops Ruby Blocks Returning from Ruby Blocks, Procs and Lambdas My Linux Laptop Finally Works: How Claude Helped Me Fix Years of Annoyances TIL: Watchexec - Modern File Watching for Development Workflows A Less Busy Mind GoForGo - Learn Go through live examples Migrating My Old Blog to Hugo with Claude The Qtile Window Manager: A Python-Powered Tiling Experience Read the RFCs that Built the Internet Py-x-Protobuf - Or How I Learned to Stop Worrying and Love Protocol Buffers Python Reverse a List New Beginnings Leaving ChainSafe Systems Screen Lock for Cinnamon Desktop using Zenity and Terminal Commands Crews Not Teams A System for Getting Better at LeetCode So Far So Rust Retrying HTTP Requests with Rust A Primer on Control Charts Learning Rust Explicit is Better than Implicit: Rust for Pythonistas Using Custom Delimiters in Jinja Templates TIL: Creating Fixed Length Iterables in Python Documentation Without Assumption Vagrant Python - A Reflection in 2022 Learning Golang No, A Virtual Machine Is Not Enough: Why Developers Need Native Linux Empathy in Tech For Those Who Came in Late A Weekend With PostgreSQL TIL: Gooey and Python Fire for Quick GUIs and CLIs TIL: 2ality - Dr. Axel Rauschmayer's JavaScript Blog TIL: MassDNS - High-Performance Bulk DNS Lookups TIL: Matomo Analytics, Google Tech Writing, Memory Programming, and NES TV Signals TIL: MontyDB - MongoDB Implemented in Python Returning to the Craft of Programming TIL: CPUFetch, OneFetch, and Learn CSS TIL: DNS Performance Testing and Pi-hole with Unbound TIL: Eli Bendersky's Blog, Awesome By Example, NoCoDB, and Martin Kleppmann TIL: CRDTs, Extreme HTTP Performance, and BYTEPATH Game TIL: AutoInvent, ASGI, Python Packaging, RAPIDS GPU Computing, and FlaskCon TIL: MangaDesk - Terminal Client for MangaDex TIL: McFly - Smart Shell History Search TIL: Siege Load Testing and Awesome FastAPI Resources TIL: Ventoy Bootable USB and Justniffer Network Analysis TIL: CLI Code Review, Git Split Diffs, and Internal Combustion Engine TIL: Benford's Law, Web Security Headers, Event Sourcing, and Mozilla Security Guidelines How to Write Documentation - The README.md File The Importance of Documentation TIL: NNgroup UX Research, SponsorBlock, and Labella Python Library TIL: The Little Book of Rust Macros and Rust Performance Book TIL: Git-Bug Distributed Issue Tracker and Omni Kubernetes Monitoring TIL: Zellij - Modern Terminal Multiplexer TIL: How Discord Handles 2.5 Million Concurrent Voice Users TIL: Volumio - The Audiophile Music Player TIL: Areopagitica - Milton's Defense of Free Speech TIL: Fast Node Manager, Zoxide Smart CD, Technical Writing, PyO3, and Qubes OS TIL: Slurm Workload Manager for HPC Clusters TIL: Data Visualization Guide and Oso Authorization Academy TIL: CORS Deep Dive, Piku Tiny PaaS, Rust Strings, and Deno Standard Library TIL: Raspberry Pi OS Development, Vim Beginner Guide, Password Management, and QueryBook TIL: uBlock Origin Performance Optimization on Firefox TIL: Breaking PostgreSQL at Scale and LeetCode Problem Patterns TIL: Awesome Tmux Resources for Terminal Multiplexing TIL: Grit - A Multitree-Based Personal Task Manager TIL: Lens 4.2 Kubernetes IDE, Shell Scripting Guide, and Dark HTTP Server Do The Job You Hate So You Won't Hate The Job You Love TIL: Innernet VPN Solution and NoteCalc Calculator App TIL: Argo CD for GitOps and Lens Kubernetes IDE TIL: Modern Rust CLI Tools - System Monitoring, HTTP Requests, and DNS TIL: tz - A Time Zone Helper Tool TIL: Distributed Systems Education, Fallacies, and Self-Hosted Internet Archiving TIL: Real-Time Voice Cloning Technology TIL: ChartMuseum for Helm, AMD's Corporate Journey, and Kubernetes Pod Scaling TIL: Docker and Kubernetes Tools - Whaler, Descheduler, and Dive TIL: Post-Mortem Collection, Terminal Plotting, and Technical Twitter TIL: Dark Mode Toggle Web Component by Google Chrome Labs TIL: Python eval(), exec(), and compile() Functions TIL: Camelot PDF Tables, PostgreSQL Row Level Security, Zerodha Varsity, and Write Yourself a Git TIL: fuser Command for Process and File Investigation TIL: i Hate Regex - The Ultimate Regex Cheat Sheet TIL: Dolt - Git for Data and Database Version Control TIL: x86 Assembly Programming and SafeEyes Break Reminder TIL: Comprehensive Distributed Systems Reading List TIL: Cosmopolitan C Library, Distributed Systems Book, High Performance Browser Networking, and Rust Roguelike Tutorial
TIL: Message Queue Organization, Readability Guidelines, GNU Stow for Dotfiles, and Essential Unix Tools
2020-12-31 · via Stonecharioteer on Tech

Organizing Background Worker Queues

Organizing Background Worker Queues | Brightball, Inc The Big Little Guide to Message Queues

Comprehensive approach to structuring and managing asynchronous job processing:

Queue Organization Principles:

Queue Separation by Characteristics:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Priority-based queue organization
HIGH_PRIORITY_QUEUE = "urgent_tasks"      # User-facing operations
MEDIUM_PRIORITY_QUEUE = "normal_tasks"    # Business logic
LOW_PRIORITY_QUEUE = "background_tasks"   # Analytics, cleanup

# Function-based separation
EMAIL_QUEUE = "email_notifications"       # Email delivery
PAYMENT_QUEUE = "payment_processing"      # Financial operations
ANALYTICS_QUEUE = "data_processing"       # Reporting and analysis
IMAGE_QUEUE = "image_processing"          # Media manipulation

Queue Design Patterns:

Dead Letter Queue Pattern:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import redis
from celery import Celery

app = Celery('worker')

@app.task(bind=True, max_retries=3)
def process_payment(self, payment_data):
    try:
        # Payment processing logic
        result = payment_gateway.process(payment_data)
        return result
    except PaymentException as exc:
        # Retry with exponential backoff
        raise self.retry(exc=exc, countdown=60 * (2 ** self.request.retries))
    except Exception as exc:
        # Send to dead letter queue for manual inspection
        send_to_dead_letter_queue('payment_failures', {
            'task_id': self.request.id,
            'args': self.request.args,
            'exception': str(exc),
            'timestamp': datetime.utcnow()
        })
        raise

Fan-out Pattern:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
@app.task
def process_user_signup(user_id):
    """Main task that triggers multiple subtasks"""
    # Fan out to multiple specialized tasks
    send_welcome_email.delay(user_id)
    setup_user_analytics.delay(user_id)
    create_user_workspace.delay(user_id)
    notify_sales_team.delay(user_id)

@app.task
def send_welcome_email(user_id):
    # Email-specific processing
    pass

@app.task
def setup_user_analytics(user_id):
    # Analytics setup
    pass

Message Queue Architecture:

Queue Topology Design:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Redis-based queue configuration
queues:
  critical:
    priority: 9
    workers: 4
    retry_policy:
      max_retries: 5
      backoff: exponential

  user_facing:
    priority: 7
    workers: 8
    retry_policy:
      max_retries: 3
      backoff: linear

  background:
    priority: 3
    workers: 2
    retry_policy:
      max_retries: 10
      backoff: exponential

Worker Scaling Strategy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Dynamic worker scaling based on queue depth
import psutil
from celery import Celery

def get_optimal_worker_count(queue_name):
    queue_depth = get_queue_depth(queue_name)
    cpu_count = psutil.cpu_count()
    memory_available = psutil.virtual_memory().available

    if queue_depth > 1000:
        return min(cpu_count * 2, 16)  # Scale up for high load
    elif queue_depth < 10:
        return max(1, cpu_count // 2)  # Scale down for low load
    else:
        return cpu_count  # Normal scaling

Monitoring and Observability:

Queue Health Metrics:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import time
from datadog import statsd

class QueueMonitor:
    def __init__(self, queue_name):
        self.queue_name = queue_name

    def track_task_execution(self, task_func):
        def wrapper(*args, **kwargs):
            start_time = time.time()
            try:
                result = task_func(*args, **kwargs)
                statsd.increment(f'queue.{self.queue_name}.success')
                return result
            except Exception as e:
                statsd.increment(f'queue.{self.queue_name}.error')
                raise
            finally:
                duration = time.time() - start_time
                statsd.histogram(f'queue.{self.queue_name}.duration', duration)
        return wrapper

# Usage
monitor = QueueMonitor('payment_processing')

@monitor.track_task_execution
@app.task
def process_payment(payment_data):
    # Task implementation
    pass

Typography and Readability Guidelines

How Margins and Line Lengths Affect Readability

Evidence-based principles for improving text readability:

Optimal Line Length:

Research-Based Guidelines:

  • 45-75 characters per line: Optimal range for reading speed
  • 50-60 characters: Sweet spot for most content
  • Maximum 90 characters: Beyond this, readability drops significantly
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
/* CSS implementation */
.readable-text {
  max-width: 65ch; /* 65 characters max */
  line-height: 1.6; /* 1.4-1.8 recommended */
  margin: 0 auto; /* Center content */
}

.article-content {
  font-size: 16px;
  max-width: 700px; /* Approximately 65-70 characters */
  margin: 0 auto 2rem;
  padding: 0 1rem;
}

Responsive Typography:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
/* Fluid typography for different screen sizes */
.content {
  font-size: clamp(16px, 2.5vw, 20px);
  line-height: calc(1.4 + 0.2 * ((100vw - 320px) / (1200 - 320)));
  max-width: min(65ch, 90vw);
  margin: 0 auto;
}

/* Breakpoint-based adjustments */
@media (max-width: 768px) {
  .content {
    max-width: 90vw;
    padding: 0 1rem;
  }
}

@media (min-width: 1200px) {
  .content {
    max-width: 60ch; /* Slightly narrower on large screens */
  }
}

Margin and Spacing Principles:

White Space Management:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
.article {
  /* Generous margins improve focus */
  margin: 2rem auto;
  padding: 2rem;

  /* Paragraph spacing */
  p {
    margin-bottom: 1.5rem;
  }

  /* Heading spacing */
  h2 {
    margin-top: 3rem;
    margin-bottom: 1rem;
  }

  h3 {
    margin-top: 2rem;
    margin-bottom: 0.75rem;
  }
}

Reading Flow Optimization:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
/* Improved reading experience */
.readable-content {
  /* Slightly larger text for better readability */
  font-size: 18px;

  /* Optimal line height for reading */
  line-height: 1.7;

  /* Better paragraph distinction */
  p + p {
    margin-top: 1.5rem;
  }

  /* List spacing */
  ul,
  ol {
    margin: 1.5rem 0;
    padding-left: 2rem;
  }

  li + li {
    margin-top: 0.5rem;
  }
}

GNU Stow for Dotfile Management

Using GNU Stow to manage your dotfiles How to store dotfiles | Atlassian Git Tutorial

Elegant solution for managing configuration files across systems:

GNU Stow Concept:

  • Symlink Management: Creates symbolic links from a central directory
  • Package Organization: Group related configs into packages
  • Version Control: Easily track dotfile changes with Git
  • Multiple Environments: Different configs for different machines

Directory Structure:

~/dotfiles/
├── vim/
│   ├── .vimrc
│   └── .vim/
│       ├── colors/
│       └── plugins/
├── zsh/
│   ├── .zshrc
│   ├── .zshenv
│   └── .oh-my-zsh/
├── git/
│   ├── .gitconfig
│   └── .gitignore_global
├── tmux/
│   └── .tmux.conf
└── scripts/
    ├── install.sh
    └── README.md

Stow Commands:

Basic Operations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Navigate to dotfiles directory
cd ~/dotfiles

# Install (create symlinks) for a package
stow vim      # Creates ~/.vimrc -> ~/dotfiles/vim/.vimrc
stow zsh      # Creates ~/.zshrc -> ~/dotfiles/zsh/.zshrc
stow git      # Creates ~/.gitconfig -> ~/dotfiles/git/.gitconfig

# Install all packages
stow */

# Remove symlinks for a package
stow -D vim

# Simulate operation (dry run)
stow -n vim   # Shows what would happen

# Verbose output
stow -v vim   # Shows each symlink created

Advanced Usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Use different target directory
stow -t /usr/local/bin scripts

# Handle conflicts
stow --adopt vim  # Move existing files into stow package

# Recursive operations
stow -R vim       # Restow (remove then install)

# Multiple operations
stow vim zsh git tmux  # Install multiple packages

Dotfile Organization Best Practices:

Modular Configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# ~/.zshrc
# Load modular config files
for config in ~/.config/zsh/*.zsh; do
    [ -r "$config" ] && source "$config"
done

# ~/.config/zsh/aliases.zsh
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
alias ..='cd ..'
alias ...='cd ../..'

# ~/.config/zsh/functions.zsh
mkcd() {
    mkdir -p "$1" && cd "$1"
}

backup() {
    cp "$1" "$1.backup.$(date +%Y%m%d_%H%M%S)"
}

Machine-Specific Configurations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# ~/.zshrc
# Load machine-specific config if it exists
[ -f ~/.zshrc.local ] && source ~/.zshrc.local

# Git config with includes
# ~/.gitconfig
[include]
    path = ~/.gitconfig.local

# Different configs for work/personal
# ~/.gitconfig.work
[user]
    name = "Work Name"
    email = "[email protected]"

# ~/.gitconfig.personal
[user]
    name = "Personal Name"
    email = "[email protected]"

xxd - Hexdump Utility

xxd(1): make hexdump/do reverse - Linux man page

Powerful tool for binary file analysis and manipulation:

Basic Usage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Create hex dump of file
xxd file.bin

# Create hex dump with different grouping
xxd -g 1 file.bin    # Group by 1 byte
xxd -g 2 file.bin    # Group by 2 bytes (default)
xxd -g 4 file.bin    # Group by 4 bytes

# Limit output
xxd -l 256 file.bin  # Show only first 256 bytes
xxd -s 100 file.bin  # Start from byte 100

# Reverse operation (hex to binary)
xxd -r hexdump.txt > restored_file.bin

Practical Applications:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Analyze binary file headers
xxd -l 64 /bin/ls   # Look at ELF header

# Create binary data from hex
echo "48656c6c6f20576f726c64" | xxd -r -p  # "Hello World"

# Patch binary files
xxd file.bin > file.hex
# Edit file.hex with text editor
xxd -r file.hex > file_patched.bin

# Compare binary files
diff <(xxd file1.bin) <(xxd file2.bin)

Hosts File Configuration

hosts(5) - Linux manual page

Essential system file for DNS resolution:

Hosts File Structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# /etc/hosts format
# IP_address canonical_hostname [aliases...]

127.0.0.1       localhost
127.0.1.1       hostname.local hostname
::1             localhost ip6-localhost ip6-loopback

# Custom entries
192.168.1.100   server.local server
10.0.0.50       database.local db

Common Use Cases:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Block websites (redirect to localhost)
127.0.0.1       facebook.com www.facebook.com
127.0.0.1       twitter.com www.twitter.com

# Development environments
127.0.0.1       dev.mysite.com
127.0.0.1       staging.mysite.com
192.168.1.50    api.dev.local

# Network services
192.168.1.10    printer.local
192.168.1.20    nas.local storage
192.168.1.30    router.local gateway

Management Scripts:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#!/bin/bash
# hosts-manager.sh

add_host() {
    local ip="$1"
    local hostname="$2"

    if ! grep -q "$hostname" /etc/hosts; then
        echo "$ip    $hostname" | sudo tee -a /etc/hosts
        echo "Added: $ip -> $hostname"
    else
        echo "Host $hostname already exists"
    fi
}

remove_host() {
    local hostname="$1"
    sudo sed -i "/$hostname/d" /etc/hosts
    echo "Removed: $hostname"
}

list_hosts() {
    grep -v "^#" /etc/hosts | grep -v "^$"
}

# Usage
# ./hosts-manager.sh add 192.168.1.100 myserver.local
# ./hosts-manager.sh remove myserver.local
# ./hosts-manager.sh list

These tools and techniques represent fundamental skills for system administration, development workflow optimization, and maintaining readable, professional documentation and interfaces.