惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

P
Proofpoint News Feed
The Last Watchdog
The Last Watchdog
Security Latest
Security Latest
P
Privacy International News Feed
T
Threat Research - Cisco Blogs
H
Help Net Security
T
The Exploit Database - CXSecurity.com
Know Your Adversary
Know Your Adversary
博客园_首页
S
Securelist
S
Schneier on Security
G
GRAHAM CLULEY
Cisco Talos Blog
Cisco Talos Blog
V
Visual Studio Blog
博客园 - 叶小钗
C
Cybersecurity and Infrastructure Security Agency CISA
有赞技术团队
有赞技术团队
Recent Announcements
Recent Announcements
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
Microsoft Azure Blog
Microsoft Azure Blog
A
About on SuperTechFans
博客园 - 三生石上(FineUI控件)
Stack Overflow Blog
Stack Overflow Blog
量子位
L
Lohrmann on Cybersecurity
Hugging Face - Blog
Hugging Face - Blog
Engineering at Meta
Engineering at Meta
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
C
CXSECURITY Database RSS Feed - CXSecurity.com
A
Arctic Wolf
P
Privacy & Cybersecurity Law Blog
Simon Willison's Weblog
Simon Willison's Weblog
S
SegmentFault 最新的问题
The Hacker News
The Hacker News
罗磊的独立博客
博客园 - 司徒正美
D
Darknet – Hacking Tools, Hacker News & Cyber Security
博客园 - 【当耐特】
Microsoft Security Blog
Microsoft Security Blog
K
Kaspersky official blog
人人都是产品经理
人人都是产品经理
博客园 - 聂微东
L
LINUX DO - 热门话题
Threat Intelligence Blog | Flashpoint
Threat Intelligence Blog | Flashpoint
V
V2EX
V
Vulnerabilities – Threatpost
AWS News Blog
AWS News Blog
小众软件
小众软件
Project Zero
Project Zero

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
How to scrape RedNote (Xiaohongshu) with Python in 2026 — the auth/signing problem and how to handle it
Sami · 2026-04-25 · via DEV Community

RedNote (Xiaohongshu, 小红书, sometimes "Little Red Book" or just XHS) is the platform a lot of Western teams realized they needed to monitor in 2024-2025, when the TikTok regulatory mess in the US sent millions of users — and brand attention — toward Chinese platforms. It's now China's #1 lifestyle and product-discovery network, with 300M+ monthly active users and a search-driven discovery model that makes it different from every other Chinese social platform.

The problem: there's no official public API. Western teams who try to monitor it usually end up either (a) paying enterprise vendors $20-50k/year for limited China coverage, or (b) trying to scrape it themselves and discovering that RedNote has one of the more aggressive anti-scraping stacks in Chinese social.

This article walks through the actual technical challenges and shows you both DIY and hosted approaches with real Python code. I've shipped a hosted RedNote scraper on Apify that I'll mention later — but the goal here is for you to understand the problem space well enough to make an informed build-vs-buy decision, not to sell you anything.

What RedNote actually serves

Before we go technical: what data does RedNote expose, and what's actually useful?

A RedNote post is structured roughly like this:

  • Title (often very short, sometimes empty)
  • Body text — long-form description with product mentions, hashtags, location tags
  • Image carousel — 1-9 images. Critical: a non-trivial portion of product info lives in image text overlays, not in the body
  • Engagement metrics — likes, saves, comments, shares
  • Author profile — username, avatar, follower/following counts, bio, verification, location
  • Tags / categories — hashtags and platform-assigned categories

For most monitoring use cases, the metric that matters more than likes is saves. Saves on RedNote are the closest equivalent to "I want to buy this later" — they correlate with purchase intent. Likes on RedNote are casual engagement, similar to Twitter likes.

Profile data is structured similarly:

  • User ID, RedNote ID (red ID), nickname, avatar
  • Bio / description
  • Follower / following counts
  • Location, gender, profile tags
  • Total likes received across all posts
  • Verification status

The technical challenges (why this is harder than scraping Twitter)

If you've scraped Western social platforms, your default toolkit is probably httpx or requests plus maybe a residential proxy. RedNote is going to break each of those defaults.

Challenge 1: TLS fingerprinting

RedNote uses TLS fingerprinting (specifically JA3/JA4) to identify and block requests that don't come from real browsers. The requests library has a Python-specific TLS fingerprint that RedNote's bot-detection layer recognizes immediately.

The standard fix is to use curl_cffi, which lets you spoof a Chrome or Safari TLS fingerprint:

from curl_cffi import requests as curl_requests

# Spoof Chrome 120's TLS fingerprint
response = curl_requests.get(
    "https://www.xiaohongshu.com/explore",
    impersonate="chrome120"
)

Enter fullscreen mode Exit fullscreen mode

This alone gets you past the first layer of detection.

Challenge 2: Request signing

RedNote signs every API request with a value called x-s (sometimes seen as xs) plus other parameters like x-t and x-s-common. These are computed client-side from a JavaScript function in their web app.

The signing function changes roughly monthly. When it changes, every scraper using the old signing logic breaks until someone reverse-engineers the new function.

Here's roughly what you need to do:

# Pseudo-code — actual signing logic is more complex

import time
import json
import hashlib

def generate_signing_headers(url_path: str, body: dict) -> dict:
    """
    The actual logic is reverse-engineered from RedNote's web client JS.
    This requires reading their obfuscated bundle and reproducing it in Python.
    """
    timestamp = str(int(time.time() * 1000))
    body_str = json.dumps(body, separators=(',', ':')) if body else ""

    # Real implementation involves:
    # - Specific input concatenation order
    # - Custom hashing scheme (not standard HMAC)
    # - Several "magic constants" that change when RedNote rotates
    # - Sometimes a captcha-derived token

    raw = f"url={url_path}&data={body_str}&t={timestamp}"
    x_s = hashlib.md5(raw.encode()).hexdigest()  # This is NOT the actual algorithm

    return {
        "x-s": x_s,
        "x-t": timestamp,
        # x-s-common is computed separately
    }

Enter fullscreen mode Exit fullscreen mode

The actual signing algorithm is more complex than what I've shown. There are open-source libraries that have reverse-engineered it (xhs-api and similar on GitHub) — they get you most of the way there, but expect to patch them when RedNote rotates.

Challenge 3: IP-level rate limiting and datacenter blocking

RedNote blocks requests from datacenter IPs (AWS, GCP, Azure, DigitalOcean, etc.) within minutes. You need residential proxies, ideally with Chinese geolocation or at least Asia-Pacific.

Even with residential IPs, there's a per-IP rate limit. Realistic throughput is around 10-20 requests per minute per IP before you start getting 412/418 errors and eventually IP bans.

Challenge 4: SPA / dynamic rendering for some endpoints

Search and the explore feed are loaded via AJAX after initial page load, but a few endpoints (some user pages, certain post types) only render their data in the Vue.js application state. You either need to extract data from the inline <script> tag (look for window.__INITIAL_STATE__) or render with Playwright.

Challenge 5: Login walls on certain features

True keyword-filtered search requires login. Without login, you get the explore feed (trending/recommended for your keyword), which is useful but not the same. This is a structural product limitation, not a scraping limitation — you can scrape it the same way logged-in users see it, you just need to either provide cookies or accept the explore-feed fallback.

Approach 1: Build it yourself

If you have ops capacity to maintain it (someone who can read JavaScript and reverse-engineer signing functions monthly), DIY is feasible. Here's a minimal example using curl_cffi plus an open-source signing library.

First install:

pip install curl_cffi xhs

Enter fullscreen mode Exit fullscreen mode

Note: xhs is one of several open-source libraries on GitHub that wrap RedNote's API. Check their commit history before depending on one — the abandoned ones break monthly.

from xhs import XhsClient
from xhs.exceptions import DataFetchError

# You need to provide your own cookies and signing function URL
# The 'sign' function comes from the library's reverse-engineered JS
def sign(uri, data=None, a1="", web_session=""):
    # Implementation provided by the library
    # When RedNote rotates, you'll need to update this
    pass

client = XhsClient(
    cookie="abRequestId=...; webBuild=...; xsecappid=xhs-pc-web; ...",
    sign=sign
)

try:
    # Get a user's posts
    user_id = "5cfbc3f10000000018023ebb"
    posts = client.get_user_notes(user_id)

    for post in posts.get("notes", []):
        print(f"Title: {post.get('display_title')}")
        print(f"Likes: {post.get('interact_info', {}).get('liked_count')}")
        print("---")

except DataFetchError as e:
    print(f"RedNote rejected the request: {e}")
    # Common causes:
    # - Signing function out of date (update from upstream)
    # - Cookie expired (re-login)
    # - IP throttled (rotate proxy)

Enter fullscreen mode Exit fullscreen mode

The cookie you need comes from logging into RedNote in a browser and copying the relevant cookies from DevTools. The cookies expire — typically a few days — so you'll need to refresh them periodically.

Here's the honest cost breakdown for DIY:

Cost component Estimate
Initial setup (researching libraries, getting first scrape working) 8-16 hours
Residential proxy (Bright Data, Oxylabs, etc.) $50-200/month for moderate volume
Per-incident maintenance when RedNote rotates 4-8 hours, 1-2x/month
Ongoing: cookie refresh, error handling 1-2 hours/week

If you have a developer whose time is worth $50-100/hour, DIY is around $400-1000/month all-in for moderate scraping volumes.

Approach 2: Use a hosted scraper

The build-vs-buy math changes if you don't have someone on the team who can read reverse-engineered JavaScript and patch signing logic. Hosted Apify Actors handle that for you.

Several developers (including me) maintain RedNote scrapers on Apify. Mine is:

Using it from Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

# Search
run = client.actor("zhorex/rednote-xiaohongshu-scraper").call(run_input={
    "mode": "search",
    "searchQuery": "skincare routine",
    "maxResults": 50,
    "filterByMinLikes": 100  # Only return posts with 100+ likes
})

# Iterate over results
for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"Title: {post['title']}")
    print(f"Likes: {post['likes']}")
    print(f"Author: {post['author']['nickname']}")
    print(f"URL: {post['postUrl']}")
    print("---")

Enter fullscreen mode Exit fullscreen mode

The output JSON is flat:

{
  "mode": "search",
  "postId": "69d269310000000023017e07",
  "postUrl": "https://www.xiaohongshu.com/explore/69d269310000000023017e07",
  "type": "normal",
  "title": "Morning skincare routine for dry skin",
  "images": ["https://sns-webpic-qc.xhscdn.com/..."],
  "likes": 15234,
  "author": {
    "userId": "575d32285e87e733f0162c0a",
    "nickname": "BeautyQueen",
    "avatar": "https://sns-avatar-qc.xhscdn.com/..."
  },
  "scrapedAt": "2026-04-25T21:14:30Z"
}

Enter fullscreen mode Exit fullscreen mode

This is one option among several on Apify Store. EasyApi has the most users by volume; OrbitData Labs has a different all-in-one approach. Pricing is roughly the same across them ($5/1000 ± $1). Differences are in:

  • Output schema (some return RedNote's raw nested API response, some flatten it)
  • Update frequency (some are abandoned and break for weeks at a time)
  • Mode coverage (some only do search; others handle profiles, comments, videos, etc.)
  • Issue response time

If you're evaluating, run a free-tier test on 2-3 of them with the same input and compare what you get back. The free tier costs you nothing.

When does each approach make sense?

DIY (build it yourself):

  • You have a Chinese-language ops team and can monitor breakages
  • You're processing > 1M posts/month (the per-result cost of hosted starts to add up)
  • You need to scrape behind login (which means you need cookies from logged-in accounts you control)
  • You have specific data needs that no hosted scraper covers

Hosted Apify actor:

  • You don't have a dedicated scraper engineer
  • Volume is variable or moderate (< 500k posts/month)
  • You want to outsource the cat-and-mouse with RedNote's anti-bot updates
  • You're prototyping and want to validate the approach before committing

The middle ground that often makes sense: use a hosted actor for production data flow, build a thin DIY layer for any specific endpoints the hosted version doesn't cover. The hosted scraper handles the maintenance burden on the parts that break most often (search, profile, posts), and you keep custom DIY logic for the edges.

What you do with the data downstream

Scraping is one third of the problem. The other two:

Sentiment analysis on Chinese text. Off-the-shelf Chinese BERT models (like bert-base-chinese from Huggingface) are a starting point but accuracy varies wildly by domain. RedNote slang, in particular, doesn't appear in the training data of general Chinese sentiment models — fine-tuning on RedNote-specific labeled samples gets you significant accuracy lift if accuracy matters.

Image text extraction. A non-trivial portion of product mentions on RedNote live in image text overlays (Chinese users frequently put product names visible in images, not in the post body). PaddleOCR is the open-source standard for Chinese OCR. Slow (~30 seconds per image) but reliable. Adds significant cost to processing pipelines but you'll miss a measurable percentage of product mentions without it.

Both of these are downstream of scraping — solve scraping first, then layer.

FAQ

Is scraping RedNote legal?

Public-data scraping legality varies by jurisdiction. RedNote's ToS prohibits automated access (as do most platform ToS). The Apify approach (and most public-scraping infrastructure) treats public web pages as accessible, the same way Google's crawler would. You should consult legal counsel for your specific use case. Not legal advice.

How fast can I scrape RedNote?

Realistic sustained throughput per IP is around 10-20 requests per minute before triggering rate limits. With residential proxy rotation and proper backoff, you can scale this horizontally — Apify's actor handles this internally. For DIY, plan for ~1 result per second per IP as a conservative number.

Do I need a Chinese IP?

Not strictly required, but residential IPs (Asian residential preferred) have notably higher success rates than US/European residential. Datacenter IPs are blocked outright.

What's xsec_token and why does it matter?

When users share posts via the RedNote app or copy URLs, those URLs include an xsec_token query parameter that authenticates the link request. Some scrapers don't handle URLs with xsec_token correctly and return errors. If you're scraping URLs collected from real users, make sure your tooling supports this.

Can I scrape video files from RedNote posts?

Yes. Video URLs are returned in the post metadata. Direct download from those URLs works without authentication for public videos.

How often does the request signing change?

Roughly monthly, sometimes more frequently around major RedNote app updates. If you're doing DIY, plan to dedicate 4-8 hours per rotation to update your signing function, or rely on an actively maintained library that pushes updates fast.

What's the difference between RedNote and Xiaohongshu?

They're the same platform. "Xiaohongshu" (小红书) is the Chinese name and means "Little Red Book". "RedNote" is the English brand they pushed during the 2024-2025 TikTok migration to be more accessible to global users. Same app, same data, same API endpoints.


If you're working on China market intelligence, brand monitoring, or competitive research and want to compare notes — drop me a comment. I write about Chinese platform scraping (Weibo, Bilibili, RedNote) and the build-vs-buy trade-offs around them.

The actor I mentioned: apify.com/zhorex/rednote-xiaohongshu-scraper. Free tier covers ~1,000 results, which is enough to validate against your specific use case before committing.