惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
Schneier on Security
博客园 - 【当耐特】
人人都是产品经理
人人都是产品经理
大猫的无限游戏
大猫的无限游戏
Last Week in AI
Last Week in AI
IT之家
IT之家
V
V2EX
CTFtime.org: upcoming CTF events
CTFtime.org: upcoming CTF events
博客园 - 叶小钗
Martin Fowler
Martin Fowler
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
M
MIT News - Artificial intelligence
博客园 - 聂微东
MyScale Blog
MyScale Blog
云风的 BLOG
云风的 BLOG
The Cloudflare Blog
WordPress大学
WordPress大学
Microsoft Azure Blog
Microsoft Azure Blog
阮一峰的网络日志
阮一峰的网络日志
酷 壳 – CoolShell
酷 壳 – CoolShell
博客园 - 三生石上(FineUI控件)
Cyberwarzone
Cyberwarzone
Scott Helme
Scott Helme
H
Help Net Security
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
The GitHub Blog
The GitHub Blog
Cisco Talos Blog
Cisco Talos Blog
D
DataBreaches.Net
P
Palo Alto Networks Blog
T
Threatpost
P
Privacy & Cybersecurity Law Blog
L
LINUX DO - 最新话题
Know Your Adversary
Know Your Adversary
C
CXSECURITY Database RSS Feed - CXSecurity.com
B
Blog
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
P
Proofpoint News Feed
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
Cloudbric
Cloudbric
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
N
News and Events Feed by Topic
T
The Blog of Author Tim Ferriss
L
Lohrmann on Cybersecurity
量子位
Security Latest
Security Latest
Google DeepMind News
Google DeepMind News
Y
Y Combinator Blog
aimingoo的专栏
aimingoo的专栏
Forbes - Security
Forbes - Security
T
Tailwind CSS Blog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant Common SOC 2 Failures (Real World) Stop Vibe-Checking Your AI App: A Practical Guide to Evals How to Use SonarQube and SonarScanner Locally to Level Up Your Code Quality Your Next To-Do App Is Dead — I Replaced Mine with an OpenClaw AI Sign a Nostr event in 60 lines of Python using coincurve — no nostr-sdk, no nbxplorer, no rust toolchain ITGC Audit Explained Like You’re in Big 4 Patch Tuesday abril 2026: Microsoft parcha 163 vulnerabilidades y un zero-day en SharePoint Stop scraping everything: a better way to track competitor price changes Listing on MCPize + the Official MCP Registry while routing payments OUTSIDE the marketplace — how I kept 100% of my x402 revenue Building an AI-Powered Risk Intelligence System Using Serverless Architecture Why We Ripped Function Overloading Out of Our AI Toolchain Testing AI-Generated Code: How to Actually Know If It Works SaaS Churn Is Killing Your Business. Here Is What to Do About It (Without a Support Team) The Speed of AI Is No Longer Linear - And Self-Improving Models Are Why How to Implement RBAC for MCP Tools: A Practical Guide for Engineering Teams From Standard Quote to Persuasive Proposal: AI Automation for Arborists I built a CLI that scaffolds complete multi-tenant SaaS apps Axios CVE-2025–62718: The Silent SSRF Bug That Could Be Hiding in Your Node.js App Right Now The dashboard that ended our friendship Data Pipelines Explained Simply (and How to Build Them with Python) The Hidden Cost of AI Systems Nobody Talks About. undefined vs undeclared, and how typeof behaves Switching from file-based jobs to NATS/Kafka in Rust without changing code io_uring Adventures: Rust Servers That Love Syscalls Why Agentic AI is Killing the Traditional Database The POUR principles of web accessibility for developers and designers Quantum Neural Network 3D — A Deep Dive into Interactive WebGL Visualization How To Install Caveman In Codex On macOS And Windows Automation Pipeline Reliability: Why Your Workflow Breaks When Nobody Is Watching I Built an 'Open World' AI Coding Agent — It Works From ANY Folder From Freelancing to Product: A Tech Service Company's SaaS Transformation China's AI Giants: Adding Tencent Hunyuan & ByteDance Doubao to AI University (74 Providers) On the Vibe Coders and Their Lies clerk: Auto-Summarize Your Claude Code Sessions AI Weekly — 2026/04/10–04/17 | The Model Lockdown Is Here, but the Toolchain Is the Real Battleground AI 週報 — 2026/04/10–2026/04/17 模型封鎖潮來了,但工具鏈才是真戰場 Maybe this is how Open-Source apps are born... 🚀 Fine-Tune LLMs with LoRA and QLoRA: 2026 Guide tRPC v11 + Next.js App Router: End-to-End Type Safety Without the Boilerplate ShadCN UI in 2026: Why I Stopped Installing Component Libraries and Started Owning My Components SaaS Billing in React Server Components: Stripe + Supabase Without a Single `useEffect` Join our DEV Weekend Challenge — $1,000 in Prizes Across TEN winners! Submissions Due April 20 at 6:59 AM UTC. Implementing FSRS Spaced Repetition in Flutter + Supabase — Adding Memory Science to an AI Learning App "I Texted My Localhost From the Train — Claude Code Fixed the Bug Before I Got Home" I Built a Sales Prep AI and It Went Deeper Than Expected Design to Code #2: One JSON, Eleven Outputs Solving the 100M-Row Problem: A Summary Table Pattern for High-Volume Push Notification Logs Flutter Web With Wasm: What Actually Changes For Developers I Built 50 Royalty-Free Soundtracks for My Side Project in a Weekend Using AI Music Generation The Vibe Coding Security Checklist: 7 Things to Check Before You Ship Stop Letting Googlebot Guess Fix Your React App's SEO Right Desconstruindo o Streaming do LinkedIn: Como Criar um Engine de Extração de Vídeo de Alta Performance com HLS e FFmpeg (EDA Part-1) EDA (Exploratory Data Analysis) Explained With Real Life — Why Looking at Your Data Is the Most Important Step in Machine Learning Brand Relationship Management at Scale: Our 4-Touch Outreach System for 200+ Brands Why String.fromEnvironment() Might Return an Empty String in Dart JGuardrails 1.0.0 — Hardening Java LLM Apps Against Jailbreaks, Toxicity, and Prompt Injection Plan and Schedule a Full Week of Threads Content From One Claude Conversation Coding Cat Oran Ep3, Five Tables Changed Everything Updated: BFF Pattern I'm done watching freelancers get buried by 200 proposals. So I'm building the alternative. This is my first post BFS Algorithm in Java Step by Step Tutorial with Examples Tracking LLM Pricing Monthly: An Open Dataset for 22 AI Models How We Measure Content ROI on a Comparison Site: Revenue Attribution Without Perfect Data Introducing Nova AI Ops: The AI-Native Operating System for SRE Teams I built a free desktop video downloader for Windows — Grabbit How Talkie OCR Helps Vision-Impaired & Dyslexic Users Read the World Around Them VRCFaceTracking安装和iPhone面捕配置教程,有bug Even CrowdStrike Can't See Your Agents The Automation Gold Rush: What n8n Workflows and Claude Are Opening Up for Developers Right Now
Searching Emojis With Casual Japanese Keywords — Why Unicode CLDR's ja Annotations Aren't Enough
SEN LLC · 2026-05-11 · via DEV Community

If you type "わらう" (laugh) into Slack's emoji search, you get nothing. The same is true for "ぴえん" (a sad-cute slang for crying that took over Japanese Twitter around 2020), or "ばんざい" (a celebratory "hooray"). Unicode's CLDR (Common Locale Data Repository) ships official Japanese annotations for every emoji — but they're written in the register of an accessibility caption, not a search query. I curated 107 emojis by hand with the casual Japanese tags people actually type, and wrapped them in a ~200-line browser search.

emoji-search-jp UI: dark theme. The search input contains

🌐 Demo: https://sen.ltd/portfolio/emoji-search-jp/
📦 GitHub: https://github.com/sen-ltd/emoji-search-jp

Why CLDR's Japanese annotations don't make a search index

Unicode ships annotations for every emoji in every CLDR locale. Each entry has a "name" plus a handful of "keywords". For Japanese, the file looks like this:

Emoji Name Keywords
😂 "うれしなき" ["うれしなき", "かお", "かおえもじ"]
🥺 "もの欲しそうな顔" ["かお", "かおえもじ", "けんめい", "もの欲しそう"]
🙏 "合掌した手" ["お辞儀", "かたを下げる", "かんしゃ"]
🎉 "クラッカー" ["クラッカー", "ハッピー", "パーティー", "紙ふぶき"]
🐶 "犬の顔" ["いぬ", "おもしろい", "かお", "どうぶつ"]

The accessibility job that produced these keywords is the right job for CLDR to do. They cover the visual content of the emoji in a formal-register, all-hiragana style that a screen reader can announce.

What they don't cover is what users actually type:

  • 😂: missing わらう / lol /
  • 🥺: missing ぴえん (the slang that defines this emoji in 2020s Japanese)
  • 🙏: missing お願い / ありがとう / ごめん — the contexts every user uses this emoji for
  • 🎉: missing おめでとう
  • 🐶: missing わんこ / ワン

There's an upstream proposal to expand CLDR annotations toward search use cases, but the file as it ships today is a captioning dictionary, not a search dictionary. The two have different shapes.

What this repo ships instead

107 emojis with 5-9 hand-curated tags each, totaling about 750 tag entries. Same schema as CLDR ({char, name_ja, name_en, tags, category}) so the two can be merged if you want both registers.

{
  "char": "😂",
  "name_en": "face with tears of joy",
  "name_ja": "嬉し泣きの顔",
  "tags": ["わらう", "大爆笑", "笑い泣き", "嬉し泣き", "lol"],
  "category": "face"
},
{
  "char": "🥺",
  "name_en": "pleading face",
  "name_ja": "うるうる目の顔",
  "tags": ["ぴえん", "かわいい", "うるうる", "おねがい", "切ない"],
  "category": "face"
},
{
  "char": "🙏",
  "name_en": "folded hands",
  "name_ja": "合掌",
  "tags": ["お願い", "おねがい", "ありがとう", "祈る", "感謝", "ごめん"],
  "category": "gesture"
}

Enter fullscreen mode Exit fullscreen mode

Tag-selection rules I followed while curating:

  1. Mix kana and kanji. "ねこ" and "猫" both belong on 🐱 because the user may or may not have committed an IME conversion.
  2. Mix register. Both the slang ("ぴえん") and the descriptive ("切ない") for 🥺.
  3. Borrow from CLDR. Take the official annotations as a baseline and add the spoken-Japanese coverage on top.
  4. A handful of English tags. lol / ok / love / cool — common enough in mixed-language Japanese chat that they earn their slot.

What I deliberately didn't do: shoot for "all 3700 emoji from the Unicode emoji-list". The lexicon is the product. A smaller, well-tagged set is more useful than a complete, badly-tagged one.

Weighted scoring, five tiers

Search is a linear scan with a five-tier scoring function. For each query token, against each emoji:

const SCORE = {
  TAG_EXACT:           10,  // token === a tag
  TAG_PREFIX:          7,   // token is a prefix of a tag
  TAG_SUBSTRING:       4,   // token is a substring of a tag (non-prefix)
  NAME_JA_SUBSTRING:   3,   // token in name_ja
  NAME_EN_SUBSTRING:   1,   // last-resort English fallback
};

export function scoreToken(emoji, token) {
  if (!token) return 0;
  let best = 0;
  for (const tag of emoji.tags) {
    const t = normalize(tag);
    if (t === token) return SCORE.TAG_EXACT;            // can't beat exact
    if (t.startsWith(token))    best = Math.max(best, SCORE.TAG_PREFIX);
    else if (t.includes(token)) best = Math.max(best, SCORE.TAG_SUBSTRING);
  }
  if (best > 0) return best;
  if (normalize(emoji.name_ja).includes(token)) return SCORE.NAME_JA_SUBSTRING;
  if (normalize(emoji.name_en).includes(token)) return SCORE.NAME_EN_SUBSTRING;
  return 0;
}

Enter fullscreen mode Exit fullscreen mode

The non-obvious choice is the weight gap between TAG_SUBSTRING (4) and NAME_JA_SUBSTRING (3) — a substring hit on a hand-curated tag still beats a coincidental substring hit in the descriptive name. So ranks "tags containing 顔" above "emojis whose name happens to contain 顔". Without the gap, every face emoji would tie with every cat emoji because both have in their name_ja.

Multi-token AND with sum-of-scores ranking

Whitespace splits the query into tokens. The default behaviour is every token must contribute at least one point, otherwise the emoji is dropped:

export function scoreEmoji(emoji, tokens) {
  if (tokens.length === 0) return 0;
  let sum = 0;
  for (const tok of tokens) {
    const s = scoreToken(emoji, tok);
    if (s === 0) return -1;   // any miss → drop
    sum += s;
  }
  return sum;
}

Enter fullscreen mode Exit fullscreen mode

So "わらう 顔" keeps 😂 (10 from tag わらう + 3 from name_ja containing = 13) and drops 🐱 (3 from but no signal for わらう).

The unit tests pin this:

test("scoreEmoji returns -1 if any token fails to match", () => {
  const s = scoreEmoji(SAMPLE_FACE_WITH_TEARS, ["わらう", ""]);
  assert.equal(s, -1);  // dropped
});

test("search drops emojis that don't satisfy ALL tokens", () => {
  const results = search(SAMPLE, "わらう 顔");
  const chars = results.map((r) => r.emoji.char);
  assert.ok(chars.includes("😂"));
  assert.ok(!chars.includes("🐱"));
});

Enter fullscreen mode Exit fullscreen mode

NFKC normalization upfront

Users will type half-width and full-width letters, with stray IME spaces, and case-mixed. One normalize call handles all of it:

export function normalize(s) {
  return String(s).normalize("NFKC").toLowerCase().trim();
}

Enter fullscreen mode Exit fullscreen mode

LOL (full-width) becomes lol; わらう becomes わらう; Pien becomes pien. Everything downstream operates on the normalized form. The boundary is one line and gets tested:

test("search is case- and width-insensitive (NFKC)", () => {
  const results = search(SAMPLE, "LOL");
  assert.equal(results[0].emoji.char, "😂");
});

Enter fullscreen mode Exit fullscreen mode

Stable sort matters more than the algorithm

Array.prototype.sort has been stable since ECMA-2019, so equal-keyed elements keep their original order. But equal scores are common in this dataset — lands at NAME_JA_SUBSTRING (3) for every face emoji — so I made the tie-breaker explicit:

matches.sort((a, b) => {
  if (b.score !== a.score) return b.score - a.score;
  return a.idx - b.idx;   // tie-break by input order
});

Enter fullscreen mode Exit fullscreen mode

The effect: when many emojis tie on score, the curated order (the order I wrote them into data.json) survives. I put the most-used emoji first in every category, so ties resolve to "the more-used one".

test("search is stable: equal-scoring matches keep input order", () => {
  // "顔" → 3 emojis all at NAME_JA_SUBSTRING (3).
  const results = search(SAMPLE, "");
  const chars = results.map((r) => r.emoji.char);
  assert.deepEqual(chars, ["😂", "🥺", "🐱"]);  // input order preserved
});

Enter fullscreen mode Exit fullscreen mode

What I didn't build

  • Trie / inverted index — 107 entries × 7 tags is 0.1 ms of linear scan on a phone. The index only pays off past ~10,000 entries.
  • Fuzzy / Levenshtein matching — typo tolerance complicates the score function. Prefix + substring already cover ~80% of real misses, and the cost of adding fuzzy is a noticeable jump in false positives.
  • Skin-tone / gender variants — exploding the entry count by 5-10× hurts search quality. Native OS emoji pickers cover this better.
  • Speech-readout accessibility annotations — that's exactly what CLDR is for. This repo is the other annotation register.

Try it

The demo at https://sen.ltd/portfolio/emoji-search-jp/ has the full lexicon. Try わらう, ぴえん, ばんざい, , ハート, 寿司, お願い, ありがとう. / focuses the search box. Click an emoji to copy it.

Source: https://github.com/sen-ltd/emoji-search-jp — MIT, ~200 lines of JS plus 107 entries of curated data, 18 unit tests, no build step, no runtime dependencies.


🛠 Built by SEN LLC as part of an ongoing series of small, focused developer tools. Browse the full portfolio for more.