惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

L
LangChain Blog
V
V2EX
爱范儿
爱范儿
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
Martin Fowler
Martin Fowler
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Apple Machine Learning Research
Apple Machine Learning Research
WordPress大学
WordPress大学
有赞技术团队
有赞技术团队
宝玉的分享
宝玉的分享
Last Week in AI
Last Week in AI
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
罗磊的独立博客
小众软件
小众软件
Vercel News
Vercel News
博客园 - 司徒正美
阮一峰的网络日志
阮一峰的网络日志
V
Visual Studio Blog
J
Java Code Geeks
P
Proofpoint News Feed
MongoDB | Blog
MongoDB | Blog
B
Blog
美团技术团队
量子位

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
Migrating a Magento 2 store from utf8 to utf8mb4 without ...
Robin Dhiman · 2026-06-22 · via DEV Community

A customer signed up with an emoji in their display name, and the row saved with everything after the emoji chopped off. No error in the log. The column was utf8, and in MySQL utf8 has never been real UTF-8.

utf8 is a three-byte lie

MySQL's utf8 is an alias for utf8mb3: at most three bytes per character. It covers the Basic Multilingual Plane and stops there. Emoji, many CJK extension characters, and a pile of modern symbols are four bytes, and they do not fit.

What happens when a four-byte character lands in a utf8mb3 column depends on your SQL mode. In strict mode you get Incorrect string value: '\xF0\x9F...'. Without it, MySQL silently truncates the string at the offending byte and saves the rest. The second case is the dangerous one: no error, partial data, found weeks later in a support queue.

utf8mb4 is the fix. It is actual UTF-8, up to four bytes per character. New Magento installs use it. Plenty of stores set up years ago are still on utf8mb3 and inherit every one of these bugs.

The migration looks like one line. It isn't.

The naive version:

ALTER TABLE customer_entity CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Run that across a real Magento schema and you hit this fast:

ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes

An index on a VARCHAR column is sized in bytes, and MySQL budgets for the widest possible character. Under utf8mb3, VARCHAR(255) costs 255 x 3 = 765 bytes, just under the old 767-byte limit. Under utf8mb4 the same column wants 255 x 4 = 1020 bytes, and the index is rejected.

Two ways out, and which one you need depends on your MySQL version:

  • MySQL 5.7 with innodb_large_prefix enabled, or MySQL 8.0 (where it is the default): the index limit is 3072 bytes on DYNAMIC or COMPRESSED row format. Most columns just work. This is where you want to be.
  • Older or misconfigured servers still capped at 767 bytes: indexed string columns have to drop to 191 characters (191 x 4 = 764 bytes) or index a prefix. 191 isn't magic, it is just the largest count that still fits.

Check the row format before you start:

SHOW TABLE STATUS WHERE Name = 'customer_entity';

The column is only half of it

Converting the column does nothing if the application still talks to the database as utf8mb3. Character set is negotiated on the connection. If the client opens with SET NAMES utf8, a four-byte character is mangled in transit before it ever reaches your new utf8mb4 column.

So the migration is three coordinated changes, not one:

  1. Database and tables: ALTER ... CONVERT TO CHARACTER SET utf8mb4. CONVERT TO rewrites the table and converts existing data, not just the default for new rows.
  2. Server defaults in my.cnf (character-set-server, collation-server), so new tables are born correct.
  3. The application connection charset, so reads and writes negotiate utf8mb4 end to end.

Miss the third and you will swear the migration worked from the mysql CLI while the storefront keeps corrupting data.

Two things that bite mid-migration

CONVERT TO is a full table rewrite. On a large sales_order or catalog_product_entity_varchar table that means a long lock. Do it in a maintenance window, or run it through pt-online-schema-change or gh-ost so the table stays writable.

Collations have to match across joins. Convert some tables and not others and the next query that joins them throws Illegal mix of collations. Pick one collation, apply it everywhere, and don't leave half the schema on the server default. utf8mb4_unicode_ci is a safe, widely compatible choice.

The lesson isn't really about Magento

This is a MySQL story, not a Magento one. Any application sitting on utf8mb3 carries the same latent bug and the same three-part fix. Magento just surfaces it early, because catalogs and customer data are full of exactly the international text and emoji that four bytes were invented for.

If you are still on utf8, you don't have a Unicode-safe store. You have one that hasn't met the wrong character yet.