Building a Hebrew-First SEO Plugin (“Project Spark”) — Part 8 of 20

A client reported that a Hebrew meta description containing a 💰 emoji arrived in their SEO data as ?, with a stray word glued to the front. The instinctive assumption — “the plugin corrupted it” — turned out wrong, and proving it wrong in minutes instead of days is what this post is about: building byte-fidelity certification into your test suite.

Key takeaway: MySQL’s utf8 charset only stores 3-byte characters; emoji are 4 bytes and get destroyed on write. If your WordPress tables aren’t utf8mb4, your SEO pipeline is corrupting data before any plugin ever sees it — and only byte-level round-trip tests can prove where the fault lies.

Why character encoding is an SEO problem, not just a database problem

It is tempting to file encoding under “infrastructure” and move on. For a Hebrew-first business, that filing error is expensive. SEO data is text: titles, meta descriptions, Open Graph fields, structured data, alt attributes. Every one of those strings travels a long pipeline — editor input, sanitization, storage, retrieval, rendering, and finally a crawler’s parser — and any stage that mishandles bytes changes what Google, social networks, and AI answer engines actually read.

Corrupted SEO text is worse than missing SEO text. A meta description with ? in the middle of a sentence signals carelessness to every human who sees it in a search result, and it changes the meaning machines extract. On a site where descriptions are written in Hebrew, punctuated with the occasional emoji, and sometimes mixed with Latin brand terms, the number of ways a string can be silently damaged is far higher than on an ASCII-only site. That is why Project Spark treats byte fidelity as a first-class SEO requirement, with the same test rigour as rankings-facing features.

A short history of MySQL’s naming mistake

The root of the most common corruption is a naming decision made two decades ago. MySQL’s charset called utf8 is not UTF-8. It is a truncated encoding, capped at three bytes per character, that predates the parts of Unicode most people now use daily. Real UTF-8 needs up to four bytes; every emoji, a large group of historic and mathematical symbols, and a meaningful slice of CJK characters live in that fourth byte.

MySQL later added a correct implementation and called it utf8mb4 (“mb4” = multi-byte, four). WordPress switched its default to utf8mb4 years ago — but defaults only apply to new tables. Sites that started life earlier, or were migrated between hosts with older dumps, still carry utf8 tables today. Nothing warns the owner. The database accepts every write and silently substitutes ? for any 4-byte character. The site works. The content looks fine in Latin and even in Hebrew, which fits in three bytes. Then one editor pastes an emoji into a meta description, and years later somebody asks why the plugin “corrupted” it.

The three-suspect lineup

When multilingual SEO data arrives mangled, there are always three suspects:

1. The database charset. Tables created as utf8 (MySQL’s misleadingly named 3-byte encoding) silently convert 4-byte characters — emoji, some CJK, many symbols — into ?. Sites migrated from old WordPress installs carry this hidden risk for years.
2. A filter chain. WordPress and plugin ecosystems run text through filters at save and render time. Any hook in the chain — a translation plugin, a theme function, a security plugin — can strip or rewrite characters. The site owner rarely knows which plugins register which filters, and the filters run invisibly on every save.
3. Your own code. Sanitizers that strip “unexpected” characters, encoders applied twice, truncation that splits a multibyte character mid-codepoint and produces invalid UTF-8 from valid input.

Without evidence, everyone blames suspect #3 — your plugin, because it is the most recent arrival and the one whose vendor is answering the support ticket. With evidence, you can point at the real culprit in minutes. Producing that evidence before the dispute exists is the whole idea of certification.

Byte-fidelity certification: the evidence machine

Project Spark’s answer was a certification suite: automated tests that push a hostile-character corpus through every write surface of the plugin and assert byte-identical recovery. The corpus covers thirteen categories: 4-byte emoji, Hebrew with RTL marks, Arabic, Chinese, Japanese, HTML entities, literal HTML, newlines, tabs, mixed content, punctuation runs like ?! and >>, Windows-style backslash paths, and quotes.

Each category exists because something in that category once broke, somewhere, in some system. Backslashes are on the list because WordPress core’s metadata API strips slashes on write unless the caller compensates — a behaviour that quietly destroyed values like C:\Users\admin until we added the canonical compensation on every write path (Part 6 tells that story in full). RTL marks are on the list because they are invisible and therefore never caught by eye (Part 7). Emoji are on the list because of the incident that opens this post.

The mechanics matter less than the contract: for every category, the test writes the hostile value through the plugin’s real write path — not a shortcut into the database — reads it back through the real read path, and asserts that the recovered bytes are identical to the submitted bytes. Not “similar”, not “renders the same”: identical. A single flipped byte fails the build.

One certification is never enough: the sister-surface audit

The first version of the suite certified only the data importer, because that is where the first corruption reports pointed. Then we applied a habit that shows up throughout this series (Part 14): after any fix, hunt the same defect class across the whole codebase.

The audit found three sister surfaces with the same risk profile that had no emoji coverage at all: the editor’s metabox save (an operator typing a title by hand), the setup wizard’s location forms (business names and addresses, where a café with an emoji in its name is entirely plausible), and the quick-edit and AI-apply paths (where machine-generated text is written on a human’s approval). Each surface got the same pins — including the exact user-reported value from the incident, Hebrew and 💰 and punctuation intact, as a permanent named test.

The result is a structural guarantee rather than a hopeful one: any 4-byte character typed, imported, or AI-generated anywhere in the plugin is now certified to round-trip. If a future refactor breaks that property on any surface, continuous integration fails before the change ships.

The payoff: diagnosis at the speed of a test run

When the 💰 report arrived, the diagnostic session took minutes, and it is worth walking through because the shape of the session is reusable:

1. Reproduce the exact input. We took the reported string — Hebrew text, the emoji, the ?! punctuation — and added it verbatim to the certification corpus.
2. Run the pipeline. The full write-and-read cycle passed, byte-for-byte, on every surface.
3. Draw the only remaining conclusion. If the plugin provably preserves the value, the corruption exists upstream: either the source plugin’s stored value was already damaged (written through a non-utf8mb4 table at some point in the site’s history), or a third-party filter mutated it in flight.
4. Hand the operator a decidable check. One database lookup of the stored source value settles which. If the stored value already shows ?, the corruption happened at the original save, years before our plugin existed.

That is the real value of certification: it converts unresolvable data disputes into a decidable question with a command-line answer. Nobody argues about reputations; everybody looks at bytes.

Check your own site: a fifteen-minute audit

You do not need a custom plugin to apply this post. Four steps:

1. Inspect table charsets. Ask your host, or check the database directly, for any WordPress table whose charset is utf8 rather than utf8mb4. Mixed states are common: core tables converted, custom plugin tables left behind.
2. Run a live round-trip. Put an emoji in a draft post’s meta description (or title), save, reload, and look. If a ? appears, you have found a 3-byte table or a hostile filter — today, on your schedule, rather than during a campaign launch.
3. Plan the conversion carefully. Converting tables to utf8mb4 is routine but not free: index length limits change, and conversions should be rehearsed on a copy first. Any competent host or maintainer can do it; the point is to schedule it deliberately.
4. Re-test after every migration. Host moves and backup restores are exactly where old dumps reintroduce old charsets. A round-trip check belongs on your post-migration checklist permanently.

If you commission plugins or bespoke systems, add one question to your vendor conversations: “Show me the test that proves a 4-byte character survives your write path.” A vendor with an answer has thought about data integrity. A vendor without one will be learning on your data.

The broader principle

Byte fidelity is the quietest feature in Project Spark and one of the most valuable. It never appears in a screenshot. It has no settings page. But it underwrites everything else in this series: structured data is only as valid as its bytes (Part 11), Hebrew titles are only as correct as their invisible characters (Part 7), and AI-generated metadata is only as trustworthy as the pipeline that stores it (Part 5). Tests are not only there to catch your bugs. They are there to prove — quickly, neutrally, reproducibly — which bugs are not yours.

FAQ

Why do emoji turn into question marks in my database?

Your table uses MySQL’s utf8 charset, which stores at most 3 bytes per character. Emoji need 4 bytes, so MySQL substitutes ? on write. Convert tables to utf8mb4.

What is the difference between utf8 and utf8mb4 in MySQL?

MySQL’s utf8 is a nonstandard 3-byte-max encoding; utf8mb4 is real UTF-8 with full 4-byte support. All modern WordPress installs should use utf8mb4.

How do I prove my plugin isn’t corrupting data?

Byte-fidelity round-trip tests: write a hostile-character corpus (emoji, RTL text, backslashes, entities) through every write path and assert byte-identical reads in CI.

Can truncating text corrupt emoji?

Yes — byte-based truncation can split a 4-byte character in half, producing invalid UTF-8. Always truncate with multibyte-safe functions.

How do I check if my WordPress site supports emoji?

Add an emoji to a draft post’s meta description, save, and reload. If it survives, your tables are utf8mb4. If it becomes a question mark, an old utf8 table or a filter is destroying 4-byte characters.

Is it safe to convert a WordPress database to utf8mb4?

Yes, and it is routine — but rehearse on a copy first, because index length limits differ. Convert every table, including plugin-created ones, and re-run an emoji round-trip test afterwards.

Previous: Part 7: The Invisible-Character Bug in Hebrew SEO Titles
Next: Part 9: AEO in Practice: llms.txt, FAQ Schema, AI Crawlers