ZYSNOR, smart solutions, real impact

Writing guide

How to remove invisible characters

By , Founder of ZYSNOR · Updated · Checked · 7 min read

In short

Paste the text into an invisible character remover: it deletes zero-width spaces (U+200B), byte order marks (U+FEFF), soft hyphens and direction marks, and turns no-break and typographic spaces into normal spaces. It keeps the zero-width joiner and non-joiner, because Persian, Hindi and emoji need them. A report lists each character it found and how many times.

Jump to a section

A word that search cannot find, a number a spreadsheet will not add up, a line of code that fails though it looks fine: hidden characters copied from web pages, PDFs and chat apps are a common cause.

The usual suspects

Zero-width space (U+200B) is a line-break hint with no width. The byte order mark (U+FEFF) often sits at the start of a file. A soft hyphen (U+00AD) only shows when a word breaks. The no-break space (U+00A0) looks like a space but is a different character, so "100 kg" with one does not match "100 kg" with a normal space.

A table of hidden characters found in one line and what the tool did with each
Hidden characters in one pasted line, measured.

What to keep

The zero-width non-joiner (U+200C) stops two letters from joining and is part of correct Persian spelling. The zero-width joiner (U+200D) glues emoji such as the family emoji into one picture. Deleting them breaks text, so the tool keeps both and tells you how many it found.

Symptoms and causes

A table matching symptoms such as failed searches or spreadsheet errors to likely hidden characters
Why hidden characters cause trouble.

Worked example 1: a search that fails

A name copied from a web page will not match in your database. The report shows "U+200B (zero width space) x1": remove it and the match works.

Worked example 2: a spreadsheet number

"1 250" from a PDF will not sum. It contains U+202F, a narrow no-break space. The tool turns it into a normal space; then remove the space in the sheet or use Find and Replace.

Worked example 3: Persian text

Text such as کتاب‌ها uses U+200C between the parts. It is kept, so the word still displays correctly.

What the ZYSNOR tool does

  • Removes zero-width space, BOM, soft hyphen, direction marks and tag characters.
  • Turns no-break and typographic spaces into normal spaces.
  • Keeps ZWNJ and ZWJ; optional straight quotes.
  • Reports every character by code point and count. Runs in your browser.

Practice problems

Problem 1. Should a remover delete U+200D in 👨‍👩‍👧?

Show the worked answer

No. It joins the three people into one family emoji.

Problem 2. "Hello​World" contains U+200B. What do you get?

Show the worked answer

HelloWorld: a zero-width space has no width, so nothing replaces it.

Ready to try it? The free tool works in your browser, and the button in the page header takes you straight to it.

Common questions

What are invisible characters?

Characters with no visible width or that look like a normal space, such as the zero-width space U+200B, the BOM U+FEFF and the no-break space U+00A0.

Why does my copied text break searches or code?

A hidden character sits inside the word or line, so it no longer matches what you type.

Does the tool delete characters Persian or emoji need?

No. It keeps the zero-width non-joiner and joiner.

What happens to no-break spaces?

They become normal spaces.

Can it straighten curly quotes?

Yes, if you tick the option.

Is my text uploaded?

No. It is cleaned in your browser.

Sources and how we researched this

Sources (3)

How we researched this

Measured on the ZYSNOR tool in Chromium on one line containing U+200B, U+FEFF, U+00AD, two U+00A0, U+202F, curly quotes, Persian text with U+200C and a family emoji with two U+200D: the report listed 6 removed or replaced, kept 1 non-joiner and 2 joiners, and straightened the quotes on request. Character roles (zero width space as a line-break hint, ZWNJ preventing joining in Persian, ZWJ building emoji sequences) are from the Eclectic Light Company's article on invisible Unicode and the Wikipedia articles on the zero-width non-joiner and joiner, read from search excerpts. Spotted something that does not match your experience? Tell us through the Contact page and we will check it and update this guide.

Related guides

Rules differ between schools and change over time. Always check your own school or university handbook.