Remove Invisible Characters

Characters you cannot see

Copied text sometimes contains a zero-width space, a byte order mark, or a bidirectional mark. The line looks normal and still fails a comparison. This page removes those format characters. It does not delete ordinary spaces, tabs or line breaks. Spaces are remove space. Extra gaps are collapse whitespace.

How to use it

1. Paste the text. The first line of the result says how many characters were removed. The rest is the cleaned text.
2. Show them as [U+…] replaces each removed character with its code, such as [U+200B], instead of deleting it.
3. All format characters widens the set to every Unicode format character. Leave it off to touch only the common invisible set.
4. Show example loads hello, a zero-width space, world, and a byte order mark. The default result is a count of 2 and helloworld.

Which code points move

The default set is the soft hyphen, combining grapheme joiner, Arabic letter mark, Mongolian vowel separator, zero-width space, zero-width non-joiner, zero-width joiner, left-to-right and right-to-left marks, bidirectional embeddings and overrides, word joiner, invisible operators, isolates, the byte order mark, and interlinear annotation characters. All format characters also includes other format code points, including variation selectors, so an emoji that relied on one can change shape. Letters, visible spaces, newlines and ordinary punctuation stay. The count is the number of matches, including when you only mark them.

Where people use it

• A pasted password or code that looks right and still does not match.
• A paragraph that starts with a byte order mark from a file saved as UTF-8 with BOM.
• Finding where a zero-width space sits by switching on the U+ marks.

Questions