Python bytes.fromhex() Deep Dive: Whitespace, Errors, Streaming

bytes.fromhex() is the standard Python 3 way to turn a hex digit string into a bytes object. This article focuses on behavior and edge cases — not a tour of every alternative. For choosing among APIs, start with 4 ways to convert hex to string in Python.

Basic usage

print(bytes.fromhex("48656c6c6f"))           # b'Hello'
print(bytes.fromhex("48 65 6c 6c 6f"))        # whitespace OK
print(bytes.fromhex("48656c6c6f").decode()) # 'Hello'

Whitespace (spaces, tabs, newlines) between hex digits is ignored. That makes paste-from-dump workflows easy.

What it does not accept

bytes.fromhex("48656c6c6")   # ValueError: odd-length string
bytes.fromhex("48GG")        # ValueError: non-hexadecimal number found
bytes.fromhex("0x4865")      # ValueError — leading 0x is not stripped

• Length must be even after whitespace is removed.
• Only 0-9A-Fa-f digits are allowed.
• Prefixes like 0x, separators like : or , must be removed yourself.

import re

def from_hex_loose(s: str) -> bytes:
    cleaned = re.sub(r"[^0-9A-Fa-f]", "", s)
    if len(cleaned) % 2:
        raise ValueError("odd number of hex digits after cleanup")
    return bytes.fromhex(cleaned)

print(from_hex_loose("0x48:65:6c:6c:6f"))  # b'Hello'

Unicode text vs raw binary

fromhex always returns bytes. Text is a separate step:

# UTF-8 Chinese "你好"
raw = bytes.fromhex("e4bda0e5a5bd")
print(raw.decode("utf-8"))

# Wrong encoding → UnicodeDecodeError or mojibake
# raw.decode("ascii")  # fails

If the hex represents a file header or ciphertext, keep it as bytes; do not force a text decode.

Performance and large inputs

fromhex is implemented in C and is appropriate for multi-megabyte hex strings in memory. For huge files, stream in even-sized chunks (keep a one-digit carry if a read splits a pair) instead of loading everything as one Python string.

def iter_hex_file_bytes(path: str, chunk_size: int = 8192):
    carry = ""
    with open(path, "r", encoding="ascii", errors="ignore") as fh:
        while True:
            block = fh.read(chunk_size)
            if not block and not carry:
                break
            data = carry + "".join(ch for ch in block if ch in "0123456789abcdefABCDEF")
            even = len(data) - (len(data) % 2)
            if even:
                yield bytes.fromhex(data[:even])
            carry = data[even:]
    if carry:
        raise ValueError("incomplete trailing hex digit")

Compared with binascii.unhexlify

Same output for clean even-length hex. Pick one style and stay consistent. More on unhexlify.

Small checklist

• Normalize separators before calling fromhex if input is messy.
• Never “fix” odd length by guessing a leading zero unless the format says so.
• Decode to str only when the payload is actually text.

Online helper: Hex to String converter. Broader context: hexadecimal encoding overview.