Skip to main content

Bits, Bytes and Text

What this concept is​

A bit has two possible values. A byte contains eight bits and can represent 256 different patterns. The pattern does not tell you its meaning by itself: a program might interpret bytes as text, an image or an integer. An encoding specifies how text is represented as bytes.

Why it matters here​

Programming Basics used Python strings and saved JSON. Real files arrive as bytes. Understanding the conversion helps you explain garbled text, wrong size assumptions and changes that are invisible in an editor.

Concrete example​

Try this in a fresh file:

text = "café"
raw = text.encode("utf-8")
assert len(text) == 4
assert len(raw) == 5
assert raw.decode("utf-8") == text
print(raw.hex())

The displayed hex is 636166c3a9. Each pair represents one byte. The final letter needs two UTF-8 bytes. Python's string length counts code points, not necessarily the number of visible characters: combining marks and emoji can make those differ. Do not use this example to claim every letter always takes one or two bytes.

Common confusion / misconception​

Encoding does not encrypt. Renaming a file does not convert its contents. Ignoring decoding errors can silently delete evidence; for this lab, invalid UTF-8 is rejected rather than repaired automatically.

How to use it​

At an input boundary, identify whether you have bytes or text. Specify an encoding before decoding. Keep the raw bytes until validation succeeds. When diagnosing a problem, inspect a small hex representation and the declared encoding; do not print private file contents indiscriminately. A byte-size limit is different from a limit on text length.

Check yourself​

Is a four-character filename guaranteed to occupy four UTF-8 bytes? No. The exact text and encoding matter. Does the same text necessarily have one byte representation across all encodings? No.

Mini drill or application​

Predict the byte lengths of "A" and "é", then check. Decode b"\xff" as UTF-8 and record the exception. Explain why replacing errors would change the input contract.

Read this only if stuck​

Python Unicode HOWTO. Read encoding/decoding and Python strings selectively. The local example is original; no Unicode-table memorization is required.