Where a program's data lives

State: what your program remembers

Every useful application remembers things: the users who signed up, what is in each cart, how much money each account holds. That remembered information is called state.

While the program runs, its state lives in memory (RAM): variables, lists and dictionaries. Memory is fast, but it is volatile. When the process exits, crashes, or the machine loses power, everything in it is gone.

Try it. Save this as counter.py, run it a few times, then stop it and run it again:

visits = {"home": 0}

while True:
    input("Press Enter to visit the home page ")
    visits["home"] += 1
    print("Visits so far:", visits["home"])

Every restart begins at zero. Nothing the program learned survived.

Writing it down

The obvious fix is to write the state to disk: to a file that is still there after a restart. Load the file when the program starts, save it after each change, and the count survives.

That works, and for one program on one machine making one change at a time it is often enough. Trouble starts with questions a toy program never asks:

• What if two copies of the program save at the same moment?
• What if the program crashes halfway through saving?
• How do you find one record quickly when the file holds millions?
• What stops someone saving an age of "banana" or two users with the same email?

Kinds of data

Not all data has the same shape, and the shape affects where it is easiest to keep.

Kind	What it looks like	Example
Structured	Every record has the same fields	A table of users with id, name, email
Semi-structured	Records have fields, but they vary	JSON events where each type carries different properties
Unstructured	No fields at all, just content	Images, PDFs, audio, free text

Most application state is structured or semi-structured, and that is what this whole field is about. Unstructured content is often kept as files in object storage, with a database holding the facts about each file.

What every application does with data

Whatever the data, applications do four things with it: create new records, read them, update them and delete them. These are often shortened to CRUD. A storage system is only as good as how safely and quickly it does those four things when many requests do them at once, and when a crash interrupts them halfway.

Check yourself

1. Why is memory alone not enough to hold an application’s state?

   Show answer

   Memory is volatile: its contents disappear when the process exits, crashes or the machine loses power. State that must outlive the process has to be written to durable storage.

   I could explain this without looking

2. In the two-writer experiment, why did a program crash instead of just losing an update?

   Show answer

   Opening a file for writing truncates it to zero bytes before the new contents are written. The other program read the file in that window, found empty or partial JSON, and failed to parse it.

   I could explain this without looking

3. Give an example of structured, semi-structured and unstructured data from an app you use.

   Show answer

   For example: a bank statement’s transactions (structured), app analytics events with varying properties (semi-structured), and profile photos (unstructured).

   I could explain this without looking