Object Storage

Imagine you're at a massive fancy restaurant and you need to park your car.

If you use traditional parking, you drive into a specific lot, go to Level 3, Aisle B, and park in Spot 42. You have to remember the exact path to your car. This is like a traditional File System (think of folders and subfolders).

If you use valet parking, you hand over your keys, and the valet gives you a ticket with a unique ID (like "Ticket #8932"). You don't know or care exactly where the car is parked. When you want your car back, you just hand them the ticket.

That valet system is exactly how Object Storage works.

The Core Concept

Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.

Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.

algobase.dev

Every object in S3 (or any object store) is made of three things bundled together: the raw data bytes, rich key-value metadata, and a globally unique key. The data is anything — a JPEG, a 4K video, a Parquet file, a serialized ML model. The metadata is highly customizable: ContentType tells clients how to interpret the bytes; custom tags like UploadedBy or Environment are queryable and can drive lifecycle policies. The key is your "valet ticket" — a string path like photos/2024/01/15/avatar.jpg. There are no real directories; the slashes are part of the key string and the storage system treats it all as a flat namespace. You use the key to GET the object back later. Without the key, you can't retrieve the object (unless you scan, which is expensive).

1 / 10.5×1×2×3×Playground

Object anatomy — data bytes, metadata, and unique key bundled together

In Object Storage, data isn't organized in a hierarchy of directories and files. Instead, everything exists in a massive, flat namespace (often called a "bucket").

Each piece of data is an object. An object consists of three things:

1. The Data itself: The image, video, or text file.
2. Metadata: Highly customizable key-value pairs (e.g., ContentType: image/jpeg, UploadedBy: User123, Environment: Production).
3. A Globally Unique Identifier (Key): The "valet ticket" used to retrieve the object.

Why Object Storage Rules the Cloud

Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.

Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.

algobase.dev

Object storage scales because reads and writes to different keys are completely independent — there's no shared mutable state, no filesystem tree to lock. S3 handles millions of requests per second across billions of objects because it can shard by key hash. A web app writes user uploads via PUT and reads them back via GET, with zero server maintenance — no disks to provision, no capacity planning. A CDN pulls objects from S3 on the first cache miss and serves all subsequent requests from the edge, so even petabytes of media reach global users at millisecond latency. Analytics pipelines (Athena, Spark, ML training) run parallel GETs across thousands of objects simultaneously — the throughput scales with how many requests you throw at it. The trade-off: you cannot do a partial write inside an object. Change one byte in a 10 GB file and you must re-upload the entire 10 GB.

1 / 10.5×1×2×3×Playground

Object storage at scale — HTTP PUT/GET, CDN, parallel analytics

When you reach internet scale, traditional file systems break down. Navigating massive directory trees becomes slow, and scaling the file system across hundreds of servers is a nightmare.

Object storage is designed from the ground up for massive scalability, durability, and cheap storage.

Real-World Examples

• Amazon S3 (Simple Storage Service): The granddaddy of them all. If you've used the internet today, you've interacted with S3.
• Netflix: Netflix stores all of its master video files on Amazon S3. They use object storage because it can hold petabytes of data endlessly and handle massive throughput.
• Dropbox: When you upload a file, Dropbox breaks it into blocks, but it ultimately relies heavily on exabytes of object storage on the backend to keep that data safe.

How Do You Talk to It?

Unlike a file system where you use OS commands like open(), read(), or write(), you interact with object storage over HTTP using RESTful APIs.

• PUT /bucket-name/movie.mp4 (Upload)
• GET /bucket-name/movie.mp4 (Download)

Trade-offs: When Not to Use It

Object storage sounds perfect, but here's the catch: You can't easily modify a piece of an object. If you have a 10GB database file stored in S3 and you want to change one single byte, you usually have to upload the entire 10GB file again.

[!WARNING] Because of this limitation, Object Storage is terrible for databases or systems that require frequent, granular updates. For those, you want Block Storage (like AWS EBS).

Summary

• Object storage uses a flat structure where data is accessed via a unique ID (key) rather than a directory path.
• Objects bundle data with rich, customizable metadata.
• It's optimized for massive scale, high durability, and "write once, read many" workloads.
• Don't use it for databases or applications requiring frequent in-place updates.