The File Abstraction

Investigate how operating systems transform raw, physical non-volatile storage sectors into persistent, named, and structured byte streams. Examine file structures, metadata attributes, POSIX operational primitives, and the kernel's three-tier file table architecture.

1. Motivation: Persistence & Device Virtualization

Process memory (the virtual address space covered in Week 07) is fundamentally ephemeral. When a process terminates, crashes, or when power drops, all state held in RAM volatile registers and dynamic page frames vanishes instantly. Furthermore, a single process's address space is strictly private; sharing vast datasets across distinct user accounts and lifetimes requires an external medium.

Physical non-volatile storage hardware (such as magnetic hard disks, NVMe SSDs, and flash arrays) presents an unforgiving interface:

• Storage is partitioned into fixed-size physical sectors or Logical Block Addresses (LBAs, typically 512 bytes or 4 KB).
• Devices accept only low-level commands: read block $k$, write block $k$, erase block $k$.
• Storage media lack human-readable names, ownership protections, concurrency arbitration, and dynamic resizing.

The File Abstraction is the fundamental operating system construct that bridges this gap. A file is a named, logical collection of persistent information recorded on secondary storage, presented to user processes as a continuous, linear address space of bytes ($0 \dots N-1$).

2. File Structure Models

Historically, operating systems have differed significantly in how much internal structure the kernel imposes upon file contents. Three canonical models define this evolution:

3. Access Methods: Sequential vs. Random Access

Processes access files through two primary paradigms:

• Sequential Access: Bytes are read or written in strict chronological order from beginning to end ($B_0, B_1, B_2, \dots$). The kernel tracks an implicit file offset pointer that automatically advances with every read or write. This mirrors magnetic tape media and remains the standard for audio streams, loggers, and video playback.
• Random / Direct Access: The application can read or write bytes at arbitrary offsets ($B_{4096}, B_{1024}, B_{0}$) in any sequence. Essential for relational databases, virtual machine disk images, and index structures. Implemented via system calls such as lseek(fd, offset, whence) or positional I/O (pread / pwrite).

4. File Attributes & Inode Metadata

In addition to raw data bytes, every file possesses associated metadata (attributes) describing its administrative and physical storage properties. In UNIX/POSIX environments, metadata is encapsulated within the struct stat record retrieved via the stat() system call:

struct stat {
    dev_t     st_dev;         // ID of device containing file
    ino_t     st_ino;         // File serial number (Inode Number)
    mode_t    st_mode;        // File mode (type and permissions: rwxr-xr-x)
    nlink_t   st_nlink;       // Number of hard links
    uid_t     st_uid;         // User ID of owner
    gid_t     st_gid;         // Group ID of owner
    off_t     st_size;        // Total size in bytes
    struct timespec st_atim;  // Time of last access (atime)
    struct timespec st_mtim;  // Time of last data modification (mtime)
    struct timespec st_ctim;  // Time of last status/metadata change (ctime)
};

5. The 3-Tier Kernel Architecture for Open Files

When an application executes int fd = open("log.txt", O_RDWR);, the operating system does not simply bind the integer file descriptor directly to a disk block. Instead, POSIX kernels coordinate three distinct layers of kernel tables:

1. Per-Process File Descriptor Table: Each process control block (PCB) contains an array of descriptors indexed by small integers ($0, 1, 2, \dots$). Standard descriptors: $0$ (stdin), $1$ (stdout), $2$ (stderr). Each valid entry contains a pointer to an entry in the System-Wide Open File Table.
2. System-Wide Open File Description Table: Shared across the entire operating system. Contains an entry for every active `open()` handle. Stores the current byte offset, access mode flags (read-only, write-only, append), and an active reference count.
3. VFS Inode Table (Active Inode Cache): Contains in-memory vnodes/inodes representing actual physical files on disk. Stores file size, device identifiers, disk block pointers, and lock state.

Why Three Tables? The fork() vs. open() Semantics

The separation between Per-Process Descriptors, Open File Descriptions, and Inodes is one of the most elegant architectural designs in UNIX:

• Sharing Offset across fork(): When a parent process calls fork(), the child inherits an exact duplicate of the parent's file descriptor table. Both descriptors point to the same Open File Description. If the parent reads 100 bytes, the offset advances to 100; when the child subsequently reads, it reads from byte 100! This enables shell pipelines and shared logging.
• Independent Offsets across separate open() calls: If two unrelated processes independently call open("data.bin"), they each receive their own independent Open File Description entry with its own offset pointer, pointing to the same underlying VFS Inode. Both processes can read the file at their own pace without interfering with each other's offsets.