Description
Objectives: Learning FAT32 file system in detail. Understanding on-disk file system
structures. Practicing formatting and mounting a file system, and parsing on-disk
file system structures.
In this project you will explore the FAT32 file system and write a program that
will read and and parse a FAT32 volume directly in raw mode. The project will be
done in Linux and using C programming language. You can work in groups of 2
students each.
First, read and learn about FAT32 file system and its on-disk structures: its
volume information structure, directory record structure, FAT structure, etc. You can
find a lot of documentation about FAT32 in Internet. We recommend starting with
[1]. Note that FAT32 terminology is using the term “cluster” instead of the term
“block”. A typical cluster size is 8 sectors, that means 4 KB. In this case, a cluster
will occupy 8 consecutive sectors. A sector is 512 bytes long. A disk that has a file
system on it is called a volume.
In this project you will work with a FAT32 volume that will not sit on a separate
physical media but on a regular Linux file. Therefore, you first need to create a
regular Linux binary file that will act as your disk, i.e. a virtual disk, with Linux dd
command. The dd command will initialize the file to all zeros. Block size will be
4KB. You can create such a file with as many blocks as you wish, of course there is a
limit, but in this example we will create a file that has 1024 blocks. Hence the size of
the file will be 4KB x 1024 = 4 MB. The name of the virtual disk will be “vdisk”.
dd if=/dev/zero of=vdisk bs=4k count=1024
Linux file vdisk will act as a disk: a virtual or pseudo disk. We can format it with
FAT32 file system and mount it to an empty directory in our local Linux file system.
Now use the losetup command to associate a loop device file (/dev/loop0) with
the virtual disk file. In this way you make the virtual disk file look like a block
storage device instead of just a regular binary file within the file system [5].
losetup /dev/loop0 vdisk
Each device in a Linux computer has a device file associated with it. A device file
starts with the /dev prefix. After running losetup command as above, the virtual disk
file is now threated as a block oriented storage device that can be mounted with the
mount command. To delete the association between the loop device file and the
virtual disk file, we can use the –d option of losetup as: loosetup –d vdisk.
You can now create a FAT32 file system in your vdisk file. This is called (logical)
formatting the disk. You can use the mkfs.fat command of Linux for this purpose,
which may take several options. Run the command with options given in the example
below. We will always use the same options in our tests. You are suggested to read
the man page of mkfs.fat or mkdosfs commands.
mkfs.fat -a -v -S 512 -s 8 -F 32 -n CS342 vdisk
Above, the –F option says that the file system will be FAT32, not FAT16 or
FAT12, -s 8 options says that the block size will be 8 sectors, that means 4KB, -S
512 says that a logical sector is 512 bytes long, and -n option gives a name for your
2
volume. Hence after creating the file system, you now have a FAT32 volume which
you can mount and use; or you can write a program to analyze the volume directly
without mounting.
Create a mount point, that means an empty directory with the mkdir command.
The name of the empty directory can be, for example, “myfat32fs”. Then mount
your volume on this mount point. Read the man page of the mount command.
sudo mount -t msdos /dev/loop0 myfat32fs
You can now change into myfat32fs directory. That means you are changing into
the root directory of the FAT32 volume you mounted. There you can create and use
files with any tools that you like: touch command, dd command, cp command, an
editor, or something else. You can also create subdirectories.
Now, unmount your volume.
sudo umount vdisk
You can now analyze the virtual disk file. You will write a program, called fatfs.c,
that will read and parse it directly one byte at time, or 512 bytes at a time (one sector
at a time), or 4096 bytes at a time (one block at a time). The executable will be called
fatfs.
As said, your fatfs program will read the virtual disk file block by block, or sector
by sector and will parse the file system structures directly. That means you will be
accessing your virtual disk in raw mode in your program. In our test cases, we will
have just have one directory, i.e., a root directory, in a FAT32 virtual disk. There
will be no subdirectories and the number of files in the root directory will not exceed
50. Hence the root directory will be fitting into a single block. You can assume that
only short filenames will be used. A short filename has at most 8 characters for the
filename and 3 characters for its extension. “project.pdf”, for example, is a short
filename. We will not use long filenames.
The program will be invoked in one of the following ways, doing the specified
things:
• fatfs
(VolumeID sector) of the FAT32 file system. Printed information will include
some numbers, like the number of sectors that FAT occupies.
• fatfs
directory record you will print the filename, its extension, and also other
information found in the record, as nicely as possible. You will use one output
line per record.
• fatds
i.e., blocks allocated to file
printed. One number per line will be printed.
Sample invocations of your program can be as follows:
• fat32 vdisk –p volumeinfo : Print info from the volumeID sector.
• fat32 vdisk –p rootdir : Print info about root directory content.
• fat32 vdisk –p blocks x.bin : Print numbers of the blocks of the file with name
“x.bin”.
You will be provided an example virtual disk, called vdisk, in the webpage of the
course. It is a regular Linux binary file. It acts as a FAT32 volume of size 4MB. That
is like a 4MB disk formatted with FAT32. In that we have some files created and
sitting. As an exercise you can start with parsing this file. You can see the content of
the vdisk file in hex using the command xxd.
xxd vdisk
3
Submission:
Submit through Moodle. You will do a demo and you will be asked very detailed
questions.
Clarifications:
— In boot (volumeID) sector, Starting at offset 36 (in decimal), you need to read 4
bytes to obtain the Sectors per Fat information.
— Note that integers (4 bytes or 2 bytes) stored in vdisk are in little-endian order;
that means the byte at offset 36 is the least significant byte (LSB) of a 4-byte integer
value, the byte at offset 39 is the most significant byte of the 4-byte (32 bit) integer
value (do not worry about the bit order in a byte).
— You can use memcpy to read 4 bytes (size of integer) from an array into an integer
variable. For example:
unsigned int x;
unsigned char sector[512];
memcpy ( (void *) &x, (void *) (sector + 36), 4);
Since Intel machines also use little-endian byte order, memcpy above will obtain an
integer value into variable x from the 4 bytes of the sector starting at offset 36.
— You can also use a structure mapping the content of the boot sector in to the fields
of the structure properly. Such a structure is defined in msdos_fs.h. It is struct
fat_boot_sector. That means:
unsigned char sector[512]; // assume sector stores boot sector data
struct fat_boot_sector *bsp;
bsp = (struct fat_boot_sector *) sector;
Then access the fields if the structure like below:
bsp->afieldname
Learn the field names from the msdos_fs.h file.
— Try to use unsigned integers and characters for various numbers stored in the boot
sector.
— You don’t have the use the msdos_fs.h file in your program. If you want, you can
use it, but you don’t have to. You can define your own macros and structures instead
of the ones available in msdos_fs.h header file.
— A project skeleton is availabe at github: https://github.com/korpeoglu/cs342-
spring2017-p5.git
— FAT32 uses litle-endian format in storing integers in on-disk structures like FAT
table. That means the least significant byte is stored first. If your machine is bigendian, you need to be extra careful. Intel x86 architecture is little-endian.
— FAT32 normally installed on disks that have 65525 clusters at least. If block size is
4KB, that means the disk should be 256 MB at least. Hence we may get a warning in
4
FAT32 installations on smaller volumes like a volume having 1024 clusters. This
should not be a problem in this project.
— The number of reserved sectors is 32 by default. Hence in the given vdisk file, the
number of reserved sectors is 32. Then starts FAT. That means after the first 32
sectors of the virtual disk, the FAT starts. Note that we have 2 FATs stored. After
FATs we have clusters (blocks) storing the data of files and directories. For example,
the first block (cluster) after the second FAT is storing the root directory records.
Record size is 32 bytes. Hence each sector can store 16 directory records, i.e.,
directory entries.
— The first sector is the boot sector and volumeID sector in the disk that we provide.
This is because the disk we provide is not a bootable disk. Hence the first sector
(sector 0) also containes volumeID information. Hence you only need to parse sector
0 to obtain important file system information, like for example, the number of sectors
that FAT is occupying.
References:
[1]. Understanding FAT32 File Systems.
http://www.pjrc.com/tech/8051/ide/fat32.html.
[2.] FAT32 File Systems Specification.
https://msdn.microsoft.com/en-us/library/gg463080.aspx
[3]. Design of FAT Filesystems.
https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system
[4]. FAT file systems.
http://wiki.osdev.org/FAT
[5]. Anatomy of the Linux File system.
http://www.ibm.com/developerworks/linux/library/l-linux-filesystem/