A long time ago now, I wrote one of my many test kernels in order to experiment with an idea I had about how to reference file systems and devices on the system. Now I'm looking back through those ideas to refine, enhance, and document them so that they can be incorporated into the latest Octopus kernel.
So, what does it need to do?
I have this idea (possibly a ill-conceived one) that my OS will have two structures for handling devices. One will be a connection-based tree of devices, the other will be the flat list of sanitised devices which will be useful to the user on a day-to-day basis.
The Amiga had a similar concept where drives would be referenced by DH0:, DF0: etc. and devices could be referenced directly by opening the device using the driver such as "serial.device".
File Systems have a structure of pointers and a structure of "data" known as a context.
Block Devices have the same, as do Character Devices.
Ok, so we'll go through and describe what each of these things are and how they are related from a data point of view, then we'll work some examples of different DosDevices, finally we'll go through and discuss how various operations are performed.
DosDevice Structures
Device
This is the device driver to actually interact with whatever device is storing the data. In the code, this is a structure of function pointers for such operations as ReadBlock(). The device can either be a Block device, such as a hard drive or CD-ROM, or it can be a characters device, such as a keyboard or serial port.
A few examples of block devices are a RAM disk, hard drive, floppy drive, CD-ROM
A block device will support functions for:
- reading or writing blocks of data,
- getting the size of each block on the device, and
- getting the number of blocks on the device.
A character device will support functions for:
- opening or closing the device,
- reading or writing data, and
- fetching or setting any metadata, such as port settings.
Device Context
It is likely that one device driver will be used to communicate with more than one device, for example, you may have a primary master hard drive as well as a primary slave. Beyond that, you may have multiple partitions on each drive. In order to deal with this, we have the Device Context object.
This object is created by the device driver itself, and is only ever understood by that device driver. I imagine two ways of creating a device context, either when the device driver performs a scan for supported devices, or when the device driver is given specific information on the location and other parameters about a connected device that it supports. Most hard drives will probably use the first method, and most standard devices such as the keyboard will likely use the second. It's also possible that an external program may locate a device and pass information to a device driver to create a context.
At minimum, the context object will contain information to uniquely identify the instance of the device being referenced such as IO ports, PCI device location, memory mapped IO locations, unit numbers etc. It may well also contain caches such as a read-write buffer, device parameters, or other information to improve performance of the device.
Some device drivers may never use a context object if it is likely that there will only ever be one instance of the device present in most machines and that it will be at a predictable address. Example of this are the pc speaker or the parallel port.
File System
The file system driver is similar to the device driver in that it is a structure of function pointers that can be invoked by the operating system in order to interpret the data on a particular block device.
Examples of a few file systems are FAT, ext2, Amiga OFS, Amigs FFS, ISO9660 (CD-ROM), etc.
Each file system will support functions for:
- opening and closing files,
- reading and writing data to an open file,
- deleting and renaming files,
- reading the contents of a directory,
- reading or writing the metadata of a particular file such as the creation date or permissions, and
- formatting the drive.
File System Context
Similar to the device driver context above, the file system context allows the device to store state information for each instance of the file system in use on the system. When the Dos layer invokes a method on a particular file system, it will need to pass the context for that file system, as well as enough information for the file system to interact with the device which is hosting the file system (so at a minimum, the device driver structure and context).
As far as each file system is concerned, it is responsible for all the data on the underlying device from block 0 to the device's maximum block number. If any partitioning mechanism is being used, this will be dealt with by the underlying device. Because of this, the context object won't normally need to contain any information to identify the file system as the device context will do that. (I guess you could have two file systems within the same partition, one after the other, in which case the second one would need to store a block offset, but there is little benefit in doing this.)
The file system context will likely contain caches of data to improve performance. These would include a cache of the file system parameters such as the volume label and free space, the linked list of file blocks such as the file allocation table, and possibly a list of free blocks in file system to speed up allocation when data is written.
(Note: currently there are no plans for the operating system to "shut down", so it is recommended that any changes to this cached data be written to the underlying storage as soon as convenient.)
DosDevice
The DosDevice is what brings all of the above information together. A DosDevice is a reference to a mounted filesystem, a device, or both.
Each one has a unique textual identifier which is used to identify it. In the Windows world this might be C:, in Unix it might be /var, in the Amiga world this might be DH0:.
The DosDevice will have at least a file system or a device driver. If a file system is present, it takes precedence for any file IO operations that are requested. If not, these requests fall back to the device.
The DosDevice will have to provide a mechanism for changing the file system on a particular device so that an existing or unformatted drive can have the new file system set and then formatted.
File Context
When the file system is asked to open a file, it creates a file context object. Similar to the file system context, this object stores information which identifies the file in the file system and caches a number of details about the it. The file context will likely contain a reference to the file's directory entry or file header to identify it, and may well cache the linked list of blocks which make up the file.
File Handle
When a process requests a file be opened, the kernel allocates a file handle and opens the file. The file handle is stored in the processes "open file" table and a reference to this file handle (usually a number) is returned to the process. This file handle stores a reference to the DosDevice in question, and to the file context which was created when the file was opened. With these two pieces of information, a file handle can be read, written or closed as required.
Examples of DosDevices
Here are a few typical usage scenarios for an everyday machine along with how the Dos system will handle them. The drive names are entirely arbitrary, but exemplary of what might be used.
HD0:
This will be a reference to the first partition on the first fixed hard drive. The DosDevice structure will contain a device driver for an ATA hard drive, a device driver context object which indicates partition 0 on drive 0, a file system for whatever kind of file system was found on the drive, and a file system context object to cache parameters.
HD1:
This will be identical to HD0: except that it will refer to the second partition on the drive. It will use the same device driver as before, but the device driver context will have parameters which indicate that the start block and length of this instance of the device relate to the second partition. The file system and file system context will depend on the contents of the partition.
CD0:
This will relate to the first ATAPI CD-ROM drive. It will use an ATAPI device driver with a device driver context which indicates the location of the CD-ROM drive (primary slave, secondary master, etc.). The file system will be an ISO9660 file system with a context which caches some volume parameters. The ISO9660 may not support writing, in which case it does not provide a function for writing to files.
IMG1:
This examples refers to a disk image that exists on another file system. The device driver is a simple driver which opens the file and reads and writes blocks into the file. The device context contains the file descriptor of the image and the size of each block on the emulated drive, and would likely maintain a cache of the total number of blocks available on the emulated device. The file system is dynamically initialised based on the content on the disk image.
NFS1:
This examples relates to a file system provided by a network resource such as an NFS or FTP server. In this instance, communication across the network is done at the logical file level so the DosDevice will have no device or device driver context for this. The file system will be responsible for communicating across the network and its context will need to contain all the required information for this, including possibly the IP address of the server, any credentials or encryption keys required, any session parameters, etc.
KBD:
This example refers to the keyboard of the system. This DosDevice will not have a file system, but will be a readable and writeable character device. The device context object will contain a buffer for incoming scan codes. Reading characters from the device will provide scan codes (and/or translated ASCII codes) for the keys being pressed, writing to the device will allow keyboard control codes to be used.
TODO: Talk more about characters devices.
TODO: How does the keyboard interrupt handler find the context object to use for its buffer?
PAR:
This refers to the parallel port of the system, bytes written to or read from this port will be communicated to the parallel port in their raw format. The device driver will be a character device and there will not be a file system. To use this device, the device will need to be opened first, specifying any control and configuration parameters are required, such as the handshaking method or port mode. Once open, data can be written and read from the port as normal.
TODO: How will port state information be fetched or set during operation?
PRT:
This refers to the parallel port as above, but in this instance we imagine that a printer driver has been loaded which will translate information between this device and the actual parallel port.
For example, an application may open this device and send it a postscript file. The device driver buffers the postscript data, interprets it as required for the printer, and sends interpreted data to the PAR: port to go to the printer.
Walk-through of typical operations
Manually Creating a Drive
In order to create a new drive, such a RAM disk, a text file containing parameters for the device driver and file system needs to be created, including references to the device and file system by name. This parameter file can then be "mounted" as a new DosDevice.
As an example:
RAM:
Device = RamBDDI
BlockCount = 32
BlockSize = 512
FileSystem = FatFSDI
SectorsPerCluster = 1
Here we see a parameter file which would create a RAM disk with 32x 512 byte sectors (an entire 16k :O ) which is given a FAT file system initially with 1 sector per cluster. Any other values which are required will be defaulted by the relevant driver. Note that the drive will be unformatted to begin with, although file system parameters are given.
When a process requests to open a file:
- The Dos system will look up the corresponding DosDevice entry by the name given if the file name.
- If the DosDevice has a file system:
- it will call the file system open function, passing the DosDevice entry. This ensures that the file system has access to the file system context, the device driver, and the device driver context.
- The file system will look up the file with the given path. If it doesn't exist, it returns an error.
- The file system creates a File Context object containing a reference to the file and any cache data, and returns a pointer to it.
- If the DosDevice doesn't have a file system.
- The Dos layer checks to see if the device driver has an open function, and calls it.
- The device driver interprets the given file path and name as parameters, and opens the device.
- The device driver creates a file context, and returns a pointer to it.
- The Dos layer creates a file handle structure containing a pointers to the DosDevice and one to the file context, and passes this to the process file handle table.
- The file descriptor from the table then gets returned as the return of the function.
Other Considerations
Paging.
If the system uses paging, all the required DosDevice, device driver, device context, file system and file system context for the page file device will have to remain in-memory at all times, it cannot be paged out to disk. In the *nix world, this is simplified because an entire partition is used for swap (paging) space, which removes the need to persist any file system in memory.
Switching device drivers.
Given that the only reference to each device driver is in the DosDevice table, it would in theory be possible to restart or even replace device drivers at run-time, without needing to close any open files. The problem with replacing the device driver with a different one is with ensuring that the data in the device context is
File Locks
File locks will be implemented in the Dos layer as a table which stores a reference to the Dos device, the file context, and an integer which uniquely identifies the file in the context of the file system (the file id).
When a request is made of the Dos layer to open a file, it will call the file system's LookupFile() function, passing it the path to the file. If the file is found it returns the file id, otherwise it returns 0. Exactly what the file id is can vary between each file system, but it must uniquely identify each physical file. Something such as the first block number of the file, or the block number of the file header block would be suitable.
The Dos layer will take this value and check with the file lock table to see if a lock exists on the file which inhibits the current open request, in which case the open fails.
If no inhibiting lock exists, the file system open function is called, passing the file id (as this saves the effort of looking up the file again). If this succeeds, a new lock is written to the table, and success returned.
No comments:
Post a Comment