With the ATA save (mostly) working now and LD not playing ball with making sensible .o COFF files for me to load, I'm going to take a break from that side and advance on a different front. I'm wanting to get some form of sound capability working (beyond the PC speaker) so I'm going to get back to the pci system.
A quick compare through the various versions that I have reveals that I haven't actually worked on this front since implementing the consolidated datatypes, so perhaps it is time to look at this anyway. I also have my trusty PCI reference manual.
I haven't figured out how to provide a generic interface for the PCI system into the rest of the OS in the ideal world system, I imagine it will be some form of Bus Device with a context, but that's not important right now.
2010-05-14
2010-05-13
2010-05-13 - Debugging the Filesystem
Continuing the investigation of the corrupting filesystem. It seems the first hurdle being failed is identifying the sector number of the start of the root directory, this is being read out as 6336 on a volume that only has 15Mb of space. Upon further investigation, the driver believes that there ate 197 FATs on the disk and 32 sectors of reserved space. Looking at the on-disk structure by hand reveals the expected 2 FATs and only 1 sector of reserved data. I believe the problem is that something is writing outside it's allocated memory and trashing the FAT context object. This could be tricky to track down.
I'm going to reimplement the ShowContext function through the DOS layer. This will allow me to examine the context at each stage of the process and see at what point the values are being corrupted.
ShowContext() function reimplemented so that it's accessible from the front end via the dos layer ("ShowContext HD1:" ... Win!). The values in it are definitely wrong for both HD0: and HD1: (the two hard drive partitions). So, either it's not allocating enough space then being overwritten, or it's being overwritten by some misbehaving code, which can only be somewhere in the MOUNT command.
A modification to the ShowMem command so that it shows the exact allocated block at that location (if one exists) and a lot more poking later and it seems the problem is somewhere within the list code. Adding the third entry to the DosDevice list seems to cause some memory corruption. I'll tweak the mountlist first to see if I can confirm this, then look into why.
AH HA! In the FAT context creation code, it allocates space for the FAT context and the boot sector (kept around for posterity), and it uses the BDD to determine the amount of space it needs to allocate for the block. However, the hard drive is returning a block size of zero because that seems to be what is coming from the drive. A mistake in the FAT code doesn't correctly account for this, causing it to allocate 0 bytes, and then write into it, which the next operation promptly overwrites. Let's fix this.
\o/ It works properly now. There are a few oddities, such as it only seems to use every other cluster for subsequent saves, but it seems to be behaving.
Identified a minor problem in the Dos subsystem where searching for "HD:" would find the first device starting with HD instead of specifically "HD:", but this was easily fixed, and a small modification to provide a global variable called CurrentDirectory which is used if no device name is identified in the given filename.
Moving on to look again at the modules now, it would be nice to be able to create a program that can be loaded, executed, then flushed, even if only for a demo. This will also be useful as having the kernel itself depend on libraries will cause no end of trouble, so I can make the loaded modules (executables) do that instead. So how about HelloWorld.exe?
Ok, so LD isn't playing ball. Firstly it linked the two .o files into an MZ executable, then it linked the two files into a COFF file, but didn't actually tie up the references or combine the sections to produce a single COFF file.
I'm going to reimplement the ShowContext function through the DOS layer. This will allow me to examine the context at each stage of the process and see at what point the values are being corrupted.
ShowContext() function reimplemented so that it's accessible from the front end via the dos layer ("ShowContext HD1:" ... Win!). The values in it are definitely wrong for both HD0: and HD1: (the two hard drive partitions). So, either it's not allocating enough space then being overwritten, or it's being overwritten by some misbehaving code, which can only be somewhere in the MOUNT command.
A modification to the ShowMem command so that it shows the exact allocated block at that location (if one exists) and a lot more poking later and it seems the problem is somewhere within the list code. Adding the third entry to the DosDevice list seems to cause some memory corruption. I'll tweak the mountlist first to see if I can confirm this, then look into why.
AH HA! In the FAT context creation code, it allocates space for the FAT context and the boot sector (kept around for posterity), and it uses the BDD to determine the amount of space it needs to allocate for the block. However, the hard drive is returning a block size of zero because that seems to be what is coming from the drive. A mistake in the FAT code doesn't correctly account for this, causing it to allocate 0 bytes, and then write into it, which the next operation promptly overwrites. Let's fix this.
\o/ It works properly now. There are a few oddities, such as it only seems to use every other cluster for subsequent saves, but it seems to be behaving.
Identified a minor problem in the Dos subsystem where searching for "HD:" would find the first device starting with HD instead of specifically "HD:", but this was easily fixed, and a small modification to provide a global variable called CurrentDirectory which is used if no device name is identified in the given filename.
Moving on to look again at the modules now, it would be nice to be able to create a program that can be loaded, executed, then flushed, even if only for a demo. This will also be useful as having the kernel itself depend on libraries will cause no end of trouble, so I can make the loaded modules (executables) do that instead. So how about HelloWorld.exe?
Ok, so LD isn't playing ball. Firstly it linked the two .o files into an MZ executable, then it linked the two files into a COFF file, but didn't actually tie up the references or combine the sections to produce a single COFF file.
2010-05-12
2010-05-12 - Loading and Saving with the Filesystem
I have the ATA driver now implementing the BDD Interface, and I've also set up the mountlist to use the device. I had actually forgotten that the mountlist system actually looks up the device symbols in the kernel library, and that I needed to add the AtaBDDI into the kernel library definition in order for it to work properly.
Little bit of debugging needed on the Ata Driver. It seems it's not picking up the partitions correctly yet. A minor mistake in the mountlist parser meant it wasn't getting the Partition number value correctly.
A few tweaks later and it seems to be creating the contexts correctly, as well as departitioning the drive ... om nom nom. The type command (otherwise known as the fatload command).
In theory, I should now be able to combine the load and save commands to make a copy command. Should be very useful in getting the library files onto the hard drive and whatnot. To the batcave ...
Ok, so attempt one wasn't completely successful. Perhaps when I was rewriting the FAT driver to use the generic directory search methods I may have missed something. The file is loading, but is spinning off into some infinite loop when trying to save the file. I suspect the FindFirst() call.
Ok, so the flag to allow creation of new directory entries (which will need to extend a cluster-based directory if needed) was not fully implemented and was not returning an empty entry in the root directory to be used for the new file. Let's try this again.
The save was successful, but the new file wasn't found. although I have a sneaking suspicion as to why, I didn't actually write the filename and whatnot to the directory entry before I saved it ... Oops.
Another significant change to the FAT filesystem driver, and one which should solve a bunch of potential issues. I have created a routine which processes the Path and splits it into an array of strings (one for each directory) in the FAT 8.3 format (with correct spacing to match the filenames). This should greatly ease the troubles of dealing with these names and comparing them for matches. It will also handle generation of a save filename and fix an issue with ensuring the file does not already exist.
Everything in place, however, something is wrong now as when using the ATA driver, the filesystem is being corrupted in some way. Needs more investigation.
Little bit of debugging needed on the Ata Driver. It seems it's not picking up the partitions correctly yet. A minor mistake in the mountlist parser meant it wasn't getting the Partition number value correctly.
A few tweaks later and it seems to be creating the contexts correctly, as well as departitioning the drive ... om nom nom. The type command (otherwise known as the fatload command).
In theory, I should now be able to combine the load and save commands to make a copy command. Should be very useful in getting the library files onto the hard drive and whatnot. To the batcave ...
Ok, so attempt one wasn't completely successful. Perhaps when I was rewriting the FAT driver to use the generic directory search methods I may have missed something. The file is loading, but is spinning off into some infinite loop when trying to save the file. I suspect the FindFirst() call.
Ok, so the flag to allow creation of new directory entries (which will need to extend a cluster-based directory if needed) was not fully implemented and was not returning an empty entry in the root directory to be used for the new file. Let's try this again.
The save was successful, but the new file wasn't found. although I have a sneaking suspicion as to why, I didn't actually write the filename and whatnot to the directory entry before I saved it ... Oops.
Another significant change to the FAT filesystem driver, and one which should solve a bunch of potential issues. I have created a routine which processes the Path and splits it into an array of strings (one for each directory) in the FAT 8.3 format (with correct spacing to match the filenames). This should greatly ease the troubles of dealing with these names and comparing them for matches. It will also handle generation of a save filename and fix an issue with ensuring the file does not already exist.
Everything in place, however, something is wrong now as when using the ATA driver, the filesystem is being corrupted in some way. Needs more investigation.
2010-05-11
2010-05-11 - ATA Driver
Looking back at the ATA driver today and trying to make it conform to the Block Device Driver Interface (BDDI). Having to rework quite a few pieces of it, and make it implement the MountList parser to read the parameters it needs. A significant modification above and beyond the Amiga implementation is the ability to specify the partition number, the device driver will take this and look up the range for that partition, and define the device accordingly.
I have also put a lot more work into the IdentifyDrive ATA command, mainly because the sector count and bytes per sector are required elements of data for the driver context. For this, I have (finally) created a C struct to correspond to the output from the command (at least to ATA-1 specifications) so that this can be extended as and when required.
I have also put a lot more work into the IdentifyDrive ATA command, mainly because the sector count and bytes per sector are required elements of data for the driver context. For this, I have (finally) created a C struct to correspond to the output from the command (at least to ATA-1 specifications) so that this can be extended as and when required.
2010-05-10
2010-05-10 - FAT FileSystem Interface
Finally finished reworking the FAT driver so that everything works through the FileSystem interface. A few things are still missing, such as directory listings, but that's not important right now. I have also (finally) implemented the ability to find a directory entry, edit it, then save it back through generic functionality, which saves greatly on repeated code.
2010-05-02
2010-05-02 - How Processes Find the Kernel
In at the deep end today. I've put together a structure that will be pointed to be a stack pointer in the initial state of every module loaded (library or process) that will give the code the "first step" in starting up. I will almost certainly be reviewing this later, but it'll do for now. All the code is being abstracted through process.c at the moment, but I'm going to have to extract this as it's more library.
2010-04-28
2010-04-28 - Debugging Multitasking
It would be nice to review some of the UI stuff that I have lying around, but I really should continue to work on the library loading parts first, because then I can work on the UI stuff as a separate library, copy it onto the machine, and have it run.
Not only would the code then be separate and compiled independently, I could also use it as a testbed to test out the Library loading parts and start planning out the API layers that will have to exist between the different areas of the OS.
Ok, the first part that I am missing is the filesystem delete code. As I'm currently working with a FAT filesystem in a single-threaded system, a delete procedure should be fairly straightforward. I know, I don't really NEED this in place to be able to proceed with UI devlopment, but I've been putting it off.
Still reviewing the current codebase, and was pleasantly surprised to find that the process system is almost entirely implemented. A process command initialises the system by hooking in the interrupt 0 handler, and a second processtest command initialises two process functions which each do nothing other than loop for a "long" time, then print a character on the screen. All of this works perfectly. If I were to work on this aspect of the OS, it would probably be good to produce a "Top" command to list running processes and also modify the Timer_Delay() function (which currently just loops until the tick counter is correct) so that it suspends the task for a given amount of time. This would be much more efficient as my processes are spinlocking because they woudn't use any CPU time when not running. They can't use the current delay function as they may well miss the expected tick count because they weren't running.
Ok, so that's TOP written. It's horrible code, but it should work for now.
Next thing I need is a state value for the processes. Each processes will either be the current task (running), be waiting to run (waiting) or suspended (sleeping). Once I have this, I can then provide functions to change the state of a task.
It seems there is still some instability in the process subsystem. I introduced a third task to observe the behaviour, and the system threw an Int13 after about 4 "A" iterations. The Code Segment in the exception must be a mistake because it's reading 0xECD7, but nothing I have changes the CS pointer, so how has that happened?
A quick change to the child Processes and they now load their registers with specific values when they start processing. Hopefully, this should not only allow me to test that their register values are being properly maintained, but also see which process it is that is causing it to crash (not that it matters, they are trivial and almost identical). An added bonus is that I should be able to see if there's a phase error in the exception stack frame which would mean that the value displayed as CS is actually from a different register ... if it crashes again.
I wonder what happens if you perform an iret operation to a hlt instruction? Does it go back to being halted? Does the IP advance beyond the hlt? If I have an "idle task" which infinitely halts the processor, can I just leave that running. The processor will awaken as required, do things, then halt again. Maybe have to experiment.
Ok, the processing code failed again, another Int13. The CS pointer is still 0xECD7 oddly enough, suggesting that the failure is the same (or at least similar) as the one before. The full detail is :
EAX = 0x00011816 EBX = 0xFFFFFFFF ECX = 0x000A6DB0 EDX = 0x00011AA0
ESP = 0x0020040F EBP = 0x00200443 ESI = 0x00000001 EDI = 0x00000048
EIP = 0x00000020 CS = 0x0000ECD7 Flg = 0x00000008 Cde = 0x00010202
The Stack and Base pointers look perfectly valid for the values I'm using, which suggests that the interrupt information does not have a phase error, however the A, B, C and D registers do not have the expected values in them. The command that was running immediately before this was interrupted by a character paint event (which happily completed within its time slice).
Not only would the code then be separate and compiled independently, I could also use it as a testbed to test out the Library loading parts and start planning out the API layers that will have to exist between the different areas of the OS.
Ok, the first part that I am missing is the filesystem delete code. As I'm currently working with a FAT filesystem in a single-threaded system, a delete procedure should be fairly straightforward. I know, I don't really NEED this in place to be able to proceed with UI devlopment, but I've been putting it off.
Still reviewing the current codebase, and was pleasantly surprised to find that the process system is almost entirely implemented. A process command initialises the system by hooking in the interrupt 0 handler, and a second processtest command initialises two process functions which each do nothing other than loop for a "long" time, then print a character on the screen. All of this works perfectly. If I were to work on this aspect of the OS, it would probably be good to produce a "Top" command to list running processes and also modify the Timer_Delay() function (which currently just loops until the tick counter is correct) so that it suspends the task for a given amount of time. This would be much more efficient as my processes are spinlocking because they woudn't use any CPU time when not running. They can't use the current delay function as they may well miss the expected tick count because they weren't running.
Ok, so that's TOP written. It's horrible code, but it should work for now.
Next thing I need is a state value for the processes. Each processes will either be the current task (running), be waiting to run (waiting) or suspended (sleeping). Once I have this, I can then provide functions to change the state of a task.
It seems there is still some instability in the process subsystem. I introduced a third task to observe the behaviour, and the system threw an Int13 after about 4 "A" iterations. The Code Segment in the exception must be a mistake because it's reading 0xECD7, but nothing I have changes the CS pointer, so how has that happened?
A quick change to the child Processes and they now load their registers with specific values when they start processing. Hopefully, this should not only allow me to test that their register values are being properly maintained, but also see which process it is that is causing it to crash (not that it matters, they are trivial and almost identical). An added bonus is that I should be able to see if there's a phase error in the exception stack frame which would mean that the value displayed as CS is actually from a different register ... if it crashes again.
I wonder what happens if you perform an iret operation to a hlt instruction? Does it go back to being halted? Does the IP advance beyond the hlt? If I have an "idle task" which infinitely halts the processor, can I just leave that running. The processor will awaken as required, do things, then halt again. Maybe have to experiment.
Ok, the processing code failed again, another Int13. The CS pointer is still 0xECD7 oddly enough, suggesting that the failure is the same (or at least similar) as the one before. The full detail is :
EAX = 0x00011816 EBX = 0xFFFFFFFF ECX = 0x000A6DB0 EDX = 0x00011AA0
ESP = 0x0020040F EBP = 0x00200443 ESI = 0x00000001 EDI = 0x00000048
EIP = 0x00000020 CS = 0x0000ECD7 Flg = 0x00000008 Cde = 0x00010202
The Stack and Base pointers look perfectly valid for the values I'm using, which suggests that the interrupt information does not have a phase error, however the A, B, C and D registers do not have the expected values in them. The command that was running immediately before this was interrupted by a character paint event (which happily completed within its time slice).
Subscribe to:
Posts (Atom)