Ok, so I have persuaded NetBeans to start a new project, and I've rewritten the example OS's multiboot header code, and wired in my Grub-Bochs image.
It now compiles the (noddy) kernel, links it into an ELF binary, and loads Bochs so Grub can load the kernel.
I'm considering how modules, libraries and executables will load as I want to take this opportunity to separate them from the kernel completely.
I think I understand now what crt0.o does in the grand scheme of things. When you write a program, you start your program with a "void main()" and stuff like "stdout" already exists and stuff like "printf" works. When your executable is first loaded, the OS actually passes control to the crt0.o section in your program (the start() function). This routine (which is part of the toolchain which builds programs specifically for that platform) then uses some platform-specific voodoo to get references to things, it gets a reference to the environment somehow, it gets some library to power write() and open(), and generally sets up stuff that the C library will need to operate.
Now, I imagine I'm going to end up with two different types of entity, Executables and Libraries.
An Executable will be loaded into memory, a new thread (or process) will be created, and sent in to the start address of the executable. If this is a C toolchain program, it'll land in crt0.o and start the process. If it's a seat-of-the-pants assembly marvel, it will do something else.
When the program exists, either by calling an Exit() function or by the main thread leaving the building, the process will end, and the memory for the program will be freed.
This can all be achieved by me creating a crt0.o as part of my C library, and compiling programs as normal. They'll run and won't know the difference.
A Library on the other hand, is slightly different. It will be loaded into memory, and will present a set of entry points. It won't have a main thread or process given to it, and it won't have a defined exit point.
I imagine that all libraries will need some form of Open() entry point so that the C library linked into the library can connect all of its functions to the basic kernel functions. I guess it must be like a crt0.o, but it won't set up stdin or stdout, and it might not look to call the main() function once it's finished (although it may call a further initialise() in the library code itself, in case it needs to open further libraries or do other set-up).
Beyond that, the libraries could just be ELF .o files (which have no unresolved symbols) and just has symbols exposed which are its entry points.
(I'd have to investigate how to make a linked .o file with specific symbols exposed and others not (public vs. private) )
This all comes down to whether the device drivers should be executables or libraries. I suspect the answer is that they should be libraries, and will be loaded in the kernel memory space (for the moment) but some of the drivers may launch a new kernel thread in their initialise.
So. A device driver is a library. The kernel will have to be told to load it at some point (possibly by a driver list file, possibly by some form of probe program). The library will be loaded and will have its Open() method called (as all libraries will). I guess the open code will register the device driver with the kernel, or I guess it could start probing the system for compatible devices.
In the Amiga world, the device driver library would be a normal library but with some specific entry points to make it into a driver and then it's loaded especially as a driver ( http://amigadev.elowar.com/read/ADCD_2.1/Libraries_Manual_guide/node029E.html ).
I think the device drivers are going to work in a similar manner. The Device driver will be a library (ELF .o file) and will be loaded into the library list, .... (thought interrupted) ...
2013-06-14
2013-04-27
2013-04-27 - Bootloader Design
So, my plan is this ...
The 512-byte boot sector loads a small 16-bit loader. This loader will then load a text file detailing modules to load, the first being the 32-bit kernel. The 16-bit loader will then load all of these 32-bit modules. Its last act will be to switch to 32-bit mode and start the 32-bit kernel proper.
The downside to this is that the 16-bit loader will have to understand the various file-systems it may encounter, but this can be mitigated by requiring a small boot partition which contains the modules.
The upside is that the config file can be modified to account for many different boot devices, including, but not limited to, the following situations:
The bigger upside is that the 16-bit bootloader will be able to request blocks from any (bootable) block device using the BIOS routines and will be able to load any file by name from a PXE network boot using the BIOS routines. That way, we're not doing the same thing as a certain other OS of booting from a USB drive using the BIOS, jumping into protected mode, then finding that our boot device went away.
One downside is that we only have 640k of space to load the modules into (which should be OK as they can be read-only drivers for the various drives)
The 16-bit loader will also have to create an identifier from the boot device in order to pass this to the 32-bit kernel so that I can find the same boot device using the 32-bit drivers (when the BIOS is no longer available).
It could even be made "intelligent" to the point of identifying (guessing) what device drivers it will need, but it is dependant on the BIOS if it's even possible to get any information about what the booted device is.
Most BIOSes just pretend that a USB hard drive is a normal hard drive, leaving a surprise for the OS when they jump to 32-bit armed with an ATA driver and find no drive to boot from
The 512-byte boot sector loads a small 16-bit loader. This loader will then load a text file detailing modules to load, the first being the 32-bit kernel. The 16-bit loader will then load all of these 32-bit modules. Its last act will be to switch to 32-bit mode and start the 32-bit kernel proper.
The downside to this is that the 16-bit loader will have to understand the various file-systems it may encounter, but this can be mitigated by requiring a small boot partition which contains the modules.
The upside is that the config file can be modified to account for many different boot devices, including, but not limited to, the following situations:
- Booting from a floppy (which will need a floppy driver)
- Booting from a hard drive (which will need a S/PATA driver)
- Booting from a CD-ROM drive (which will need an ATAPI driver)
- Booting from a network (which requests file by name)
- Booting from a USB device (which requires PCI, USB, and class drivers)
The bigger upside is that the 16-bit bootloader will be able to request blocks from any (bootable) block device using the BIOS routines and will be able to load any file by name from a PXE network boot using the BIOS routines. That way, we're not doing the same thing as a certain other OS of booting from a USB drive using the BIOS, jumping into protected mode, then finding that our boot device went away.
One downside is that we only have 640k of space to load the modules into (which should be OK as they can be read-only drivers for the various drives)
The 16-bit loader will also have to create an identifier from the boot device in order to pass this to the 32-bit kernel so that I can find the same boot device using the 32-bit drivers (when the BIOS is no longer available).
It could even be made "intelligent" to the point of identifying (guessing) what device drivers it will need, but it is dependant on the BIOS if it's even possible to get any information about what the booted device is.
Most BIOSes just pretend that a USB hard drive is a normal hard drive, leaving a surprise for the OS when they jump to 32-bit armed with an ATA driver and find no drive to boot from
2012-01-01
2012-01-01 - Time and Video Drivers
Being the start of a new year, I figure what better "time" to write the C library time.h functions.
Although, the other idea of adding a SetTransform() to the Video Driver interface to allow the code to set a 2D transformation matrix to allow subsequent plots and whatnot to be scaled is inspired.
I've separated out the definitions for the DOS driver interface and created a file for the Video Driver interface. Unlike previous incarnations of this, I have added the same Context approach as other drivers use so that multiple displays on one adapter (or multiple adapters of the same type) can be handled easily.
Although, the other idea of adding a SetTransform() to the Video Driver interface to allow the code to set a 2D transformation matrix to allow subsequent plots and whatnot to be scaled is inspired.
I've separated out the definitions for the DOS driver interface and created a file for the Video Driver interface. Unlike previous incarnations of this, I have added the same Context approach as other drivers use so that multiple displays on one adapter (or multiple adapters of the same type) can be handled easily.
2011-12-28
2011-12-28 - Structure Changes
I've added the Libs, list and DOS elements back into the kernel, and wired them in. I've also changed the way that the standard C library features are structured into the kernel. They now have their own virtual directory courtesy of NetBeans. I've also changed the naming convention for the standard function to remove the prefix which I had been using before.
2011-12-05
2011-12-05 - NetBeans and Subversion
Recovering from the flu, I decided to have a look at NetBeans. I have it installed on this new machine, along with MinGW and MSYS, but I haven't gotten around to doing any OS dev work yet.
I looked at NetBeans and poked around with a few test projects, trying to figure out how best to get my structure of OS to work through the IDE. I've come up with the idea that creating a C/C++ Static Library is the best approach, and putting 99% of the OS code into it. That seems to use the ar tool to create a .a file which contains all the .o files. I have then added post-build steps into the project makefile to take this .a file and link it in with my existing start.o to create the binary kernel files as I have before. The makefile also compiles the bootblock. I have set the Project run command to run the Bochs bxrc file (via a batch file) to then fire up Bochs with my OS loaded.
A "quick" install of Apache and Subversion later, and NetBeans is happy to push the code into the source control repository, and I can keep track of changes. All looks good.
Currently, this set of code only contains a few of the previous files, it sets 32-bit protected mode, clears the VGA Mode 3 screen, and displays a title. It's more of a test for the environment. In any case, if this approach works, it will make a very good starting point to re-integrate all of the code and create the modular kernel that should have been working ages ago.
I have sent a copy of the project to Pink to test on his Linux install. He reports that he has been able to get it to build by adding a Debug-Linux configuration into the project and tweaking some settings. We may be able to get a Subversion repository exposed so that it can become a collaborative project, at least from the point of view of being able to get the code and run the progress so far.
I looked at NetBeans and poked around with a few test projects, trying to figure out how best to get my structure of OS to work through the IDE. I've come up with the idea that creating a C/C++ Static Library is the best approach, and putting 99% of the OS code into it. That seems to use the ar tool to create a .a file which contains all the .o files. I have then added post-build steps into the project makefile to take this .a file and link it in with my existing start.o to create the binary kernel files as I have before. The makefile also compiles the bootblock. I have set the Project run command to run the Bochs bxrc file (via a batch file) to then fire up Bochs with my OS loaded.
A "quick" install of Apache and Subversion later, and NetBeans is happy to push the code into the source control repository, and I can keep track of changes. All looks good.
Currently, this set of code only contains a few of the previous files, it sets 32-bit protected mode, clears the VGA Mode 3 screen, and displays a title. It's more of a test for the environment. In any case, if this approach works, it will make a very good starting point to re-integrate all of the code and create the modular kernel that should have been working ages ago.
I have sent a copy of the project to Pink to test on his Linux install. He reports that he has been able to get it to build by adding a Debug-Linux configuration into the project and tweaking some settings. We may be able to get a Subversion repository exposed so that it can become a collaborative project, at least from the point of view of being able to get the code and run the progress so far.
2011-07-13
2011-07-13 - Threads and Processes
A few more tweaks to the Process subsystem today. I've taken an old set of code that I was working on before this issue with Bochs not working came up, and I applied the fixes to make it work in Bochs to it. After a little tweaking, it started to work and tick along correctly.
Tested a few threads as well (the character generator program that I developed previously) and it all seemed to work. There is a slight issue with the display when scrolling which the semaphore should have fixed, but I think it's actually a render issue with Bochs. The semaphore itself tested correctly.
The next step from here is to remove the Sleep(Time) code from the process system and implement a GetPid() function, Sleep(Pid) and Wake(Pid) to allow threads to control one another. The Sleep(Time) will be reimplemented as a separate subsystem which will use its own thread (or even an interrupt handler) to wake a thread up once its timer has expired.
This should then give me the following functions:
The Top command would be modified to display the Pid in the table, and command versions of Sleep() and Wake() will be added to allow thread control for testing and lolz.
Tested a few threads as well (the character generator program that I developed previously) and it all seemed to work. There is a slight issue with the display when scrolling which the semaphore should have fixed, but I think it's actually a render issue with Bochs. The semaphore itself tested correctly.
The next step from here is to remove the Sleep(Time) code from the process system and implement a GetPid() function, Sleep(Pid) and Wake(Pid) to allow threads to control one another. The Sleep(Time) will be reimplemented as a separate subsystem which will use its own thread (or even an interrupt handler) to wake a thread up once its timer has expired.
This should then give me the following functions:
- GetPid(): Returns the current Pid
- Sleep(Pid): Sets the given Pid to sleep
- Wake(Pid): Sets the given Pid to waiting
- Sleep(Time): Sleeps the thread for some time
- TickHandler(): Checks for threads needing waking and wakes them.
The Top command would be modified to display the Pid in the table, and command versions of Sleep() and Wake() will be added to allow thread control for testing and lolz.
2011-07-12
2011-07-12 - Bochs and Threads
Recently I have been looking into Bochs to try to figure out why the multitasking code (which works on physical machines) fails under Bochs.
The discovery was made yesterday that the reason it was failing under Bochs specifically is because Bochs does not start with zero-initialised memory, and that the initialisation code for my multitasking did not clear its variables. What's more is that the compiler assumed that the BSS would be zeroed, and even refused my explicit zero initialisation. This was fixed up and the code got further before crashing.
Last night and today, I was looking into the Bochs debugger and trying to figure out why it seemed that the code compiled on my laptop was causing it to fail even after fixing the above issue. After hours of stepping and deleting code and stepping and disassembling, I could not figure how the EBP register was being overwritten by the EIP value.
Finally, I spotted it. The code I was using (modified from some tutorial code) set up the tasking jump using unnamed registers, and the GCC compiler was choosing its own registers to use. Unfortunately, one of it's choices conflicted with a register I was using. A few tweaks later and the code compiled correctly. Also setting the "clobber" registers to the call made it compile reliably.
This was a particularly difficult issue to track down as each time the code was changed and recompiled, it could have compiled differently, and some changes would "fix" the problem, even though they did not directly affect it. This made the problem intermittent at best.
The discovery was made yesterday that the reason it was failing under Bochs specifically is because Bochs does not start with zero-initialised memory, and that the initialisation code for my multitasking did not clear its variables. What's more is that the compiler assumed that the BSS would be zeroed, and even refused my explicit zero initialisation. This was fixed up and the code got further before crashing.
Last night and today, I was looking into the Bochs debugger and trying to figure out why it seemed that the code compiled on my laptop was causing it to fail even after fixing the above issue. After hours of stepping and deleting code and stepping and disassembling, I could not figure how the EBP register was being overwritten by the EIP value.
Finally, I spotted it. The code I was using (modified from some tutorial code) set up the tasking jump using unnamed registers, and the GCC compiler was choosing its own registers to use. Unfortunately, one of it's choices conflicted with a register I was using. A few tweaks later and the code compiled correctly. Also setting the "clobber" registers to the call made it compile reliably.
This was a particularly difficult issue to track down as each time the code was changed and recompiled, it could have compiled differently, and some changes would "fix" the problem, even though they did not directly affect it. This made the problem intermittent at best.
Subscribe to:
Posts (Atom)