2016-10-10

2016-10-10 Thread Synchronisation

I went back to an old kernel (the DriveAccess kernel from 2011) to review the code I created for the network stack there.  It wasn't anything fancy, it could send and receive UDP packets, and could respond to ICMP pings.  I had a function set up in the kernel so that it could download files from the TFTP server and display them, but the rest of the kernel wasn't complete enough to really take advantage of it.  For one thing, it was only a single thread with interrupts.

Having kernel threading now working, I set about researching the BSD Socket library and working out how best to fit an implementation into the kernel, particularly with a loopback interface so that I could create a server thread and a client thread and let them talk.

Unfortunately, I didn't get far down this road before I really needed some good old fashioned thread synchronisation.

The Basics


Interrupts can happen at any time, they don't care what you're doing, they will interrupt the currently running code and let some other code run for a while.  That's all well and good until you're half-way through updating some structure, you get interrupted, then the next task tries to read that data and finds it broken.  Bad things ensue.  In SQL Server, there are lot of fanciful things such as transaction isolation levels and snapshots and whatnot to assist you in not reading half-written data unless you really want to.

2016-09-25

2016-09-25 Kernel Threads and VT100 Console

The Ring 3 test from yesterday is useful and proves the TSS, but the rest of my OS is not quite set up yet to handle everything from process space with system calls and protected memory and the like.  What I really need is a way to run multiple tasks that are within kernel space.  What I need is KERNEL THREADS!

Because I already have much of a window manager in place, I made a quick change to the kernel main function so that instead of just opening a window for the console, it opens two more windows.  My test case is then a couple of functions which write characters into those other windows with a delay of of a few thousand nops, which will then get set to run as separate threads.

The method signature for the function to create a kernel thread looks like this
int32 process_kernelThreadStart( void* pThreadStart, size_t pStackSize, int32_t pParamCount, ... );
It allocates a stack for the thread and copies the parameters from its own stack into the thread stack it just allocated.  It also writes the address of a termination function to the stack so that if the thread function ever exits, it "returns" to termination function is called which collects the return value and terminates that thread.

2016-09-24

2016-09-24 More Multitasking

A significant feature of many operating systems is the ability to multi-task.  You may be running Calculator and Sound Recorder and Wordpad and Sticky Notes and Doom and Paint and Command Prompt and Clippy, but they aren't actually running at the same time in the processor.  Instead, the processor is very quickly switching between them (with each running for maybe a few microseconds at a time) to make it appear if they are happening at once.

My multitasking system will be what is called pre-emptive multitasking, which is where the OS uses interrupts to pause the running programme in order to switch to the next programme.  This means that the programmes can be written as if they're the only programme on the system, the kernel takes care of the heavy lifting, and the programmes aren't usually aware that they were even interrupted.

Before I could design the multitasking system (or even fully understand a lot of the tutorials and articles on designing a multitasking OS) I had to go back and do a lot more research to get a solid footing in the concepts (hence the confused previous post).

2016-08-21

2016-08-21 Multitasking

One of the last big hurdles to conquer before my kernel can have "v1.0" stamped on the packaging is that of multitasking.  There's a lot of theory to cover off and a specifics for the x86 architecture to grasp before it can be done, so ... to the Vroomster!

Context Switching


The biggest part of multitasking is being able to switch context.  This is when the processor decides to stop what it is doing, record the state of where it got to with the current task, fetching where it got to with a different task, then carrying on.
In order to do this, we need to store and restore the Processor Registers, the Stack, and the Memory Space,  These aren't all necessarily required, of course, restoring the registers can mean switching the stack or the memory space, and switching the memory space probably means you've replaced the stack.

So we need to make sure we understand which of these things we need to store and restore for specific circumstances, and how we get to them.

When we're switching context, we're likely to need to do this for two reasons; we are surrendering the context voluntarily (maybe because we're waiting for some IO operation before we continue), or an interrupt has happened (either a device requires attention, or the PIT has indicated that our time slice is up).

Registers


This is the biggest part of the context switch that we need to worry about, so let's make sure we don't miss anything.  We're a 32-bit operating system so the registers we need to worry about are:

  • General Purpose registers: EAX, EBX, ECX, EDX
  • Index Registers: ESI, EDI
  • Instruction Pointer: EIP
  • Flags: EFLAGS
  • Stack Registers: EBP, ESP
  • Floating Point / MMX registers: ST(0) to ST(7)
  • Segment Registers: CS, DS, ES, FS, GS, SS
  • Descriptor Table Registers: GDTR, IDTR, LDTR
  • Task Register: TR
  • Control Registers: CR0 to CR4, MSW
  • SME Registers: MXCSR, XMM0 to XMM7
  • Debug Registers (what are these?): DR0 to DR3


Next up, when might we need to persist these registers?  Well, there are two instances where persisting registers may be needed; when an interrupt occurs, or when the process surrenders the processor.

We'll look at the interrupts first because the interrupts persist some of the registers to the stack anyway, so it's quite likely that if we're going to have a function to surrender the processor, it's going to want to do exactly the same thing as the interrupt handler.  If we can achieve this, then we don't need a special function to reinstate a process that surrendered context differently to one that was pre-empted.

Looks like when an interrupt fires, the CPU persists the EFLAGS register, and the instruction pointer (EIP) as the return address.

I wonder if we can implement the "process surrendering control" as an interrupt itself?  Possibly, but as most process surrendering will be waiting for an IPC message, the process is going to be in the kernel anyway when it surrenders.  So most processes will be making a request of the kernel, the kernel will be suspending the process.

There's also the question of privilege level rings, which in the x86 world are from ring 0 (highest privilege kernel stuff) to ring 3 (lowest privilege userland).  When interrupts fire, the kernel has to change stacks and do all sorts of black magic (apparently) to maintain security.

I think maybe more research is needed on the basics of the processor architecture, particularly around interrupt handling between ring levels.  Maybe I'll put a pin in this and come back to it later.


2016-05-22

2016-05-22 Recent Progress Update

I haven't posted an update in a while because I haven't had a lot of time to dedicate to my OS unfortunately.

Instead, most of my free time in the last year or so has been spent learning about and investigating newer .NET web technologies and how the Microsoft web development stack is changing.  That said, I have spent some time working on Atomicity and have advanced the plot on a few angles.  I'll write these up in full in future, but here are the headlines.