Submit: Turn in your entire xinu-hw7 directory source files using the turnin command on morbius.mscsnet.mu.edu or one of the other Systems Lab machines. Please run "make clean" in the compile/ subdirectory just before submission to reduce unnecessary space consumption.

Work should be completed in pairs. Be certain to include both names in the comment block at the top of all source code files, with a TA-BOT:MAILTO comment line including any addresses that should automatically receive results. It would be courteous to confirm with your partner when submitting the assignment.

Preparing for Multicore

Multicore Raspberry Pi 3 B+

It is time for multicore. Thus far in the term, we've been using only the first core, ("Core 0"), on the Raspberry Pi 3 Model B+ platform, which has four cores within its ARM Cortex-A53 processor. In prior years, this course used single-core platforms (i.e., PowerPC machines, Linksys MIPS routers, Raspberry Pi 1 boards) to build our O/S. Since 2019, Marquette has used real multicore hardware (as opposed to virtual machines) to lead hands-on laboratory assignments in Operating Systems. As best we can tell, we are one of the first universities in the world (possibly the first!) to do this successfully as part of a regular undergraduate course. (You're welcome!)

Now that your embedded O/S will be taking advantage of this multicore architecture, you need to know the multicore hardware operations built in for your use.

 

Unparkcore()

Although the concept of unparking is general to multicore platforms, the specific implementations vary from one platform to another. When the Pi 3 B+ boots, only core 0 is setup to run, and cores 1-3 are put in a sleep state. (The startup sequence in start.S executes the WFE opcode, "Wait For Event", on each of the other cores.) To wake up, or unpark, a core, core 0 sends an event (ARM opcode SEV, "Send EVent",) to wake another core. Upon waking the new core will check the value in a special predetermined location known as its mailbox. The core then begins execution at the address that it finds in its mailbox.

In this example, core 3 is unparked and sent to execute a given test_procedure() with no arguments (NULL). In practice, your O/S will unpark each core with its own dedicated NULL Process.

Note: unparkcore() is already implemented for you and can be found in system/unparkcore.c

 unparkcore(int core, void* address, void* argument);

In the implementation of unparkcore(), you will see that it will unpark the core to an assembly routine called setupCore() (found in system/setupCore.S). setupCore() is necessary because the core needs to undergo specific initializations before any of your code can run.

Note: Source files unparkcore.c and setupCore.S are already implemented for you.

Preparation

First, make a copy of your Project 6 directory:
    cp -R xinu-hw6 xinu-hw7
Then, untar the new project files on top of it:
    tar xvzf ~brylow/os/Projects/xinu-hw7.tgz

New files:

• system/
• atomic.c
• atomic_utils.S
• unparkcore.c
• setupCore.S
• mmu.c
• mmu_util.S
• include/
• atomic.h
• cas_lock.h
• mmu.h
• core.h

New versions of existing files:

• system/
• clkinit.c
• clkhandler.c
• initialize.c
• platforminit.c
• start.S
• include/
• bcm2837.h
• clock..h
• xinu.h

The new files provide the utilities and code for initializing the new cores, building a NULLPROC for each core, and adapting the clock system for multiple cores and preemption.

But default, the first for PIDs in the system will be reserved, for NULLPROC0, NULLPROC1, NULLPROC2 and NULLPROC3. As in our single core implementation, the NULLPROC is not expected to do anything meaningful once core initialization is complete. However, the NULLPROC for a given core is always available to run, so when a core has no other work, it may twiddle its thumbs running its NULLPROC. Changes in start.S and initialize.c allocate dedicated stack space for each NULLPROC, setup the initial process control blocks, and unpark each core. A copy of the main() process is launched for each core, but only the main() process on Core 0 will call your testcases() function.

New clock code initializes the timer hardware on each of the four cores, setting up each to fire timer interrupts at 1 millisecond intervals. When the kernel.h PREEMPT constant is TRUE, clock interrupts will still force a call to resched(), preempting whatever process is currently running on that core. However, only Core 0 timer interrupts will update global clocktimer variables. The single preemption counter is replaced with an array of preemption counters, one per core.

Locks

With multiple cores in the system, mutual exclusion and locking of critical sections become of paramount importance. In atomic_utils.S, we provide ARM assembly implementation of a "Compare and Swap" ("CAS") primitive that works correctly across mulitple cores using the LDREX and STREX opcodes. In atomic.c, we provide functions for safely, atomically incrementing and decrementing global variables. In lock.c, we provide a simple multicore spinlock implementation. You are expected to study these and understand how they work.

TODOs

All of your work in this assignment involves modifying existing files in your O/S, so there are no new TODOs in the files. Instead, they are here.

• Protect the global variable numproc from multicore race conditions. First, mark this global as "volatile". This tells the C compiler to be wary of other cores potentially changing the variable concurrently, and prevents some kinds of compiler optimizations that will otherwise break multicore code. More importantly, updates to global numproc, which tracks the total number of processes in the system, must now be guarded as atomic operations. Use the atomic functions in atomic_utils.S to protect numproc whenever it is incremented or decremented. The number of processes in the system changes in two places: create.c and kill.c.
• The global variable currpid must now become an array of volatile ints, one for each core to track the current running PID. Any referencesto currpid in the O/S must now be indexed by the ID of the current core, which can be found with the provided getcpuid() function.
• The mechanism for allocating a fresh PID in newpid() at the bottom of create.c is completely unsuitable for multicore concurrency. Between the time that newpid() checks the state of a prospective new PID, and the time when create() claims that PID by settings its state to PRSUSP, another core could well have concurrently claimed the same PID. This is a dangerous race condition, and you should replace the newpid() design with a new mechanism that relies on atomic operations to find and claim a fresh PCB from the table.
• The resched.c function is now a critical section of code. Because it modifies the global readylist, only one core at a time should be executing this code. Guard the critical section of resched() with a spinlock (see lock.c), but be sure to release the lock before calling ctxsw().
• The kprintf() function is also now a critical section. Serial port output will be indecipherable if multiple cores are allowed to access the serial port hardware simultaneously. Guard the function with a spinlock to guarantee sensible output.
• Modify your lottery scheduling algorithm in resched.c to ignore the four NULLPROCs in both counting tickets and selecting a winning ticket. When your lottery scheduler finds no processes eligible to run next, it should default to running the NULLPROC for that core instead.

Notes

• When guarding a critical section of code, we need a lock location that is globally visible to all cores, but ideally that is not also readily visible to every other part of the O/S. The C language permits us to declare a static volatile int as a local variable in a function. "Static" in this context means that other functions will not be able to see this location by name, while "volatile" discourages problematic compiler optimizations, as described above.
• To submit your project, please run "make clean" in your compile directory, change directory back two levels, and submit the entire system with the command turnin xinu-hw7. You can verify what you have submitted using turnin -v.