diff options
Diffstat (limited to 'DESIGN')
| -rw-r--r-- | DESIGN | 77 |
1 files changed, 65 insertions, 12 deletions
@@ -1,19 +1,72 @@ SCHEDULER -- 4-heap (or 2-heap) with linked nodes -- epoll -- edge-triggered monitoring for file i/o (no syscalls to modify fdset) +- worker thread pool executes read to be run fibers +- thread pool executes fibers from FIFO queues (one per each priority) +- possible to make main thread separate scheduler so we can bind some + fibers to run in main thread only (not needed?) + +THE BIG SCHEDULING QUEUE (shared queue with all worker threads) + --- works also between machines, in which case we just always queue + work items to remote vmach (need to send wakeups too) + - steal first item from the local cache if any + - mutex lock + - if stuff to push + - while idle_vcpu not empty + - pop idle_vcpu from list + - push item to idle_vcpu + - push rest of items to global lists accordingly + - mutex unlock + - call futex_wake for all idle_cpu's we stuffed + - return stolen item + - if stuff on list + - pop item from list + - mutex unlock + - return popped_item + - insert self to idle_vcpu list + - mutex unlock + - futex_wait on local_pointer + - return local_pointer + - mutex unlock + +WAKEUP + - as minimum keep wakeup bitfield for most common types, and use + prepare sleep, and sleep primitives with acceptable wakeup reasons + - might add so that wakupper calls wakeupee to execute their wakeup + condition check once ask rescheduling based or that or not; this way + empty spin does not require pushing to scheduler queue and thread + pool wakeup + +IO SLEEPING/DISPATCHING +- epoll in edge-triggered monitoring for file i/o +- epoll_wait done in separate thread (or possibly as regular fiber) + which sole purpose is to wakeup threads for thread pool - signalfd for signal handling -- eventfd for thread pool wakeup +- timerfd for timeout handling + +INTER FIBRE CALLS (IFC) +- sends a callback to be execute under some other fibre +- executed at tf_ifc_process() points, or at tf_exit() time +- might need tfc_ifc_wait() for synchronizing with ifc completion +- sending uses atomic LIFO (single atomic cmpxchg) +- receive does LIFO flush (single atomic xchg) and reverses order(?) +- need way to check if IFC is queued or not (so it's safe to reuse it) + +TIMEOUTS +- one (or more) fibers serving as alarm generators +- 4-heap with mremappable heap array +- receives IFC calls from other threads +- receives io wakeups from timerfd +- sends async signal to fiber after timeout FIBERS -- timer node on fiber control data (for delayed heapify; and reuse on timeouts) - fd, signal, pid wait struct on stack of wait function - fiber_kill can interrupt wait -SCHEDULER <-> THREAD POOL -- scheduler queues to thread pool array fifo queue, - - semaphore protects threads so no underqueueing happens - - eventfd notifies scheduler when queue has free space again - - head/tail updated atomically and item position recovered -- thread pool atomically pushes to resume atomic stack and sets eventfd - - scheduler thread atomically pops all and updates fiber states +FIBRE WAKE UP QUEUES +- mostly not needed (alarms, io uses async signal wakeups) +- for mutex, semaphores, possibly send io +- use thread mutexes for list access +- wait on single queue only(?) + +FIBRE MUTEX +- simple wakeup queue, sleep on queue, uninterruptible? + |