Revised thread/timer handling, work queue and scheduling.

Updated quagga thread handling to use qtimers when using the new
qpnexus -- so all timers are qtimers in the new scheme.

Updated work queue handling so that each work queue item is a single
malloced structure, not three.  (Only bgpd and zebra use the work
queue system.)

When using qpnexus the background thread queue is no longer a timer
queue, but simply a list of pending background threads.  When a
background thread is waiting on a timer, it is in the qtimer pile,
same like any other thread.

When using qpnexus, the only remaining quagga thread queues are the
event and ready queues.

Revised the qpnexus loop so that only when there is nothing else to
do will it consider the background threads.

Revised write I/O in the BGP Engine so that all writing is via the
connection's write buffer.  Revised the write I/O in the Routeing
Engine, so that it passes groups of updates in a single mqueue
message.  This all reduces the number of TCP packets sent (because
BGP messages are collected together in the connection's write buffer)
and reduces the number of mqueue messages involved.
(No need for TCP_CORK.)

Code and comments review for the new code.

  modified:   bgpd/bgp_advertise.c
  modified:   bgpd/bgp_common.h
  modified:   bgpd/bgp_connection.c
  modified:   bgpd/bgp_connection.h
  modified:   bgpd/bgp_engine.h
  modified:   bgpd/bgp_fsm.c
  modified:   bgpd/bgp_main.c
  modified:   bgpd/bgp_msg_read.c
  modified:   bgpd/bgp_msg_write.c
  modified:   bgpd/bgp_network.c
  modified:   bgpd/bgp_packet.c
  modified:   bgpd/bgp_packet.h
  modified:   bgpd/bgp_peer.c
  modified:   bgpd/bgp_peer_index.h
  modified:   bgpd/bgp_route.c
  modified:   bgpd/bgp_route_refresh.h
  modified:   bgpd/bgp_session.c
  modified:   bgpd/bgp_session.h
  modified:   bgpd/bgpd.c
  new file:   bgpd/bgpd.cx
  modified:   lib/mqueue.h
  modified:   lib/qpnexus.c
  modified:   lib/qpnexus.h
  modified:   lib/qpselect.c
  modified:   lib/qtimers.c
  modified:   lib/qtimers.h
  modified:   lib/sigevent.c
  modified:   lib/stream.c
  modified:   lib/stream.h
  modified:   lib/thread.c
  modified:   lib/thread.h
  modified:   lib/workqueue.c
  modified:   lib/workqueue.h
  modified:   tests/heavy-wq.c
  modified:   zebra/zebra_rib.c

1 files changed, 7 insertions, 13 deletions

diff --git a/lib/qpselect.c b/lib/qpselect.c
index 7df59752..d3f8e5ad 100644
--- a/lib/qpselect.c
+++ b/lib/qpselect.c
@@ -270,13 +270,8 @@ qps_set_signal(qps_selection qps, int signum, sigset_t sigmask)
     } ;
 } ;
 
-/* Execute a pselect for the given selection -- subject to the given timeout
- * *time*.
- *
- * The time-out time is an "absolute" time, as measured by qt_get_monotonic().
- *
- * A timeout time <= the current qt_get_monotonic() is treated as a zero
- * timeout period, and will return immediately from the pselect.
+/* Execute a pselect for the given selection -- subject to the given maximum
+ * time to wait.
  *
  * There is no support for an infinite timeout.
  *
@@ -289,7 +284,7 @@ qps_set_signal(qps_selection qps, int signum, sigset_t sigmask)
  * The qps_dispatch_next() processes the returns from pselect().
  */
 int
-qps_pselect(qps_selection qps, qtime_mono_t timeout)
+qps_pselect(qps_selection qps, qtime_t max_wait)
 {
   struct timespec ts ;
   qps_mnum_t  mnum ;
@@ -334,16 +329,15 @@ qps_pselect(qps_selection qps, qtime_mono_t timeout)
   qps->tried_fd_last = qps->fd_last ;
   qps->pend_fd       = 0 ;
 
-  /* Convert timeout time to interval for pselect()     */
-  timeout -= qt_get_monotonic() ;
-  if (timeout < 0)
-    timeout = 0 ;
+  /* Make sure not trying to do something stupid        */
+  if (max_wait < 0)
+    max_wait = 0 ;
 
   /* Finally ready for the main event                   */
   n = pselect(qps->fd_last + 1, p_fds[qps_read_mnum],
                                 p_fds[qps_write_mnum],
                                 p_fds[qps_error_mnum],
-                                qtime2timespec(&ts, timeout),
+                                qtime2timespec(&ts, max_wait),
                                 (qps->signum != 0) ? &qps->sigmask : NULL) ;
 
   /* If have something, set and return the pending count.               */