Reworking of peer state handling.

This fixes a reported assert() in 'no neighbor xxx'.

It also fixes other issues found when reviewing and testing that.

Also fixed is a reported segfault caused by prefix lists without explicit
sequence numbers.

Peer State
==========

Tightened up the peer state handling, including:

  * shades of pIdle -- depending on some peer flags and the session, if any

  * state transitions -- particularly pEstablished -> pIdle or pClearing

  * handling if deleting peer and associated session

  * handling of PEER_DOWN_XXX -- why peer was last downed

  * handling of last NOTIFICATION sent/received

RS Client RIBs
==============

Cleared up places where RS Client RIBs were not disposed of properly when a
peer's afi/safi state changed, in particular:

  -- when deactivating an afi/safi
  -- when unsetting the rs client state for an afi/safi
  -- when binding a peer to a group for an afi/safi

In the past these issues were probably invisible, except for a slight leak of
memory.  With the newer code these issues triggered some asserts when closing
down a peer or the entire program.

Program Termination
===================

Now terminates by deleting all peers -- essentially 'no neighbor' for all
peers.  Each time a peer is set to be deleted bm->peer_linger_count is
incremented, when the peer finally is deleted, the count is decremented.

So while in program_terminating state, all nexuses continue to run until
the Routing Engine spots that there are no peers left to delete.  Then all
nexuses are shut down and the program finally terminates.  To do this, when
termination starts a new Routing Engine foreground task is added, which
calls program_terminate_if_all_peers_deleted().

Accept() Status for Session
===========================

The accept() code needs to find the session associated with the incoming IP
address.  Then it needs to see if can accept the incoming connection.

It looks up the IP address in the Peer Index (under its Mutex).

Previously: the Peer Index entry contained a pointer to the accepting
connection (if any), and the session had a pointer to the Peer Index entry
so that it could set/clear the accept field in that entry.

Now: have removed the accept field in the Peer Index entry, and the pointer
from the session to that entry.

There is now an "accept" flag in the session structure -- so setting/clearing
accept does not have to fiddle with any state to do with the peer.  Which
seems cleaner.

To find the session, the accept() code goes via the Peer Index and then via
the peer to find the session.  This is done under the Peer Index Mutex.  To
support that, the Routing Engine only changes the peer->session pointer under
the same Mutex.

General Changes
===============

  1. name changes: peer_lock -> bgp_peer_lock etc.

  2. bgp_notify_send           -> bgp_peer_down_error
     bgp_notify_send_with_data -> bgp_peer_down_error_with_data

  3. name changes: bgp_peer_sIdle -> bgp_peer_pIdle etc.

  4. changing int to bool

  5. added "received" flag to bgp_notify structure

Files Affected
==============

  configure.ac                -- set version to 0.99.15ex02

  bgpd/bgp.h                  -- format changes only

  bgpd/bgp_advertise.c        -- (1)

  bgpd/bgp_attr.c             -- (2)

  bgpd/bgp_common.h           -- (3)

  bgpd/bgp_connection.c       -- (4) for session->active
                               - adding session->accept flag
                               - removing peer index accept entry
                               - adding bgp_connection_query_accept()

  bgpd/bgp_connection.h       -- adding bgp_connection_query_accept()

  bgpd/bgp_debug.c            -- (3) (5)
                               - changed bgp_notify_print() to remove "sending"
                                 parameter and use (5)

  bgpd/bgp_debug.h            -- changed bgp_notify_print()

  bgpd/bgp_fsm.c              -- added TODO for NSF and for
                                 CollisionDetectEstablishedState

  bgpd/bgp_main.c             -- (4) for various flags
                               - added static bool program_terminating
                               - used "mqb_priority" name instead of "1"
                               - ignore SIGHUP and SIGTERM messages once is
                                 "program_terminating"
                               - added program_terminate_if_all_peers_deleted()
                               - in SIGTERM: set program_terminating and add
                                 the foreground hook.

  bgpd/bgp_mplsvpn.c          -- (3)

  bgpd/bgp_msg_read.c         -- (5) set the "received" flag on incoming
                                     notifications
                               - update call of bgp_notify_print()

  bgpd/bgp_msg_write.c        -- update call of bgp_notify_print()

  bgpd/bgp_network.c          -- (4)

  bgpd/bgp_nexthop.c          -- (3)

  bgpd/bgp_notification.c     -- add "received" flag to notification structure,
                                 which is false by default.

  bgpd/bgp_notification.h     -- add "received" flag to notification structure
                               - add bgp_notify_set_received()
                               - add bgp_notify_get_received()

  bgpd/bgp_open.c             -- (2)

  bgpd/bgp_open_state.c       -- in bgp_peer_open_state_receive():
                                  - copy the session->hold_timer_interval
                                         and session->keepalive_timer_interval
                                    values (as negotiated by the BGP Engine)
                                    into the peer
                                  - set PEER_CAP_RESTART_RCV if have
                                  - fix typo, use: open_recv->can_preserve
                                              not: open_recv->can_g_restart

  bgpd/bgp_packet.c           -- delete bgp_notify_send() and
                                                     bgp_notify_send_with_data()
                               - (1) (2) (3)
                               - bgp_clear_route_normal() -> bgp_clear_routes()
                                 and now returns "completed" state.

  bgpd/bgp_packet.h           -- delete: bgp_keepalive_send()
                                         bgp_open_send()
                                         bgp_notify_send()
                                         bgp_notify_send_with_data()

  bgpd/bgp_peer.c             -- changed:
                                   bgp_session_has_established()
                                   bgp_session session()
                                   bgp_session_has_disabled()
                                 to void and to take session, not peer.
                               - removed lock of session structure in
                                 bgp_session_do_event() -- was holding the lock
                                 for far longer than necessary, particularly
                                 when clearing routes !
                               - in bgp_session_has_established():
                                  - lock session structure where and only where
                                    required
                                  - tidied up timer handling
                               - in bgp_session_has_stopped():
                                  - extend IdleHoldTime
                                  - examine notification etc to see why session
                                    came down.
                                  - invoke bgp_peer_down_notify(), which will
                                    start the process of downing the peer.
                               - in bgp_session_has_disabled():
                                  - removed defer_enable
                                  - removed calls to bgp_peer_stop() etc.  That
                                    is done in bgp_peer_down() et all.
                                  - now if session is marked delete_me, then do
                                    that; otherwise, can enable again.
                                  - removed program_terminate_if_all_disabled(),
                                    replaced by new mechanism
                               - in bgp_peer_stop():
                                  - changed to void function and added nsf
                                    parameter.
                                  - sets pClearing state.
                                  - MUST now only be called when pEstablished.
                                  - removed some code to bgp_peer_reset_idle().
                                    So... bgp_peer_stop() brings pEstablished
                                    peer to halt, while bgp_peer_reset_idle()
                                    prepares it for new session.
                                  - now passes nsf to bgp_clear_all_routes(),
                                    which returns flag to say whether task is
                                    complete or whether it continues in
                                    background.
                                  - clearing of NSF_MODE and flags moved to
                                    bgp_peer_reset_idle().
                                  - sets pIdle state if route clearing completed
                               - renamed peer_nsf_stop() -> bgp_peer_nsf_stop()
                                  - if is pIdle or pClearing and have NSF
                                    routes, then stops timers and clears out
                                    all the NSF routes.
                               - added bgp_peer_clear_all_stale_routes()
                               - added bgp_peer_shutdown() -- used when peer
                                 is downed for PEER_DOWN_USER_SHUTDOWN !
                               - added bgp_peer_reset_idle() -- used when peer
                                 goes pIdle or is about to enable session.
                               - deleted bgp_peer_timers_stop() -- see
                                 bgp_peer_change_status().
                               - replaced bgp_peer_clearing_completed()
                                  - if pClearing, sets pIdle and enables if can
                                  - if pDeleting, unlocks peer
                               - replaced bgp_timer_set() by
                                 bgp_peer_timers_set() -- deals in new peer
                                 states only.
                               - renamed peer_new() -> bgp_peer_new()
                               - renamed peer_create() -> bgp_peer_create()
                                  - added setting of PEER_STATUS_REAL_PEER
                                  - changed auto activation to reflect what
                                    actually happens.
                                  - changed bgp_session_init_new() call because
                                    it now sets peer->session.
                                  - sets timers suitable for pIdle. before any
                                    auto enable.
                               - renamed peer_delete() -> bgp_peer_delete()
                                  - removed call of peer_nsf_stop()
                                  - added bgp_peer_down() with
                                    PEER_DOWN_NEIGHBOR_DELETE -- which does all
                                    the work of flattening an active peer, and
                                    returns it pIdle or pClearing.
                                  - if pClearing, lock the peer so that when
                                    clearing completes, it can unlock it.
                                  - sets pDeleting state, and increments
                                    bm->peer_linger_count.
                                  - tightened procedure for dealing with various
                                    references to peer -- including use of the
                                    PEER_STATUS_REAL_PEER flag.
                                  - tidied up dealing with rsclient RIBs and
                                    shared pointers to group versions of same.
                                  - removed call of bgp_timer_set(), now done
                                    in bgp_peer_change_status().
                                  - now unregisters the peer immediately, so
                                    can register a new one before this one
                                    is completely deleted.
                                  - deletes session if it can.
                               - moved peer_lock() & peer_unlock() from bgpd.c
                                 and renamed bgp_peer_lock() & bgp_peer_unlock()
                               - renamed peer_free() bgp_peer_free() and made
                                 static.
                                  - peer must be pDeleting -- so have been
                                    through bgp_peer_delete()
                                  - peer->session must be NULL
                                  - decrements bm->peer_linger_count
                                  - deleted peer->clear_node_queue handling
                                  - deleted bgp_session_free() -- that's done
                                    in bgp_peer_delete() or elsewhere.
                                  - unlocked bgp at end
                                  - assert peer->session == NULL, to be sure
                                  - set peer->lock == -54321
                               - in bgp_peer_enable():
                                  - recast as switch() on peer state
                                  - added bgp_peer_reset)idle() before enabling
                                    the session.
                               - renamed bgp_peer_disable() -> bgp_peer_down()
                                  - takes PEER_DOWN_XXX argument, which drives
                                    what notification is sent, and sets the
                                    peer->last_reset status.  A small number of
                                    PEER_DOWN_XXX are special.
                                  - removed the IdleHoldTimer stuff.
                                  - copies outbound notification to session.
                                  - for PEER_DOWN_NSF_CLOSE_SESSION, keep non-
                                    stale routes.
                                  - for PEER_DOWN_USER_SHUTDOWN, do
                                                             bgp_peer_shutdown()
                                  - after disabling any session and doing any
                                    shutdown, proceed as per peer->status:
                                       pIdle        -- flush stale routes
                                                       bgp_peer_enable()
                                       pEstablished -- bgp_peer_stop()
                                       pClearing    -- flush stale routes
                               - added bgp_peer_down_notify().
                               - added bgp_peer_down_error(), which replaces
                                 bgp_notify_send().
                               - added bgp_peer_down_error_with_data(), which
                                 replaces bgp_notify_send_with_data()
                                 The "down_error" functions calculate the
                                 appropriate PEER_DOWN_XXX value, and call
                                 bgp_peer_down_notify().
                               - added bgp_peer_map_peer_down(), to map
                                 PEER_DOWN_XXX to a notification message.
                               - added bgp_peer_map_notification, to map
                                 notification message to a PEER_DOWN_XXX.
                               - renamed peer_change_status() ->
                                                       bgp_peer_change_status()
                                  - do most things only if state changes.
                                  - add call to bgp_peer_reset_idle() as enter
                                    pIdle state.
                                  - at all times do bgp_peer_timer_set()
                               - renamed peer_timers_set() ->
                                                           bgp_peer_timers_set()
                               - commoned up code for Graceful Restart and
                                 Graceful Restart Stale timers and stale routes.
                               - changed Graceful Restart Stale time to cope if
                                 it should expire before Graceful Restart !

  bgpd/bgp_peer.h             -- added PEER_DOWN_XXX values and tidied up
                                  + PEER_DOWN_NULL
                                  + PEER_DOWN_UNSPECIFIED
                                  + PEER_DOWN_CONFIG_CHANGE
                                  + PEER_DOWN_AF_DEACTIVATE
                                  + PEER_DOWN_PASSWORD_CHANGE
                                  + PEER_DOWN_ALLOWAS_IN_CHANGE
                                  + PEER_DOWN_INTERFACE_DOWN
                                  + PEER_DOWN_MAX_PREFIX
                                  + PEER_DOWN_HEADER_ERROR
                                  + PEER_DOWN_OPEN_ERROR
                                  + PEER_DOWN_UPDATE_ERROR
                                  + PEER_DOWN_HOLD_TIMER
                                  + PEER_DOWN_FSM_ERROR
                                  + PEER_DOWN_DYN_CAP_ERROR
                                  - PEER_DOWN_NOTIFY_SEND  (deleted)
                               - added typedef peer_down_t
                               - struct peer:
                                  - deleted redundant clear_node_queue
                                  - removed PEER_STATUS_ACCEPT_PEER flag
                                  - added PEER_STATUS_REAL_PEER flag
                               - (3)
                               - deleted bgp_peer_reenable() -- redundant
                               - deleted bgp_peer_stop() -- now static
                               - replaced bgp_peer_disable() by bgp_peer_down()
                               - added bgp_peer_down_error()
                               - added bgp_peer_down_error_with_data()
                               - deleted peer_change_status() -- now static
                               - renamed peer_new() -> bgp_peer_new()
                               - renamed peer_create() -> bgp_peer_create()
                               - renamed peer_delete() -> bgp_peer_delete()
                               - added bgp_peer_lock()
                               - added bgp_peer_unlock()
                               - deleted peer_free()
                               - deleted peer_nsf_stop()

  bgpd/bgp_peer_index.c       -- removed accept entry from bgp_peer_index_entry
                                 structure
                               - added explicit next_free entry to the structure
                               - sets next_free to point at self in entries
                                 which are in use -- and checks this.
                               - change bgp_peer_index_seek_accept() to link to
                                 session via the peer data structure, and to
                                 call bgp_connection_query_accept() under the
                                 Peer Index Mutex.
                               - (4) for bgp_peer_index_seek_accept()

  bgpd/bgp_peer_index.h       -- removed accept entry from bgp_peer_index_entry
                                 structure
                               - added explicit next_free entry to the structure
                               - (4) for bgp_peer_index_seek_accept()

  bgpd/bgp_route.c            -- (1) (2) (3)
                               - in bgp_process_rsclient(), bgp_process_main(),
                                 and bgp_processq_del():
                                  - extra dasserts()
                                  - clear rn->wq_next
                                  - unlock table *after* unlock node (bug fix)
                               - in bgp_process(), lock bgp before table.
                               - in bgp_maximum_prefix_restart_timer(), replace
                                 call of peer_clear() by unset of flag and
                                 bgp_peer_enable() -- peer is already down.
                               - added bgp_maximum_prefix_cancel_timer()
                               - deleted bgp_clear_this_route() -- code now
                                 inline in only caller.
                               - renamed bgp_clear_route_normal() ->
                                                              bgp_clear_routes()
                                  - takes an "nsf" argument to invoke NSF
                                    "clearing", iff nsf set for afi/safi.  Sets
                                    PEER_STATUS_NSF_WAIT if so.
                                  - returns bool "completed" if clearing has
                                    completed immediately -- ie no background
                                    work left to be done.
                               - renamed bgp_clear_route_all() ->
                                                          bgp_clear_all_routes()
                                  - takes "nsf" argument and returns "completed"
                                    as for bgp_clear_routes().
                                  - removed call: bgp_peer_clearing_completed()
                               - renamed bgp_clear_route_rsclient() ->
                                                        bgp_clear_rsclient_rib()
                               - deleted bgp_cleanup_routes() -- was used
                                 during termination, no longer required because
                                 termination deletes all peers.

  bgpd/bgp_route.h            -- deleted bgp_cleanup_routes() -- program
                                 termination now deletes all peers, which
                                 implicitly cleans up all routes.
                               - renamed: bgp_clear_route_normal() ->
                                                        bgp_clear_routes()
                               - renamed: bgp_clear_route_rsclient() ->
                                                        bgp_clear_rsclient_rib()
                               - renamed: bgp_clear_route_all() ->
                                                        bgp_clear_all_routes()
                               - added: bgp_maximum_prefix_cancel_timer()

  bgpd/bgp_session.c          -- (3)
                               - deleted bgp_session_defer_if_limping()
                               - in bgp_session_init_new()
                                  - changed to void and removed session argument
                                    -- always creates a new session
                                  - peer MUST not have a session
                                  - removed Peer Index pointer stuff as Peer
                                    Index no longer has accept field
                                  - sets session->peer and locks peer
                                  - sets peer->session under Peer Index Mutex
                                  - sets session->delete_me false
                                  - sets session->accept flag false
                               - replaced bgp_session_free() by
                                                            bgp_session_delete()
                                  - changed to void function
                                  - if session is active, set the delete_me flag
                                    so session will be deleted when goes
                                    sDisabled.
                                  - make sure that session Mutex has been
                                    released by the BGP Engine before destroying
                                    it... otherwise: tears.
                                  - unhook session from peer under Peer Index
                                    mutex -- for accept() stuff.
                                  - unhook peer from session.
                                  - unlock peer.
                               - in bgp_session_enable()
                                  - assert that peer is pIdle.
                                  - clear delete_me for completeness
                                  - clear additional fields
                               - in bgp_session_disable()
                                  - clear session->accept
                               - in bgp_session_is_active()
                                  - no longer interested in Peer Index stuff
                               - deleted bgp_session_defer_if_limping()

  bgpd/bgp_session.h          -- in bgp_session structure:
                                  - removed index_entry  pointer to Peer Index
                                  - added delete_me flag
                                  - removed defer_enable flag
                                  - added accept flag
                               - removed session parameter from
                                                         bgp_session_init_new()
                               - deleted bgp_session_free()
                               - added bgp_session_delete()
                               - bgp_session_is_active() now returns bool

  bgpd/bgp_table.c            -- bgp_node_free() sets lock count = -54321
                               - in bgp_table_free():
                                  - assert that route node is empty
                                  - set lock count = -54321
                               - bgp_node_delete() asserts that is not on_wq
                               - (1)

  bgpd/bgp_vty.c              -- (1) (3) (4)
                               - change peer_af_flag_modify_vty() to call
                                 peer_af_flag_modify() not set or unset.
                               - change name: bgp_clear_route_rsclient() ->
                                                        bgp_clear_rsclient_rib()
                               - in peer_rsclient_set_vty():
                                  - add peer to bgp->rsclient list after all
                                    validation is complete
                               - in peer_rsclient_unset_vty():
                                  - removed code for deleting the rsclient RIB
                                    etc to peer_rsclient_unset().
                               - added peer_rsclient_unset()

  bgpd/bgp_zebra.c            -- bgp_peer_disable() -> bgp_peer_down() and now
                                 takes PEER_DOWN_INTERFACE_DOWN argument.

  bgpd/bgpd.c                 -- (1) (3)
                               - replaced setting peer->last_reset and call of
                                 bgp_notify_send() by call of the new
                                 bgp_peer_down().
                                  - bgp_router_id_set()
                                     ... PEER_DOWN_RID_CHANGE
                                  - bgp_cluster_id_set()
                                     ... PEER_DOWN_CLID_CHANGE
                                  - bgp_cluster_id_unset()
                                     ... PEER_DOWN_CLID_CHANGE
                                  - bgp_confederation_id_set()
                                     ... PEER_DOWN_CONFED_ID_CHANGE
                                     ... PEER_DOWN_CONFED_ID_CHANGE
                                  - bgp_confederation_id_unset()
                                     ... PEER_DOWN_CONFED_ID_CHANGE
                                  - bgp_confederation_peers_add()
                                     ... PEER_DOWN_CONFED_PEER_CHANGE
                                  - bgp_confederation_peers_remove()
                                     ... PEER_DOWN_CONFED_PEER_CHANGE
                                  - peer_as_change()
                                     ... PEER_DOWN_REMOTE_AS_CHANGE
                                  - peer_activate()
                                     ... PEER_DOWN_AF_ACTIVATE
                                  - peer_deactivate()
                                     ... PEER_DOWN_AF_DEACTIVATE
                                  - peer_group_bind()
                                     ... PEER_DOWN_RMAP_BIND
                                  - peer_group_unbind()
                                     ... PEER_DOWN_RMAP_UNBIND
                                  - peer_change_action()
                                     ... why_changed
                                     ... why_changed
                                  - peer_flag_modify_action()
                                     ... action->peer_down
                                  - peer_update_source_if_set()
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                  - peer_update_source_addr_set()
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                  - peer_update_source_unset()
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                     ... PEER_DOWN_UPDATE_SOURCE_CHANGE
                                  - peer_local_as_set()
                                     ... PEER_DOWN_LOCAL_AS_CHANGE
                                     ... PEER_DOWN_LOCAL_AS_CHANGE
                                  - peer_local_as_unset()
                                     ... PEER_DOWN_LOCAL_AS_CHANGE
                                     ... PEER_DOWN_LOCAL_AS_CHANGE
                                  - peer_password_set()
                                     ... PEER_DOWN_PASSWORD_CHANGE
                                     ... PEER_DOWN_PASSWORD_CHANGE
                                  - peer_password_unset()
                                     ... PEER_DOWN_PASSWORD_CHANGE
                                     ... PEER_DOWN_PASSWORD_CHANGE
                                  - peer_clear()
                                     ... PEER_DOWN_USER_RESET
                                  - bgp_terminate()
                                     ... PEER_DOWN_USER_RESET
                               - deleted peer_lock() & peer_unlock().  See
                                 bgp_peer_lock() & bgp_peer_unlock() in bgp_peer
                               - in peer_as_change(), move downing of peer to
                                 after all config changes have been made.
                               - in peer_remote_as() implicitly activate iff
                                 !BGP_FLAG_NO_DEFAULT and is IPv4/Unicast.
                                 (but only ever called with IPv4/Unicast or
                                  nothing at all.)
                               - in peer_deactivate()
                                  - if cannot dynamically reconfigure, then will
                                    down the peer PEER_DOWN_AF_DEACTIVATE.
                                  - uses new peer_rsclient_unset() to tidy away
                                    any rsclient RIB etc.
                               - in peer_change_action():
                                  - added 'why_down' argument
                                  - replace if's by switch()
                               - in struct peer_flag_action, updated entry types
                               - in peer_flag_action_list[], added the
                                 appropriate PEER_DOWN_XXX values.
                               - in peer_af_flag_action_list[]
                                  - added the appropriate PEER_DOWN_XXX values
                                  - added multiple flag entries
                               - in peer_flag_action_set():
                                  - changed to return const address of
                                    peer_flag_action structure -- or NULL.
                                  - table may now contain entries which the
                                    given flag must be a subset of.
                               - in peer_flag_modify_action():
                                  - now takes peer_flag_action* and whether flag
                                    has been set or not.
                                  - allow *only* peer_change_none or
                                    peer_change_reset
                                  - deal with clearing PEER_FLAG_SHUTDOWN,
                                    otherwise bgp_peer_down().
                               - in peer_group_bind():
                                  - uses new peer_rsclient_unset() to tidy away
                                    any rsclient RIB etc.
                               - in peer_flag_modify():
                                  - takes bool set flag
                                  - changed to suit peer_flag_action_set() and
                                                       peer_flag_modify_action()
                               - in peer_flag_set() and peer_flag_unset()
                                 changed to bool flag
                               - added peer_af_flag_modify_action(), common
                                 code for use in peer_af_flag_modify().
                               - in peer_af_flag_modify():
                                  - takes bool set flag
                                  - changed to suit peer_flag_action_set() and
                                                       peer_flag_modify_action()
                                  - use peer_af_flag_modify_action()
                               - in peer_af_flag_set() and peer_af_flag_unset()
                                 changed to bool flag
                               - in peer_clear(): adjust for new bgp_peer_down()
                                 mechanics.
                               - in bgp_master_init(): account for
                                 peer_linger_count (starting at 0)
                               - in bgp_terminate():
                                  - removed program_terminating -- see flag now
                                    in bgp_main.c
                                  - implement "retain_mode" by using
                                    BGP_OPT_NO_FIB flag to turn off changing
                                    the FIB as routes are deleted.
                                  - either bgp_peer_delete() if terminating
                                    or bgp_peer_down() all peers.
                                  - flush process queues.
                               - deleted program_terminate_if_all_disabled()
                               - in peer_lookup(), removed handling of
                                 PEER_STATUS_ACCEPT_PEER().
                               - deleted peer_lookup_with_open().
                               - in bgp_config_write_family(), removed handling
                                 of PEER_STATUS_ACCEPT_PEER().
                               - in bgp_config_write(), removed handling
                                 of PEER_STATUS_ACCEPT_PEER().

  bgpd/bgpd.h                 -- add peer_linger_count entry to the
                                 bgp_master structure.
                               - remove: peer_lock(), peer_unlock() and
                                 peer_delete()
                               - (4) for peer_af_flag_modify()
                               - added peer_rsclient_unset()

  lib/plist.c                 -- fixed handling of prefix lists with no
                                 explicit sequence numbers.

  lib/qpnexus.c               -- (4) for main_thread & terminate flags
                               - change qpn_terminate() so does nothing if
                                 terminate flag is already set.

  lib/qpnexus.h               -- (4) for main_thread & terminate flags

  tests/bgp_capability_test.c -- (3)

1 files changed, 147 insertions, 82 deletions

diff --git a/bgpd/bgp_session.c b/bgpd/bgp_session.c
index 8d8bfea2..9de8678a 100644
--- a/bgpd/bgp_session.c
+++ b/bgpd/bgp_session.c
@@ -40,7 +40,6 @@
 #include "lib/zassert.h"
 
 /* prototypes */
-static int bgp_session_defer_if_limping(bgp_session session);
 static void bgp_session_do_enable(mqueue_block mqb, mqb_flag_t flag) ;
 static void bgp_session_do_update_recv(mqueue_block mqb, mqb_flag_t flag);
 static void bgp_session_do_update_send(mqueue_block mqb, mqb_flag_t flag);
@@ -99,7 +98,7 @@ static void bgp_session_do_route_refresh_recv(mqueue_block mqb, mqb_flag_t flag)
  */
 
 /*------------------------------------------------------------------------------
- * Initialise new session structure -- allocate if required.
+ * Allocate & initialise new session structure.
  *
  * Ties peer and session together.  Sets session sIdle, initialises mutex.
  *
@@ -108,30 +107,34 @@ static void bgp_session_do_route_refresh_recv(mqueue_block mqb, mqb_flag_t flag)
  * NB: if not allocating, the existing session MUST be sIdle/sDisabled OR never
  *     been kissed.
  *
- * NB: in any event, the peer's peer index entry MUST have a NULL accept
- *     pointer.
+ * NB: peer MUST NOT have a session set up:
+ *
+ *      (a) because if there was a session, there would have to be code here
+ *          to worry about its state, and tearing it down etc.
+ *
+ *      (b) so that do not have to worry about BGP Engine reaching the old
+ *          session while it was being replaced or whatever.
  */
 extern bgp_session
-bgp_session_init_new(bgp_session session, bgp_peer peer)
+bgp_session_init_new(bgp_peer peer)
 {
+  bgp_session session ;
+
   assert(peer->session == NULL) ;
-  assert(peer->index_entry->accept == NULL) ;
 
-  if (session == NULL)
-    session = XCALLOC(MTYPE_BGP_SESSION, sizeof(struct bgp_session)) ;
-  else
-    memset(session, 0, sizeof(struct bgp_session)) ;
+  session = XCALLOC(MTYPE_BGP_SESSION, sizeof(struct bgp_session)) ;
 
   qpt_mutex_init_new(&session->mutex, qpt_mutex_recursive) ;
 
-  peer->session  = session ;
   session->peer  = peer ;
-  session->state = bgp_session_sIdle ;
+  bgp_peer_lock(peer) ;             /* Account for the session->peer pointer  */
 
-  session->index_entry = peer->index_entry ;
+  session->state = bgp_session_sIdle ;
 
   /* Zeroising the structure has set:
    *
+   *   delete_me      -- 0    -- false
+   *
    *   event          -- bgp_session_null_event
    *   notification   -- NULL -- none
    *   err            -- 0    -- none
@@ -173,25 +176,73 @@ bgp_session_init_new(bgp_session session, bgp_peer peer)
    *
    *   connections[]  -- NULL -- none
    *   active         -- false, not yet active
+   *   accept         -- false, not yet ready to accept()
    */
   confirm(bgp_session_null_event == 0) ;
 
+  /* Once the session is fully initialised, can set peer->session pointer.
+   *
+   * NB: this is done last and under the Peer Index Mutex, so that the
+   *     accept() code does not trip over.
+   */
+  bgp_peer_index_set_session(peer, session) ;
+
   return session ;
 } ;
 
-/* Free session structure
+/*==============================================================================
+ * Routing Engine: delete session for given peer.
+ *
+ * This is for use when the peer itself is being deleted.  (Peer MUST be in
+ * pDeleting state.)
+ *
+ * Does nothing if there is no session !
+ *
+ * If the session is active, simply sets delete_me flag, which will be honoured
+ * when the session goes dDisabled.  Note, it is the callers responsibility
+ * to arrange for that to happen.
+ *
+ * If the session is not active, it is immediately freed.
  *
+ * NB: if the session is freed, the peer may vanish at the same time !
  */
-extern bgp_session
-bgp_session_free(bgp_session session)
+extern void
+bgp_session_delete(bgp_peer peer)
 {
+  bgp_session session = peer->session ;
+
   if (session == NULL)
-    return NULL;
+    return ;                    /* easy if no session anyway    */
+
+  assert(peer == session->peer) ;
+
+  /* If is active, set flag so that session is deleted when next it becomes
+   * sDisabled.
+   */
+  if (bgp_session_is_active(session))
+    {
+      session->delete_me = true ;
+      return ;
+    } ;
+
+  /*--------------------------------------------------------------------------*/
+  /* Proceed to free the session structure.                                   */
+
+  /* Make sure that the BGP Engine has, in fact, let go of the session.
+   *
+   * The LOCK/UNLOCK makes sure that the BGP Engine has unlocked the session.
+   *
+   * Without this, the qpt_mutex_destroy() can fail horribly, if the BGP
+   * Engine sends the disable acknowledge before finally unlocking the session.
+   */
+  BGP_SESSION_LOCK(session) ;   /*<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<*/
 
-  assert(!bgp_session_is_active(session)) ;
+  BGP_SESSION_UNLOCK(session) ; /*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>*/
 
   qpt_mutex_destroy(&session->mutex, 0) ;
 
+  /* Proceed to dismantle the session.                                  */
+
   bgp_notify_unset(&session->notification);
   bgp_open_state_free(session->open_send);
   bgp_open_state_free(session->open_recv);
@@ -206,12 +257,24 @@ bgp_session_free(bgp_session session)
   sockunion_unset(&session->su_local) ;
   sockunion_unset(&session->su_remote) ;
 
+  /* Drop the peer->session and session->peer pointers
+   *
+   * NB: the peer->session pointer is cleared under the Peer Index Mutex,
+   *     so that the accept() code does not trip over.
+   *
+   * NB: at this point it is possible for the peer structure to suddenly
+   *     vanish -- if peer has been deleted, and has been waiting for the
+   *     session to go sDisabled.
+   */
+  bgp_peer_index_set_session(peer, NULL) ;
+
+  session->peer = NULL ;
+  bgp_peer_unlock(peer) ;       /* NB: peer->session == NULL    */
+
   /* Zeroize to catch dangling references asap */
   memset(session, 0, sizeof(struct bgp_session)) ;
   XFREE(MTYPE_BGP_SESSION, session);
-
-  return NULL;
-}
+} ;
 
 /*==============================================================================
  * Routing Engine: enable session for given peer -- allocate if required.
@@ -226,6 +289,8 @@ bgp_session_enable(bgp_peer peer)
   bgp_session    session ;
   mqueue_block   mqb ;
 
+  assert(peer->state = bgp_peer_pIdle) ;
+
   /* Set up session if required.  Check session if already exists.
    *
    * Only the Routing Engine creates sessions, so it is safe to pick up the
@@ -238,27 +303,24 @@ bgp_session_enable(bgp_peer peer)
   session = peer->session ;
 
   if (session == NULL)
-    session = bgp_session_init_new(NULL, peer) ;
+    session = bgp_session_init_new(peer) ;
   else
     {
       assert(session->peer == peer) ;
-      /* if session is limping then defer the enable */
-      if (bgp_session_defer_if_limping(session))
-        return;
       assert(!bgp_session_is_active(session)) ;
     } ;
 
   /* Initialise what we need to make and run connections                */
-  session->state    = bgp_session_sIdle;
-  session->defer_enable = 0;
-  session->flow_control = 0;
-  session->event    = bgp_session_null_event;
-  bgp_notify_unset(&session->notification);
-  session->err      = 0;
-  session->ordinal  = 0;
+  session->state        = bgp_session_sIdle ;
+  session->delete_me    = false ;
+  session->flow_control = 0 ;
+  session->event        = bgp_session_null_event ;
+  bgp_notify_unset(&session->notification) ;
+  session->err          = 0 ;
+  session->ordinal      = 0 ;
 
-  session->open_send = bgp_peer_open_state_init_new(session->open_send, peer);
-  bgp_open_state_unset(&session->open_recv);
+  session->open_send = bgp_peer_open_state_init_new(session->open_send, peer) ;
+  bgp_open_state_unset(&session->open_recv) ;
 
   session->connect  = (peer->flags & PEER_FLAG_PASSIVE) == 0 ;
   session->listen   = 1 ;
@@ -306,12 +368,22 @@ bgp_session_enable(bgp_peer peer)
   session->hold_timer_interval          = peer->v_holdtime ;
   session->keepalive_timer_interval     = peer->v_keepalive ;
 
-  session->as4       = 0;
-  session->route_refresh_pre = 0;
+  session->as4               = false ;
+  session->route_refresh_pre = false ;
+  session->orf_prefix_pre    = false ;
 
   /* su_local set when session Established */
   /* su_remote  set when session Established */
 
+  /* TODO: check whether session stats should persist   */
+  memset(&session->stats, 0, sizeof(struct bgp_session_stats)) ;
+
+  memset(&session->connections, 0,
+                                sizeof(bgp_connection) * bgp_connection_count) ;
+
+  session->active    = false ;
+  session->accept    = false ;
+
   /* Routeing Engine does the state change now.                         */
 
   /* Now pass the session to the BGP Engine, which will set about       */
@@ -340,7 +412,7 @@ bgp_session_do_enable(mqueue_block mqb, mqb_flag_t flag)
 
       BGP_SESSION_LOCK(session) ;   /*<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<*/
 
-      session->active = 1 ;
+      session->active = true ;
       bgp_fsm_enable_session(session) ;
 
       BGP_SESSION_UNLOCK(session) ; /*>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>*/
@@ -352,6 +424,8 @@ bgp_session_do_enable(mqueue_block mqb, mqb_flag_t flag)
 /*==============================================================================
  * Routing Engine: disable session for given peer -- if enabled (!).
  *
+ * Does nothing if the session is not sEnabled or sEstablished.
+ *
  * Passes any bgp_notify to the BGP Engine, which will dispose of it in due
  * course.
  *
@@ -390,7 +464,6 @@ bgp_session_disable(bgp_peer peer, bgp_notify notification)
 
   /* Now change to limping state                                        */
   session->state = bgp_session_sLimping;
-  session->defer_enable = 0;
 
   /* Ask the BGP engine to disable the session.
    *
@@ -470,7 +543,10 @@ bgp_session_event(bgp_session session, bgp_session_event_t  event,
   mqueue_block   mqb ;
 
   if (stopped)
-    session->active  = 0 ;              /* ignore updates etc           */
+    {
+      session->active  = false ;        /* ignore updates etc           */
+      session->accept  = false ;        /* for completeness             */
+    } ;
 
   mqb = mqb_init_new(NULL, bgp_session_do_event, session) ;
 
@@ -774,6 +850,8 @@ bgp_session_update_recv(bgp_session session, struct stream* buf, bgp_size_t size
 
 /*------------------------------------------------------------------------------
  * Routing Engine: process incoming update message -- mqb action function.
+ *
+ * Discard the update if the session is not sEstablished.
  */
 static void
 bgp_session_do_update_recv(mqueue_block mqb, mqb_flag_t flag)
@@ -781,10 +859,9 @@ bgp_session_do_update_recv(mqueue_block mqb, mqb_flag_t flag)
   bgp_session session = mqb_get_arg0(mqb) ;
   struct bgp_session_update_args* args = mqb_get_args(mqb) ;
 
-  if ((flag == mqb_action) && (session->state == bgp_session_sEstablished))
+  if ( (flag == mqb_action) && (session->state == bgp_session_sEstablished) )
     {
-      bgp_peer peer = session->peer;
-
+      bgp_peer  peer = session->peer;
       stream_free(peer->ibuf);
       peer->ibuf = args->buf;
       bgp_update_receive (peer, args->size);
@@ -826,8 +903,8 @@ bgp_session_do_route_refresh_recv(mqueue_block mqb, mqb_flag_t flag)
   struct bgp_session_route_refresh_args* args = mqb_get_args(mqb) ;
   bgp_session session = mqb_get_arg0(mqb) ;
 
-  if ((flag == mqb_action) && (session->state == bgp_session_sEstablished))
-    bgp_route_refresh_recv(session->peer, args->rr);
+  if ( (flag == mqb_action) && (session->state == bgp_session_sEstablished) )
+    bgp_route_refresh_recv(session->peer, args->rr) ;
 
   bgp_route_refresh_free(args->rr);
   mqb_free(mqb);
@@ -860,9 +937,10 @@ bgp_session_do_XON(mqueue_block mqb, mqb_flag_t flag)
 {
   bgp_session session = mqb_get_arg0(mqb) ;
 
-  if ((flag == mqb_action) && (session->state == bgp_session_sEstablished))
+  if ( (flag == mqb_action) && (session->state == bgp_session_sEstablished) )
     {
       int xoff = (session->flow_control <= 0);
+
       session->flow_control = BGP_XON_REFRESH;
       if (xoff)
         bgp_write (session->peer, NULL) ;
@@ -923,59 +1001,46 @@ bgp_session_do_set_ttl(mqueue_block mqb, mqb_flag_t flag)
  */
 
 /*------------------------------------------------------------------------------
- * See if session exists and is active.
+ * Routing Engine: see if session exists and is active.
  *
- * Ensure that if exists and is not active, that the peer index entry accept
- * pointer is NULL -- this is largely paranoia, but it would be a grave
- * mistake for the listening socket(s) to find a session which is not active !
+ * If exists then performs a few checks, just to make sure things are straight.
  *
- * NB: accessing Routing Engine "private" variable -- no lock required.
+ * NB: accessing Routing Engine "private" variable  -- no lock required.
  *
- *     accessing index_entry when not active -- no lock required.
+ *     checks session->active, only when not active -- no lock required.
  */
-extern int
+extern bool
 bgp_session_is_active(bgp_session session)
 {
-  int active ;
+  bool active ;
 
   if (session == NULL)
-    active = 0 ;
+    active = false ;
   else
     {
-      active =  (   (session->state == bgp_session_sEnabled)
-                 || (session->state == bgp_session_sEstablished)
-                 || (session->state == bgp_session_sLimping) ) ;
-
-      if (!active)
-        assert(session->index_entry->accept == NULL) ;
+      switch (session->state)
+      {
+        case bgp_session_sIdle:
+        case bgp_session_sDisabled:
+          assert(!session->active) ;
+          active = false ;
+          break ;
+
+        case bgp_session_sEnabled:
+        case bgp_session_sEstablished:
+        case bgp_session_sLimping:
+          active = true ;
+          break ;
+
+        default:
+          zabort("invalid session->state") ;
+      } ;
     } ;
 
   return active ;
 } ;
 
 /*------------------------------------------------------------------------------
- * Routing Engine: if session is limping we defer re-enabling the session
- *                 until it is disabled.
- *
- * returns 1 if limping and defer
- * returns 0 if not limping
- *
- * NB: accessing Routing Engine "private" variable -- no lock required.
- */
-static int
-bgp_session_defer_if_limping(bgp_session session)
-{
-  int defer_enable = 0 ;
-
-  if (session == NULL)
-    defer_enable = 0 ;
-  else
-    defer_enable = (session->state == bgp_session_sLimping) ;
-
-  return session->defer_enable = defer_enable ;
-} ;
-
-/*------------------------------------------------------------------------------
  * Get a copy of the session statistics, copied all at once so
  * forms a consistent snapshot
  */