C9k FNF Not collecting source / destination interface on L3 Interface

bturner · ‎09-04-2019

After upgrading to Catalyst 9500 core / distribution switches, we were excited to get FNF into the management tool.

Initially on 16.11.1 code, we noticed that all interfaces in the FNF tables were listed as Null, which the management tool LiveAction detects as traffic to / from Null0 interface. After a TAC case we were told that 16.12.1 resolves this bug.

CSCvm33593 - Interface input/output values show as "Null" in flow cache
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvm33593/?reffering_site=dumpcr

CSCvo09803 - cat9500 interface input/output values shows as "Null" in the flow cache
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvo09803/?reffering_site=dumpcr

Now on 16.12.1, we at least see traffic flowing in / out of interfaces, but all of that traffic is destined to or coming from interface Null0.

The configuration matches on output interface, and collects input interface, yet the switch is not collecting the interface on the flows as you can see below :

CORE1-9532QC#show flow monitor LIVEACTION-FLOWMONITOR-EGRESS cache
  Cache type:                               Normal (Platform cache)
  Cache size:                                10000
  Current entries:                           16232

  Flows added:                           155034484
  Flows aged:                            155018252
    - Active timeout      (    60 secs)    8847467
    - Inactive timeout    (    10 secs)  146170785

IPV4 SRC ADDR    IPV4 DST ADDR    TRNS SRC PORT  TRNS DST PORT  INTF OUTPUT           FLOW DIRN  IP TOS  IP PROT  tcp flags  intf input                      bytes long             pkts long  time abs first  time abs last     bytes layer2 long
===============  ===============  =============  =============  ====================  =========  ======  =======  =========  ====================  ====================  ====================  ==============  =============  ====================
10.1.2.3         198.18.1.1                 123            123  Po1                   Output     0xC0         17  0x00       Null                                   304                     4    07:58:19.657   07:58:25.657                   376
10.1.2.3         198.18.1.1                 123            123  Po1                   Output     0x00         17  0x00       Null                                    76                     1    07:58:29.657   07:58:29.657                    94
10.1.2.3         198.18.1.1                   0              0  Po1                   Output     0xC0          1  0x00       Null                                   952                    17    07:58:03.657   07:58:35.657                  1258
10.1.2.3         198.18.1.1                   0              0  Po1                   Output     0x00         47  0x00       Null                              35441800                124619    07:57:37.657   07:58:36.657              37684942
10.1.2.3         198.18.1.1               50107            443  Fo1/0/15              Output     0x00          6  0x10       Null                                    50                     1    07:58:27.657   07:58:27.657                    68
10.1.2.3         198.18.1.1                 443          54979  Fo1/0/3               Output     0x00          6  0x18       Null                                    83                     1    07:58:36.657   07:58:36.657                   101
10.1.2.3         198.18.1.1               51807            443  Fo1/0/15              Output     0x00          6  0x11       Null                                   100                     2    07:58:35.657   07:58:35.657                   136
10.1.2.3         198.18.1.1                 554          56525  Fo1/0/3               Output     0x00          6  0x18       Null                               8982014                  6111    07:58:08.657   07:58:36.657               9092012

Our netflow record configuration is as follows :

flow record LIVEACTION-FLOWRECORD-INGRESS
 description DO NOT MODIFY. USED BY LIVEACTION.
 match flow direction
 match interface input
 match ipv4 destination address
 match ipv4 protocol
 match ipv4 source address
 match ipv4 tos
 match transport destination-port
 match transport source-port
 collect counter bytes layer2 long
 collect counter bytes long
 collect counter packets long
 collect interface output
 collect timestamp absolute first
 collect timestamp absolute last
 collect transport tcp flags
flow record LIVEACTION-FLOWRECORD-EGRESS
 description DO NOT MODIFY. USED BY LIVEACTION.
 match flow direction
 match interface output
 match ipv4 destination address
 match ipv4 protocol
 match ipv4 source address
 match ipv4 tos
 match transport destination-port
 match transport source-port
 collect counter bytes layer2 long
 collect counter bytes long
 collect counter packets long
 collect interface input
 collect timestamp absolute first
 collect timestamp absolute last
 collect transport tcp flags

The problem is two fold. It is difficult to stitch together the flows if you do not know source / destination interface, and the management software is interpreting Null as Null0. If there is no plan to fix this then the configuration to collect the non indexed interface for source or destination should leave the field blank or reference something other than the well known interface name of Null.

Ideally some future version of software will resolve this issue as well.

Has anyone else run into this? Are the developers aware of the issue? FNF is not very helpful if it cannot provide the basic function of source / destination interface.

The resulting display of the flows looks like everything is talking to Null0

Brian S. Turner
CCIE 6145