Sunday, March 27, 2011

CSCtk46796 - N5K NX-OS 5.0(2)N1(1) ICMP Echo Request Packets Greater than the Default Ethernet MTU 1500 Bytes to SVIs Not Being Fragmented

When performing verification upon the implementation of Jumbo Frame support on Nexus 5000 switches, I discovered the following problem.

Network Setup

Firstly, there was no problem with NX-OS 4.2(1)N2(1):
n5010-sw1# sh ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
  BIOS:      version 1.3.0
  loader:    version N/A
  kickstart: version 4.2(1)N2(1)
  system:    version 4.2(1)N2(1)
  power-seq: version v1.2
  BIOS compile time:       09/08/09
  kickstart image file is: bootflash:/n5000-uk9-kickstart.4.2.1.N2.1.bin
  kickstart compile time:  7/28/2010 18:00:00 [07/29/2010 01:10:19]
  system image file is:    bootflash:/n5000-uk9.4.2.1.N2.1.bin
  system compile time:     7/28/2010 18:00:00 [07/29/2010 05:18:12]


Hardware
  cisco Nexus5010 Chassis ("20x10GE/Supervisor")
  Intel(R) Celeron(R) M CPU    with 2074284 kB of memory.
  Processor Board ID JAF1429ERMA

  Device name: n5010-sw1
  bootflash:    1003520 kB

Kernel uptime is 51 day(s), 4 hour(s), 45 minute(s), 38 second(s)

Last reset at 125383 usecs after  Thu Oct  7 07:25:30 2010

  Reason: Reset Requested by CLI command reload
  System version: 4.2(1)N2(1)
  Service:

plugin
  Core Plugin, Ethernet Plugin, Fc Plugin
n5010-sw1#
n5010-sw1# ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56 data bytes
64 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=2.281 ms
64 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=1.11 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=1.006 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=1.018 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=1.019 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.006/1.286/2.281 ms
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 1472
PING 192.168.1.2 (192.168.1.2): 1472 data bytes
1480 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=1.741 ms
1480 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=1.124 ms
1480 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=1.1 ms
1480 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=1.058 ms
1480 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=1.192 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.058/1.243/1.741 ms
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 1473
PING 192.168.1.2 (192.168.1.2): 1473 data bytes
1481 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=1.32 ms
1481 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=0.907 ms
1481 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=0.9 ms
1481 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=1.052 ms
1481 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=3.502 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.9/1.536/3.502 ms
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 3000
PING 192.168.1.2 (192.168.1.2): 3000 data bytes
3008 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=2.031 ms
3008 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=1.474 ms
3008 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=1.388 ms
3008 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=1.397 ms
3008 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=1.473 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.388/1.552/2.031 ms
n5010-sw1#

Below shows the problem scenario observed on NX-OS 5.0(2)N1(1):
n5010-sw1# sh ver
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2010, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.

Software
  BIOS:      version 1.3.0
  loader:    version N/A
  kickstart: version 5.0(2)N1(1)
  system:    version 5.0(2)N1(1)
  power-seq: version v1.2
  BIOS compile time:       09/08/09
  kickstart image file is: bootflash:/n5000-uk9-kickstart.5.0.2.N1.1.bin
  kickstart compile time:  10/15/2010 0:00:00 [10/15/2010 08:00:43]
  system image file is:    bootflash:/n5000-uk9.5.0.2.N1.1.bin
  system compile time:     10/15/2010 0:00:00 [10/15/2010 09:34:05]


Hardware
  cisco Nexus5010 Chassis ("20x10GE/Supervisor")
  Intel(R) Celeron(R) M CPU    with 2074284 kB of memory.
  Processor Board ID JAF1429ERMA

  Device name: n5010-sw1
  bootflash:    1003520 kB

Kernel uptime is 0 day(s), 23 hour(s), 19 minute(s), 23 second(s)

Last reset at 296097 usecs after  Fri Nov 26 12:43:08 2010

  Reason: Disruptive upgrade
  System version: 4.2(1)N2(1)
  Service:

plugin
  Core Plugin, Ethernet Plugin, Fc Plugin
n5010-sw1#
n5010-sw1# ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2): 56 data bytes
64 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=7.09 ms
64 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=1.216 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=1.223 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=1.196 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=1.142 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.142/2.373/7.09 ms
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 1472
PING 192.168.1.2 (192.168.1.2): 1472 data bytes
1480 bytes from 192.168.1.2: icmp_seq=0 ttl=254 time=1.884 ms
1480 bytes from 192.168.1.2: icmp_seq=1 ttl=254 time=1.059 ms
1480 bytes from 192.168.1.2: icmp_seq=2 ttl=254 time=1.383 ms
1480 bytes from 192.168.1.2: icmp_seq=3 ttl=254 time=0.985 ms
1480 bytes from 192.168.1.2: icmp_seq=4 ttl=254 time=1.36 ms

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.985/1.334/1.884 ms
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 1473
PING 192.168.1.2 (192.168.1.2): 1473 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
Request 3 timed out
Request 4 timed out

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 0 packets received, 100.00% packet loss
n5010-sw1#
n5010-sw1# ping 192.168.1.2 packet-size 1500
PING 192.168.1.2 (192.168.1.2): 1500 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
Request 3 timed out
Request 4 timed out

--- 192.168.1.2 ping statistics ---
5 packets transmitted, 0 packets received, 100.00% packet loss
n5010-sw1#

Note: This problem has nothing to do with the Jumbo Frame configuration, as the ICMP Echo Request packets are sent without the DF (Don't Fragment) bit set, and therefore we are expecting fragmentation to occur.

Lastly, the Cisco TAC engineer filed a new bug for this. :-)

No comments:

Post a Comment