Whenever someone other than a network engineer plugs a cable into an Ethernet port, bad things can happen. A good set of network management tools becomes essential for network engineers who have to sort through someone else's mess.
When the performance of the Extreme Networks switches in his campus network degraded recently, network engineer Brian Saunier looked at his network monitoring tool, Ipswitch’s WhatsUp Gold, to see what was happening.
"When we started looking at it, we could see that the CPU usage was getting high on certain ports and bandwidth was just through the roof," said Saunier, who manages the network at Cobb EMC, a not-for-profit electric utility based in Marietta, Ga.
Using the WhatsUp Gold network monitoring tool, Saunier traced the problem to two specific ports on a core switch. He realized he was seeing the telltale signs of a network loop, with the CPU usage on both ports sky high. A quick visit to the core switch and a little detective work revealed the problem. A help desk technician had plugged a cable into two ports on the same switch, creating a loop. One port on the switch had been pulled to an Ethernet port mounted on a nearby wall. Apparently the help desk tech thought he had found a cable that had been unplugged from the wall, and he simply plugged it back in. A network engineer would have known that the wall port was an extension of one of the ports on the core switch.
Is your portfolio of network management tools complete?
Learn about the network management capabilities of Netflow v9
What network management tools are engineers using on their smartphones?
Asset management is a key element of a network management plan.
Saunier uses several Ipswitch network management tools to keep an eye on what's happening in his network. In addition to the networking monitoring tool, he uses the WhatsConnected network discovery tool and the WhatsConfigured network configuration management tool.
WhatsConnected is a simple network discovery tool that automatically creates a network topology map, offering details on all switches and routers, along with what devices are plugged into each port. By cross-referencing results of that topology map with the records Saunier has of where each port is physically located on Cobb's campus, he can quickly find the physical location of a device. This capability is essential when looking for devices that are plugged in where they shouldn’t be. WhatsConnected has come in handy for Saunier as he prepares his network for a new network access control installation.
"We did a discovery of printers because we were doing a NAC project where we're trying to lock down the network with 802.1x," he said. "We were trying to locate where all the non-authenticated devices were. Printers were our main problem. We did it once or twice during that early phase, and we'll probably have to do it again to make sure nothing else has been plugged in recently."
Ultimately Saunier will apply this knowledge to use Unified Access Control from Juniper Networks to lock down every port that is supporting a printer. .
"We [will] lock down those ports according to the MAC address of the printer so that no other network device could be plugged in there," he said. "If someone came up and said, 'I want to unplug this printer and plug in my laptop,' their laptop is not going to work."
WhatsConnected originally had some trouble with the switches in Saunier's network. Extreme Networks' method for forwarding SNMP information on multi-link aggregation came through garbled, he said.
"Extreme switches do some funky things with their forwarding database, and they didn't release some information through SNMP that Ipswitch needed. It didn't reveal information for [Extreme's] link aggregation groups. When we would run [WhatsConnected], it would look like we had links going everywhere."
Ipswitch provided some quick software customization to deal with the bug, Saunier said. His topology maps have been accurate ever since.
Even network admins can make network configuration mistakes
End users aren't the only ones who can threaten a network by doing something they shouldn't. Even a network administrator can cause trouble by making a network configuration mistake on a switch.
Saunier uses WhatsConnected to audit network configuration changes on his network. If he detects a problem on the network, he can use those audits to track down whether a network configuration change caused the problem.
"It's on a schedule to download configs from all my switches daily," he said. "I can audit it to make sure certain lines are in the configs in a certain way. We're auditing to make sure the RADIUS servers are set on every switch and that the NTP [network time protocol] server is set right on every switch."
WhatsConfigured alerts Saunier if a major configuration change happens or if some other configuration change failed. For instance, if a NTP server configuration change didn't take on a certain switch, he'll hear about it. However, Saunier is careful not to let WhatsConnected overload him with alerts.
“We don't go down into the very granular stuff, like if someone logs in and deletes a port off a certain VLAN,” he said. “I'm not auditing for that because those changes happen so often. When you have a 100-switch network, stuff changes hourly, and I don't really want alerts when every little thing changes."
0 comments:
Post a Comment