May 23, 2023
Manage and Contextualize What's Happening On Your Network with Cisco EPNM
Evolved Programmable Network Manager (EPNM) is management tool for carrier-grade networks. WWT's Tyler Tappy and Josh Hogan discuss the benefits of EPNM, its ability to help usher in the age of 5G and how it can bridge the gap between taking traditional SONET and MPLS IP.
Brian Feldt: Hi there. Brian Feldt with Worldwide Technology and I'm pleased today to be joined by Josh Hogan and Tyler Tappy, two technical solutions, architects within our carrier networking practice here at Worldwide Technology. Today, we'll be talking to you about Evolved Programmable Network Manager, or EPNM, and its ability to help improve operational agility and deliver a simplified carrier grade network management experience. Josh, we'll start with you and we'll start broad. Walk us through what EPNM actually is and why we're talking about it today.
Josh Hogan: Yeah. EPNM, it's Evolved Programmable Network Managers, is what it stands for. It's manifestly management tool for carrier grade networks. What that means is it's a web interface, basically, running over HTML5, and it's designed specifically around workflows that are most common to carrier networking. This includes optical functions like wavelength provisioning, as an example. It also includes packet based services and functions like MPLS, the ability of the platform to manage those multiple layers of a network.
The optical packet layers is critical to describing what it is and what it's supposed to do because what's happening right now, in these few years, is that carrier networks, they're facing significant change with 5G coming, SONET Technologies, things like that, are retiring. It really changes how carrier networks do business. As this transition really accelerates, you're seeing that they need a way to manage and contextualize what's happening on the transport network. Again, that's across optical and packet. How do you take information and see the whole big picture? There is this burgeoning need for a platform such as what EPNM is that gives carrier networks the ability to manage things at high scale and with a high degree of programmability.
Tyler Tappy: Yeah, I'm just going to jump in here. I think Josh made a really good point when he referenced multiple layers of the network, like SONET and 5G. A large percentage of operators don't, for example, have the ability to take what is a traditional SONET engineer and turn them over into a MPLS segment routing IP engineer overnight. EPNM can really help bridge this gap. Just to add a little bit on the major benefits of what EPNM is, and we'll expand more on these later, but I just want to call them off for now. It can improve the customer experience. It can maintain the network efficiency. It can accelerate the service delay. It can manage multiple domains, and it can reduce the errors and create efficiencies in the network.
Brian Feldt: We toss out the word carrier grade a lot, and I think it can mean different things to different people, but what types of companies specifically are using EPNM today and how does it help those organizations run their own network infrastructure?
Tyler Tappy: All sorts of companies use EPNM from search providers, to enterprise, to public sector. I mean, really, anybody who's looking to leverage what EPNM has to offer and to take advantage of that. I want to call out that EPNM does use HTML5 and not Java as some systems have in the past. This was historically a contentious point, but I just want to make everybody aware of that. As far as EPNM helping organizations, first off, EPNM is really a good management tool. There's customizable dashboards where you can pick and choose different dashlets to set your homepage to. Dashlets are really just containers that can hold information like CPU utilization or memory utilization, interface statistics, things like that. You set your dashboard up to have the relevant information to you up first and foremost, rather than having to go through the system and searching for it. I think it's pretty common for a lot of operators to use, probably, your network alarms, interface statistics, probably on your major up links to the internet and maybe a network topology or something like that.
I think secondly, EPNM is really good at notifying you when there's something wrong with the network, reporting of issues as they arise. EPNM displays these reported issues in the network, and moreover, it can email them, or we can send down SNMP traps to capture these and we can get as granular as we want with that. Anything from the minor alarms that really aren't going to affect much of anything to the major service affecting critical alarms in the network. Another cool thing about EPNM is its ability to graphically display the chassis, the racks, the cards, the optics, everything within a site that makes a site, EPNM can graphically display that. If you're used to looking at CLI, this can be a major upgrade for you. Having a picture of a chassis with cards and optics and things like that is a really cool thing to look at.
On the flip side of that, if you're not a CLI person, having the ability to actually see a chassis and know what's in there is a big upgrade too. Sometimes managers and C-level personnel are looking for something like that too, to be able to actually be more knowledgeable of the system and not have to use command lines, so it's really helpful. Additionally, some operators, and we'll say this time and time again, don't have the luxury of having experts in every single technology vertical. EPNM can really help to offload that. A good example of this is maintaining and managing circuits with something like, it's called 360 degree, multi-trace of view. It's a long fancy word, but really what it's meant to do is to say, as an operator scales, you have different layers of the network.
There might be an access layer where you're pulling in multiple business clients, residential customers, things of that nature. That feeds up through the core route switch, and then that might go over an optical transport layer where we're feeding traffic to your major up links to the internet and doing data center interconnects and things like that. There's all these different complicated parts of the network. With 360 degree multi trace view, you can see the entirety of that circuit from end to end. Moreover, if you want to then dive into a specific section, you want to say, I want to look at the optical. You can start to break that down and get a really good idea of what's going on, how it's physically connected, what light levels may look like, and things of that nature. 360 degree multi trace view is a really cool thing.
Finally, I'll stop on the major benefits of it, but one other real amazing thing that I love, being an optical guy, is the ability to scale OTDR traces in the event of a fiber cut. An OTDR, if somebody doesn't know what an OTDR is, is an optical time domain reflectometer. What it really is, it's a tool that we've had forever and optical and we can shoot light down a fiber and we can find out where that break in the fiber may be. It can tell you, "40 kilometers out, that's where the end of the fiber is." Historically, operators have went out and had a technician plugin to the outside fiber and launch an OTDR, which is a truck roll, it's time, and it's money. There's a lot of process in that.
We've been able to scale OTDR capabilities within the chassis and then more over, EPNM can tie that OTDR capability to have it give you up to a street level address using geo coordinates. We can tie the insights to a geo coordinate. We can tie geo coordinates along the path of the fiber and EPNM talks with the OTDR functionality within the optical chassis and booms, it gives you a street level address, 1234 Sesame Street, that's where your fiber break is. You can go out and troubleshoot from there. Really, that can be automated as well. If there's a fiber cut, the system kicks in, it tells you where it is, and you don't have to send trucks out to plugin an OTDR anymore. It's really cool.
Brian Feldt: Josh, seemingly, everything is virtualized these days. Maybe walk us through, and talk about, how EPNM fits into an overall software defined strategy and how it can help with automation, orchestration, or application driven solutions.
Josh Hogan: Yeah, that's a great question. This is the aspect of the solution since programmable is in the name of all programmable network manager. It really plays at several levels, different levels of automation and orchestration. If you can think about it, down at the base layer, you have your devices, which are just supposed to execute data, plane, traffic, do things. Down at that layer, EPNM really helps you to standardize your configuration, helps you to automate the provisioning of that configuration in the way that it's pushed to the devices. It really helps you set up device standards, first of all, if you think of it as almost the base of a pyramid. That's going to help you eliminate manual toil, reduce user error, stuff like that. You move up from that, one more level in the pyramid, and you get to a more network view or a services view.
There, what EPNM really does is it speeds up your rollout of services. It gives you a much higher level of service assurance. Just basic stuff like testing, troubleshooting capabilities, it's got a ton of different protocols that it can run that really helps with stuff like that. At the very top of the pyramid, there's the organizational architectural level is what I'd call it. That's where you look at how does it interact with operations support systems, OSS tools, basically. How does it use its open API, rest comp, et cetera? With all of those things, you're able to get EPNM functioning within an overall or broader dev ops strategy within your organization. That just helps move, again, your carrier network mindset away from all of these devices out there are pets, we need to treat them like pets, towards what everybody talks about with dev ops, where we want to treat them more like cattle. We want to move the herd in the direction that we want to go.
A key thing in the whole question that's worth mentioning is that EPNM has these great hooks in to allow for programmability and automation, but the really nice thing about it is its still manifestly a day-to-day management tool. A lot of the workflows within the system are designed specifically to allow a user to go in there and look at what's happening on the network right now, to implement a new service right now, or to troubleshoot something that's happening. It is the best of both worlds from that perspective, because there are a lot of workflows specifically designed for that, but at the same time, you have all of your hooks to integrate with the common tools we all know about, Cisco, So, Way, CrossWork, even the SDN controllers like Open Daylight.
For anyone who's actually curious about automating using EPNM specifically, we highly recommend going to Cisco DevNet. Cisco DevNet has a page set aside just for EPNM. If you search EPNM, you can find this whole page that they've set up with different examples of using the API hooks, getting different things automated. Maybe we'll try to include a link to it here, but that's a really good resource to use.
Brian Feldt: Absolutely. Tyler, anybody investing in this type of technology is doing so to drive efficiencies and help make their network more efficient. Does EPNM help customers? How does it help them save money? Is it free software? Talk about how it's helping these organizations make the network more efficient.
Tyler Tappy: Yeah, fair question there. EPNM does come at a fee, however, there are efficiencies with EPNM. One of the most expensive costs an operator can incur our operator induced errors that just take down the network. They're nearly impossible to plan for, and they happen when nobody expects it. It's typically a function that we've done a thousand times, but somebody fat fingers something, and boom, there goes the network. With EPNM, you can create configuration templates to be sure that standardized configs are being pushed down during maintenance windows. This can drastically reduce operator error and things like that. It also leverages the ability to auto roll back if things don't go as planned. This will allow the network to, somewhat, self recover or self heal and without needing manual intervention. Just in terms of avoiding self inflicted wounds, I think EPNM can definitely save a lot of time, money, and headache.
Brian Feldt: Do we have any specific examples of where EPNM fits on a vertical basis? I know we did touch a little bit on who is EPNM for, in terms of organizations, but maybe detail a little bit on a vertical basis or industry basis, where this fits.
Josh Hogan: Yeah, Tyler hit the nail on the head earlier answering that question relative to it's a carrier networking tool. It makes a lot of sense in the traditional service provider type of network. Small, medium sized service providers, but really just from a broader picture, it applies anywhere that there's a transport network that needs to be managed. When I use the term transport network, I'm typically talking about anyone who runs their own MPLS services, so not hooking up to a provider that's giving you the services, but you are the provider. You're running your MPLS services, or if you run your own optical network like DWDM or other things. In either of those cases, it makes a lot of sense. EPNM is focused at that transport capability. That's the big picture.
A specific example we could give a particular vertical or industry is the industrial and utility vertical, where a really common thing that's happening there, and I touched a little bit on this in the beginning is, really heavy usage and reliance on legacy technologies like SONET. Time division, multiplexing interfaces is very common. The other lowest types of low speed in our space is very common in those environments. They've just been hanging out there, they're serving the more OT rather than IT functionality in that industry. EPNM is a really, really good fit there because as those legacy technologies fall away, and the organizations move towards something like MPLS to replace the core infrastructure, EPNM has a lot of workflow, specifically for the things like circuit emulation of time division multiplexing, helping you to transport that across an MPLS score.
In fact, to this specific use case, Worldwide Technology, we've partnered with Cisco and a company called Schweitzer Electric, or SEL, to develop this integrated solution. It showcases EPNM. The whole thing is based on, essentially, using Cisco routers to provide an IP or MPLS core, and then SEL multi-flexors that do all of that circuit emulation to replace those legacy technologies. The nice thing about it is, of course, EPNM orchestrates the whole thing soup to nuts. We have an article on this architecture, which we might try to include the link to that as well.
There are specific industries where it makes sense, again, based on who's transporting, who's acting as an MPLS or an optical transport network. I would use that as my rule of thumb. If you have either of those two things, especially if you have both, then EPNM makes a lot of sense.
Brian Feldt: Tyler, maybe back to a technical question here. How does EPNM manage these devices? Are there standard interfaces that it uses?
Tyler Tappy: Yeah, so each device that supported in EPNM has its own native language. Optical is TL1, routers and switches have some variant of iOS xe or iOS xr, or CLI. EPNM is able to push down the native language of each of these specific devices to deploy a specific function. For example, in optical, we create these things called circuits. When you run through the provisioning wizard, which is pretty much, "Site one, check. Hundred gig, check. Next. Site two, check. A hundred gig, check. Next. Is this the path you want to take? Sure, next." You're at the final screen and it prompts you with the t01, but the actual native language of the optical chassis. It prompts you, per site, what it's going to do, what it's going to push down and what is planning to do.
There's a check and balance there. If you're savvy enough to look at that and say, "Yeah, this looks good," or "No, it doesn't." I've never seen an instance where it doesn't, but it does the same thing for iOS xr, iOS xe, whatever. It's a really good tool if you're savvy enough with the language. iOS xr, for example, you create an MPLS circuit, you hit that final screen, do your checks and balances, you're good to go. On the flip side, again there's two sides of the coin in all that I talk about with EPNM, I've learned a lot from EPNM in just looking at the t01 output. I'm sure a novice in route switch, for example, maybe a NOC technician who's looking to learn more, when he pushes these standardized configurations down, it's going to tell you what that iOS xr output is, for example. You can use it as a learning experience, I did.
Another thing, in terms of software upgrades, EPNM does a fantastic job. It stages it out. You have four stages. There's the adding of the software, distributing it, activating it, and finally committing it. Not all of these functions are going to be used for all the products, but for the most part, they're pretty relevant. When you add in the software into the system, you're storing it within the EPNM server itself. When you're distributing it, you're choosing which devices you want to push that down to. It sits in that device's memory or cache, wherever it's going to store it.
There's an activation piece. On some products, the activations the last step. On other products, the commit is actually the last step. It just depends, but it does a really good job at this. Those can be scheduled so you can have an automated process for that to reduce the human error. Not that we're not perfect, right? Device backups are really important thing for operators. EPNM has the ability to do this. It can be automated as well, so you can do this daily if you wanted, midnight for example. A lot of operators want to have their most recent configurations.
Finally, I'll end on maintenance mode. Maintenance mode is a feature. I see it most beneficial towards network operations center or folks who have NOCs. When they're using this feature maintenance mode, basically, the network is somewhat self aware that there's maintenance activities happening. Maybe you have, I was talking to different portions of the network before, you have maybe the route switch layer over here and the rest of your networks over here, and we're going to do a big router upgrade tonight. With maintenance mode, the NOC personnel aren't going to see those router alarms coming at them. They're not going to be bombarded by a ton of information that's really not relevant, because we know there's an upgrade happening. Its happening regardless, so it can suppress that information and just make it easier on the rest of the network and the NOC personnel.
Brian Feldt: At the end of the day, organizations are using EPNM to get to market faster. Josh, how specifically is it helping them get their products or solutions or services to market faster? Can you spell it out for us?
Josh Hogan: Yeah, It goes to a couple of key points Tyler had just made, especially around the web interface of EPNM taking your instruction and your commands, what you want the network to do, converting that into language that the devices understand and going forth and executing it. That functionality, it's really surprising and amazing. I remember the first time that I used EPNM. I used it to configure a pretty basic ethernet VPN service. It's your run of the mill MPLS service, nothing special if you go and do it in CLI. I got to the end of the workflow within EPNM and I was, "Is this it? Is that all I have to do?" My brain was in CLI mode. I was so used to having to do it here, here, and here, and all these different things you have to stitch together inside of the command line interface. EPNM takes that process, it really turns it into a few clicks and some basic information you need. It does the rest, regardless of the operating system that's on the router and where it needs to go and all of that.
I'd say the first thing is the provisioning of new services, provisioning of circuits, provisioning the capabilities, greatly accelerated by EPNM. That's going to save time and save money and free up people who operate on the network to do other things. A second feature, I think we've touched on this as well, is configuration templates. I don't want to beat a dead horse with configuration templates, but the ability to standardize your configurations across different network device types let's say, different whole lines of routers, different operating systems, different network functions that those boxes are performing. That really helps to turn out efficiencies and it streamlines efforts. You can build up your configuration templates in a hierarchical manner, however best suits your organization. It's most especially helpful in the situation where those types of tasks are being done manually today. We see that surprisingly often.
The last thing I'd really want to mention is lifecycle management. EPNM makes a wonderful, wonderful [inaudible 00:21:12]. It offers things like the SONET management, Tyler spoke to the whole software upgrade process it can help with. It has configuration backup, fall back. It even does things like network discovery. Really, by simplifying all of those arduous tasks that are associated with lifecycle management, which is by no means the sexiest part of operating a network, it takes a heavy load off of those operators. Again, it allows standardization and automation, which just frees them up to work on other things. It increases the wheels.
Brian Feldt: As with any technology, you can like the benefits and the value proposition that it affords, but you still got to deploy it. How are we helping organizations deploy EPNM and what goes into that deployment process?
Tyler Tappy: EPNM can be installed on a new appliance or integrated with the existing data center. This can be done as an OVA/VM installation, or can be an ISO [inaudible] install. This can be done standalone, so a standalone EPNM instance, or if you're looking for the redundancy option, there's high availability. That's available as well. Some operators prefer to utilize their own virtual environment, which is completely fine. Other operators prefer to have a standalone appliance, which is completely fine. There are no wrong answers here, it's just up to the operator's preference. I would say, for me, I like redundancy, so maybe HA would be something to consider. For the most part, it's really up to how they want to run their network and how they want to install EPNM. That basically gets the software installed into a server, essentially.
Once that software is installed and communicating with the local LAN, then we can start adding devices into the network. Devices can be done, they can be added in one by one if that's what you choose to do, or you can run a network discovery to grab multiple devices at one. If you have a lot of devices out there that you need to pull down into EPNM, a network discovery is probably good idea. You can do things like ping sweets, SNMP polling, import CVS file, or you could create some other type of custom discovered profile if you wanted. Now that we have all the devices, in EPNM, or all the devices that we want in EPNM, and they're discovered and we can see them, now we can start provisioning services over that network.
We touched on it. A few points, you can do services one by one. That's what we did in our optical lab, our EPNM optical and route switch lab actually. I just did them one by one, which is fine because we only have a few in that lab, but you can also create, as we spoke to before, there's configuration templates, there's a lot of options there to streamline that process and just make it quicker and easier, and to avoid some type of human error in that instance. Configuration templates are always an option. I know it's hard to sit here and talk about EPNM in a 30 minute to 20 minute conversation. If anybody wants to go spin the tires on EPNM, Josh and myself, we created a lab. It's available on wwt.com.
If you go to wwt.com, and you search for EPNM, the lab itself will actually show up in the results. You can schedule that lab and you can kick the tires. There's optical hardware, there's route switch hardware, it walks you through it. We'll be adding more and more stuff over time, as well, as EPNM goes along this path. Moreover, if anybody has any questions, please feel free to go to wwt.com. You can search for myself, Tyler Tappy, or Joshua Hogan on there, and you can connect with us through our platform.
Brian Feldt: Well, excellent guys. I'm out of questions, but you did a fantastic job of walking us through EPNM, the benefits, the life cycle, and just deploying it for organizations. Certainly, thanks to you for taking time out of your busy schedule and thanks to our viewers for taking an interest in the topic. As Tyler had mentioned, all of these resources that we talked about today, including more carrier networking solutions, are available on our wwt.com platform. We encourage you to check that out. Until next time, to the two of you, thanks so much for joining us and enjoy the rest of your day.
Tyler Tappy: Thanks.
Josh Hogan: Thank you.