Part 1: Transforming IT Operations with Large Language Models
Note: This is the first in a series of posts exploring how AI agents can support IT Operations teams. In this introductory post, we outline the broader rationale behind the initiative. Future posts will dive into specific use cases within IT Operations.
Introduction
Large language models (LLMs) excel at parsing and understanding vast amounts of text, making them suitable for countless practical applications. The latest models are especially adept at quickly absorbing large volumes of information, summarizing key details and interactively uncovering insights in collaboration with users.
At WWT, we work closely with our partners at HPE and NVIDIA to help customers identify high-impact opportunities to apply this technology and elevate their business operations. These enhancements can benefit both back-office and front-office functions alike.
As we observe the current state of AI technology, we also consider the pain points we've heard from our customers over the years of working with IT departments, particularly IT Operations teams, and the challenges they face in managing day-to-day tasks.
If you have worked within an organization, you have likely interacted with your IT Operations team, whether by opening a ticket to resolve an issue or requesting support for a technology-related task. In larger organizations, dozens or even hundreds of team members handle these requests as quickly as possible.
Behind this familiar scene, key systems help coordinate IT work in a manageable way. One such system is the IT Service Management (ITSM) platform, which supports a wide range of functions, from ticket management and budgeting to project planning and change management across applications and data centers. A well-known example of an ITSM platform is ServiceNow.
What's the problem?
During daily operations over the years, IT systems accumulate huge troves of data that track everything an IT team finds pertinent. Some of this data is well-structured in the traditional sense, stored in specific form fields and easily-managed relational databases. However, quite a lot of it is very unstructured data, large blocks of text stored in many places throughout the ITSM system.
Service desk incidents, for example, often involve many rounds of back-and-forth communication to help find the correct solution. All that dialogue, along with final instructions on how to fix the problem, is stored within the ITSM system. And there is a lot more data like that generated and stored behind the scenes in technical notes, knowledgebase articles, work instructions and the like.
This information ecosystem only continues to expand when considering the broader IT organization. Networking devices, asset management, security updates and routine maintenance all add to the information overload that modern IT teams must manage and understand. While rich in information, the data terrain of any organization quickly becomes an overwhelming task for even the most skilled teams.
For service management specifically, the challenge intensifies as the system grows. For example, having thousands of articles in the knowledge base is valuable because useful information is preserved. However, to resolve a problem, you need to know which item in the collection best matches your situation, and you want to find it quickly. This challenge applies equally to knowledge articles, incidents, changes, requests and other ITSM activities.
So, what can be done?
Together with HPE and NVIDIA, we work closely with clients across industries to help tackle this issue head-on. While every organization is different and faces unique challenges, we've observed some common impact streams for many IT teams. Our solution is not to supplant the systems that IT teams have built their work around, but to augment them. By leveraging AI's strengths, we aim to make the data captured in these systems readily accessible, understandable and relevant.
Where AI can make an impact
We've identified a series of use cases that will cause head nods amongst the staff of any IT organization. These are tasks that they field daily and that consume a significant amount of time and resources.
Our initial focus areas include:
- Generating AI summaries of the current state of operations in the IT department (problems being worked on, changes being made, customer service delays, overall trends that should be followed up on, etc.)
- Assisting an individual worker in resolving a specific reported problem
- Helping assess the utilization of computing equipment and capacity planning
- Prioritizing patches and other vulnerability fixes
- Proactively monitoring telemetry, log and performance data to identify an issue before the IT services are impacted
For each of these scenarios, we are building an AI-powered solution to assist IT staff in executing these tasks more efficiently. As we complete the initial set, additional use cases are under consideration.
Security and privacy first
Given the nature of the data involved, privacy and security are top priorities. As such, each solution needs to be deployed in a secure, carefully crafted environment. To support this, HPE Private Cloud AI, co-developed under the NVIDIA AI Computing by HPE portfolio, introduced earlier this year and is designed specifically for enterprise-grade AI deployments. This platform is powered by NVIDIA AI Accelerators, Networking, and software which offers a range of development and management features.
WWT's AI Proving Ground
We have partnered with HPE to install the HPE Private Cloud AI systems within our AI Proving Ground, a world-class environment where we design, build and showcase these IT Ops AI solutions live for customers. These software service packages are specifically designed and optimized to leverage HPE Private Cloud AI systems using the latest in NVIDIA technology, including development frameworks like NVIDIA NIM™ and the NVIDIA NeMo™ Agent Toolkit.