WWT's Live Malware Test Lab: How to Select the Right Endpoint Security Solution
In this article
The endpoint is the most common entry point for cybersecurity attacks. With the proliferation of remote access, cloud computing, microservices and other "perimeter-less" technologies, endpoint security has become more important than ever for securing today's enterprise.
Today's endpoint solutions go far beyond the antivirus capabilities that sufficed in the past. New endpoint security offerings enter the market all the time; it seems like every month, some vendor announces an innovative new way to detect or block threats. In this dynamic environment, how does an enterprise identify the best product to meet its unique requirements?
WWT's new Live Malware Test Lab, housed in our Advanced Technology Center (ATC), is a permanent "air-gapped" installation that simulates customer environments and allows for safe testing of security solutions using live malware. It supports both physical and virtual endpoints and can be customized to simulate a wide range of testing conditions.
Using the Live Malware Test Lab, or the "Petri Dish" as it's fondly nicknamed, customers can see different products in action and compare them based on specific requirements. When combined with WWT's established proof of concept (PoC) testing methodology and vendor-neutral approach, customers can gain a 360-degree understanding of how these security tools will react in their own environments when under fire.
As one major healthcare provider recently learned, the results can be surprising.
Proof of concept yields surprising results
A major healthcare provider approached WWT looking for assistance in evaluating endpoint solutions. Specifically, they wanted to compare their incumbent product to three best-of-breed competitors to determine if a replacement was warranted. Specifically, they posed the following questions:
- How effective is each security product in detecting both known and unknown threats?
- What is the impact of each security agent on [the customer's] end-user platform?
- Knowing that any replacement would require overlap, can existing and new solutions coexist on the endpoint platform without causing conflicts or performance issues?
- What is the end-user experience associated with each product?
- How tamper-proof are the endpoint agents?
Working with the customer to identify the drivers behind these questions, WWT developed a set of detailed evaluation methods and criteria. Using our Live Malware Test Lab, WWT then worked with the customer to evaluate the four endpoint security solutions (one incumbent, three candidates) side-by-side on qualitative and quantitative bases.
The engagement began with the development of a test plan. Prompted by thorough templates provided by WWT, the customer identified and prioritized the test criteria most important to them. In addition to testing, the customer asked WWT to help them review the products in terms of ease of deployment, reporting capabilities, solution scalability, operational impact, usability, policy granularity and the quality of vendor support.
The PoC consisted of control-based testing in the Live Malware Test Lab and evaluation of questionnaires issued to each vendor. As the neutral arbiter, WWT worked to establish a level playing field built around the defined objectives. Testing involved injection of more than 150 different varieties of documented and undocumented attacks (classified as "Rats," "Ransomware," "Trojans" and "Droppers/Maldocs") against infrastructure installed in the isolated lab environment. Carefully designed controls ensured that collected performance data could be compared against baselines.
Tools were analyzed for whether they detected each particular attack and how effectively they responded.
End-to-end performance of each tool was evaluated, with the endpoint in both offline and online modes, to adjust for different methods of prevention, detection and response.
Ultimately, WWT compiled results vendor questionnaire evaluations to develop an evaluation matrix that compared the usefulness of each tool within the customer's environment. Weighting was applied by the customer based upon their specific requirements.
Result and lessons learned
Over the course of the PoC, WWT helped the healthcare provider understand and refine their own operational requirements and how each product addressed those requirements. Taken in this light, the incumbent product supported the customer's specific operational requirements in ways the others didn't. They decided to keep that product and supplement its shortcomings with other tools and operational changes.
The customer used what it learned in our Live Malware Test Lab to change its perspective on the best way to reduce cybersecurity risk at the lowest cost. WWT provided significant value throughout the process through our strategic consultative approach, our OEM vendor relationships and our neutral arbitration of the PoC.
For anyone working through the process of selecting an endpoint security solution today, several lessons learned stand out that might help you on your journey:
Lesson 1: No endpoint security decision should be made without first testing the product(s) under consideration.
There's no "one size fits all" security strategy or product. The characteristics that make a solution appropriate vary from enterprise to enterprise based upon a variety of factors. These can include maturity level, risk appetite and management strategy, end-user impact, culture, personnel, infrastructure compatibility and even personal preference. "Live-fire" testing, especially conducted under controlled conditions with the support of a trusted advisor like WWT, can help you identify and refine true requirements and narrow the options quickly.
While security efficacy of a tool is the most important decision criteria, testing often reveals only slight differences in the effectiveness of different products. In the case of the live testing described above, the effectiveness of the security tools varied within the margin of error. At the same time, appropriate testing can provide useful insights far beyond security effectiveness.
In addition to testing the effectiveness of a tool, "live-fire" PoCs like this one can help answer some important questions around how the tool you select impacts existing tools, workflows and users:
- Live testing can show the impact a tool has upon end users
How much security functionality must be turned off or adjusted to produce a reasonable user experience? What level of interruption and/or annoyance will the tool cause end users? What effect will the security tool have on the performance of end-user devices? How does the tool interact with other tools that require agents on end devices?
- Live testing can reveal how a tool fits with existing SOC workflows
Testing an endpoint security tool allows an organization to gauge the fit a certain product will have on their SOC workflow and maturity level. How well does the product identify actionable alerts? Is it easy to use? Does it provide sufficient access to information to support extensive threat hunting or investigations? What granularity of policy control does the platform provide?
- Live testing can demonstrate how a tools fits with currently deployed technologies
How will the tool fit into the organization's existing ecosystem. Will it integrate with the other security technologies already deployed? Will it conflict with other solutions?
WWT's PoC testing labs provide actionable data about how products will act in their environment, allowing customers to cut through conflicting marketing claims with confidence.
Lesson 2: While each product tested may be effective at stopping malware, full endpoint security involves a lot more.
Over a small sample size, the four products detected and responded to 88 to 93 percent of the attacks deployed during our live PoC testing. While these results demonstrate significant effectiveness, just one successful attack is enough to ruin a CISO's day.
Instead of over-relying on one endpoint solution to secure the enterprise, customers need to evaluate how that endpoint solution fits into their overall security architecture. Organizations should consider the following questions when shopping for a solution:
- Does the solution align with your organization's security staff culture?
Is the staff is used to complex security policies with lots of knobs and dials, or do they require simplicity? Complexity provides flexibility but isn't always easy to manage, especially with inexperienced or entry-level staff.
Several newer solutions attempt to solve this problem by adopting simple front ends and incorporating more logic into analyzing events before they're presented to analysts. This approach can be easier to use but may limit the information available for deeper threat-hunting or the response and remediation options available to the analyst. Some teams can accept this approach, others cannot.
Each enterprise needs to determine what tools should be integrated based on fit with their security strategy.
- Does your organization's security staff have the skillset to manage the new solution?
More complicated solutions carry hidden costs. How many people are needed to manage the tool? If the current tool requires only one FTE, but the replacement requires at least three, does this still fit the requirements of the organization? Will the OEM tell you that? Also, does the organization require skilled threat hunters who know what they're doing? Can a security generalist run the tool?
In today's environment varied skillsets may be required to install a firewall in the morning, install an identity management product in the afternoon and then hunt for threats in the evening. Who is on the team? What plays are they capable of running? What additional tools are needed to round out the suite and enable these capabilities?
- Will the tool help to automate mundane security tasks?
Security resources are limited. With an ever-growing threat landscape, automation and orchestration will become a necessity for keeping up. Endpoints can be the most important component of an automated security infrastructure for providing visibility and acting as an enforcement point.
Every tool incorporated into an organization today must have an Advanced Programming Interface ("API") that allows efficient interactions with other tools. Every manufacturer will say, "Oh yes, we have an API." But the devil can be in the details. What collected information is exposed through the API? What endpoint enforcement actions can be activated through the API? Can the API be accessed through the cloud or must an organization open firewall ports on the network perimeter (potential new exploitable weaknesses) to get full functionality? These are all characteristics not readily seen on a Gartner MQ or Forrester Report. Nor will they show up in manufacturer white papers.
- Does the tool complement the rest of your organization's security stack?
What gaps does the tool fill in the organization's overall security strategy? Will it become just one of multiple tools that have functional overlap? A thorough Security Tools Rationalization process can help an organization stand back from the point-product problem and see the forest from the trees. This is an architectural conversation that requires white boarding behind closed doors with no OEMs in the room. This is where WWT becomes the trusted advisor.
WWT's Live Malware Test Lab is an important value-add that helps our customers answer these questions to identify the endpoint security product that best fits their needs.
Have confidence in your solution
Our dedicated Endpoint Security Team offers workshops and pre-built service engagements built around the customer endpoint security needs we encounter every day. These low-cost offerings are designed to help customers reduce risk while achieving a return on investment.
In addition to our Live Malware Test Lab, check out these additional cybersecurity tools in our ATC:
- Endpoint Security Workshop: We work with customers to understand current state, identify specific requirements and build a future roadmap.
- Cyber Posture Assessment: We develop a current baseline of organizational health from an endpoint security perspective, including system hardening, vulnerabilities present and patching levels for operating systems and applications.
- Security Tools Rationalization Workshop: We identify gaps and overlaps in the customer's security tools infrastructure.
- Patch Management Assessment: We evaluate and improve an organization's patch efficiency.
Finally, our Security Consulting Group brings together professionals with real practitioner experience, the competencies and offerings of our OEM partners and in-depth technology expertise to address readiness, compliance, risk and operations.
Through these capabilities and more, WWT engages with customers at all levels of maturity to reduce risk while streamlining tool footprints and identifying the right tools to meet individual objectives.