MemVerge Memory Machine
In this blog
Summary
These real-time workflows and applications have exposed a large performance gap in existing solutions. In an ideal world, these applications would live entirely in system memory, but legacy architectures cannot support such a deployment due to a lack of capacity, performance, or availability required. This data center challenge has opened the door to new, innovative software-defined memory solutions like MemVerge Memory Machine.
MemVerge Memory Machine solution enables administrators to combine DRAM and Intel Persistent Memory Modules (PMem) into a single memory pool and then, present it to their in-memory applications without having to rewrite the application. In addition, Memory Machine provides data services such as in-memory snapshots and replication, addressing application fault tolerance and data durability challenges when applications are deployed on persistent memories.
In the ATC Insight section, you see how we tested the performance of MemVerge Memory Machine with a real-world in-memory database application called KDB+.
Expectations
The MemVerge Lab is a comprehensive set of tests that are recommended as general-purpose Proof of Concept testing. The purpose of this test is to showcase the features, performance, and functionality of the Memory Machine, which in turn informs MemVerge customers on how to utilize it in serving their business goals. The application we will be using in the demo will be KDB+ which is a Time Series DBMS.
ATC Insight
Hardware
A pair of Dell PowerEdge Intel-based dual-socket servers are used for the testing. Both systems with 384Gb of DRAM and 3TB of Intel Optane Persistent Memory. It is recommended that users have at least 1TB of PMEM within each server node.
Software Versions:
Performance Test
KDB Performance Testing
Kx's KDB+ is a time-series in-memory database. It is known for its speed and efficiency and for that reason very popular in Financial Service Industry. One big constraint for KDB+ is the limitation of DRAM capacity. MemVerge Memory Machine fits perfectly here so KDB+ can take full advantage of PMEM for expanded Memory space with similar performance to that of DRAM.
KDB Benchmark test with bulk insert
Using a small amount of DRAM as cache, MemVerge Memory Machine closes up the performance gap, and in some cases even surpasses the DRAM performance because of the advantage of using a huge page and a better memory allocator.
KDB Benchmark test with Read Test
The results show that with more DRAM caching MemVerge Memory Machine can even surpass the result from a pure DRAM-only configuration.
Test Cases
KDB+ is a column-based relational time-series database with in-memory abilities, developed and marketed by Kx Systems. The database is commonly used in high-frequency trading to store, analyze, process, and retrieve large data sets at high speed.
KDB Benchmark test with bulk insert
Bulk insert on KDB when it runs on DRAM
|
Bulk insert on KDB when it runs on Memory Machine with only PMEM
|
Enable 20 GB DRAM cache by adding these lines to /etc/memverge/mvmalloc.yml:
|
Bulk insert on KDB when it runs on Memory Machine with PMEM and small DRAM cache
|
KDB Benchmark test with Read Test
Read test on KDB when it runs on DRAM
|
Read Test on KDB when it runs on Memory Machine with only PMEM. The Optane reading is to be ignored as MemVerge is transparent to KDB so no Optane usage will be reported by the KDB.
|
Enable 20GB DRAM cache by modifying /etc/memverge/mvmalloc.yml:
|
Bulk insert on KDB when it runs on Memory Machine with PMEM and small DRAM cache
|
Technology under test
MemVerge Memory Machine
Conclusion
As you can see from the graphical results above MemVege Memory machine is able to present a pool of high-speed memory to a natively installed KDB+ database instance to perform everyday database tasks faster than it ever could with a conventional solution. The combination of DRAM, Intel Persistent Memory Modules, and MemVerge Memory machine allows administrators to build a highly performant, large memory solution with a great return on investment when compared to previous solutions which rely on slower, more expensive LRDIMMs. The solution also provides an environment with more than twice the memory capacity of conventional solutions without the headache of needing to rewrite the application or sacrificing performance for capacity.