Quadric Software

Windows Memory Utilization and I/O Performance in Alike 4.1

Understanding Windows Memory Utilization and I/O Performance on the Alike Server

Before we can understand the significance of the windows memory utilization and I/O performance changes in v4.1, we need to understand potential issues on earlier releases of Alike, and why 4.1 will outperform them. First, it is important to note that Alike is a 32-bit application. As such, it can only address 4GB of memory at most, due to the maximum addressable offset for a 32-bit pointer.

Realistically, like most 32-bit apps, Alike cannot address more than 2GB because of a legacy restriction from the era of 32-bit OS’s. Therefore, when a Windows system running Alike is getting low on memory, it’s exceedingly rare that the Alike process (BackupScheduler.exe) is the the problem.

If Alike Isn’t Using Up All the Memory, What’s the Problem Then?

Windows, especially server versions of Windows, is trying to help boost performance by aggressively allocating I/O buffers in kernel memory. It does this so aggressively, in fact, that it’s quite possible to consume double-digit gigabytes of RAM while Alike is busily committing, validating, and vaulting. This is especially true in environments with powerful disks where Alike has been tuned to use more threads and more simultaneous file connections to aid in performance.

This results in the OS working harder to prefetch Alike backup data in order to speed up subsequent read operations—the theory being that if it simply reads enough disk data into RAM, future requests will not have to hit disk. Unfortunately, when you have terabytes of data stored and are looking to validate blocks that are sized at 512KB (less with compression), there’s really no way to “prefetch” effectively, because even if you have 100GB of RAM available, you’ve only cached less than 1% of block data.

With only a tiny fraction of data in memory cache, reads are going to miss the cache anyway, ensuring that all that buffering is wasted effort. In other words, the OS is expending precious IOPS while consuming an exorbitant amount of RAM in the process.

How Does 4.1 Solve the Memory Problem?

Our support team has spent a lot of time on this issue, and it’s only in rare cases (and large environments) that the Windows OS completely runs out of RAM and starts relying on virtual memory for routine operations: a situation known as “page thrash.” This issue is more common in Windows 2008R2 than in Windows 20012R2—the latter does a better job at buffer management.

Historically, we’ve supplied workarounds to this situation via PowerShell scripts that forces Windows to reserve more memory for itself. But this wasteful and counterproductive issue is still there, and poses an obstacle in scaling Alike data stores.

The solution in Alike 4.1 is to switch the style of data store access away from buffered reads—the default Windows file access behavior—to unbuffered reads, which skip kernel buffering by using page-aligned application memory for I/O. The simple, effective result? We cut out the buffering middle man.

In our internal tests, unbuffered reads in Alike 4.1 outperform 4.0 by a large margin, up to 13x faster in some tests. For smaller Alike installs, the performance gains will be moot. Even dramatic improvements in background maintenance operations aren’t especially visible since they don’t make backup jobs complete faster; they only make the validate part happen sooner.

But for our larger customers, this change is very important. The more data you’re moving simultaneously, the faster background maintenance operations will be, which means more IOPS to spare to do other things. In other words, if you run Alike in a large environment, and you’ve had concerns about memory utilization on the Alike server environment in the past, it’s worth your while to revisit Alike’s performance in 4.1.

Article by Max Ekstrom, Quadric Software CTO