Ambari dashboard memory usage explanation for spark cluster

Question

I am using Ambari to monitor my spark cluster, and I'm a little confused by all the memory categories; Can somebody with expertise explain what these terms mean? Thanks in advance!

Here is a screen shot of the Ambari Memory Usage zoom out:

Basically what do swap, Share, Cache and Buffer memory usage stand for? (I think I understand Total well)

score 2 · Accepted Answer · answered Apr 04 '18 at 17:14

2

There is nothing specific to Spark or Ambari here. These are basic Linux / Unix memory management terms:

In short:

Swap is a part of memory written to disk. See Wikipedia and What is swap memory?.
Buffer and cache are used for caching filesystem data and file data. See What is the difference between buffer vs cache memory in Linux? and Overview of memory management
Shared memory is a part of virtual memory used for shared libraries.

answered Apr 04 '18 at 17:14

user9598211

36
1

Thanks for the answer. It makes more sense to me now. One more question though, as you may have noticed from the graph, the swap memory is pretty high. Why would that happen, as it seems there's enough total memory to use? – Psidom Apr 04 '18 at 17:30

Ambari dashboard memory usage explanation for spark cluster

1 Answers1