threshold of Memory exceeded alerts

[Maxim Yegorushkin]

Maxim Yegorushkin [http://community.zenoss.org/people/max0x7ba] created the discussion

"threshold of Memory exceeded alerts"

To view the discussion, visit: http://community.zenoss.org/message/74764#74764

--------------------------------------------------------------
Since upgrade to Zenoss 4.2.4 it fills the event log with the following event:

    Resource:          xyz.com
    Component:          <some_process>
    Event Class:          /Perf/Memory
    Status:          New
    Message:          threshold of Memory exceeded: current value 52391936.000000
    Event Management...
    agent          zenprocess
    component          <some_process>
    dedupid          <some_process> a46c5d5b5d3faa76d0d56c73190403cf|/Perf/Memory|mem_mem|Memory|3
    eventClass          /Perf/Memory
    eventClassKey
    eventClassMapping
    eventGroup
    eventKey          mem_mem|Memory
    eventState          New
    evid          00505694-5e42-a852-11e3-273cce55fe36
    facility
    message          threshold of Memory exceeded: current value 52391936.000000
    ntevid
    priority
    severity          3
    summary          threshold of Memory exceeded: current value 52391936.000000
    Device State...
    DeviceClass          /Server/Linux
    DeviceGroups
    DevicePriority          Normal
    Location          /Abc
    Systems
    device          xyz.com
    ipAddress          a.b.c.d
    monitor          localhost
    prodState          Production
    Event Data...
    clearid
    count          1483
    firstTime          2013-09-27 06:19:46
    lastTime          2013-09-30 09:05:35
    ownerid
    stateChange          2013-09-27 06:19:46
    Event Details...
    current          52391936.0
    how          exceeded
    manager          xyz.com
    max          90
    zenoss.device.device_class          /Server/Linux
    zenoss.device.ip_address          a.b.c.d
    zenoss.device.location          /Abc
    zenoss.device.priority          3
    zenoss.device.production_state          1000

I can not find where these event come from.

1. What emits this alert?
2. Why does it quote absolute memory values (e.g. 52391936 bytes), rather than relative (e.g. 90%)?
3. Where/how to configure this threshold?

-- Maxim
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74764#74764]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dhopp]

dhopp [http://community.zenoss.org/people/dhopp] created the discussion

"Re: threshold of Memory exceeded alerts"

To view the discussion, visit: http://community.zenoss.org/message/74821#74821

--------------------------------------------------------------
The 'agent' of that event is zenprocess so that is the daemon that created the event.  This is an event for a process running on xyz.com (the resource in the event) using too much memory.  If you go to the 'xyz.com' resource in Zenoss and then go to "OS Processes" and then find whatever component is in the component field of the event you should be able to see the memory graph for that component, along with it's threshold.  In the middle of the screen you will see a drop down that probably says 'Graphs'.  Drop that down to 'Templates' and you can see the template that is being used to monitor the process.  On the right side of that is the threshold.  You can change the threshold here but if you want to change to ONLY affect this one process make sure you click "Create Local" to make the template local first.

It's showing absolute memory because that's how the threshold of a process is apparently defined.  A process technically doesn't have a 'max' memory (it could just use all of the memory the OS has) so it's hard to define a percentage base threshold for that.

--Dennis
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74821#74821]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Maxim Yegorushkin]

Maxim Yegorushkin [http://community.zenoss.org/people/max0x7ba] created the discussion

"Re: threshold of Memory exceeded alerts"

To view the discussion, visit: http://community.zenoss.org/message/74825#74825

--------------------------------------------------------------
Thanks Dennis.

The trouble is that the threshold defined in %, i.e. 90%, Whereas the process memory usage is reported in bytes, so that mixing these units results in threshold being always breached, and hence it constantly generates that "threshold of Memory exceed" alert for all of my processes.

Is there a way to normilize process memory usage in bytes to percentage of total physical memory and compare the result to the threshold?

-- Maxim
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74825#74825]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dhopp]

dhopp [http://community.zenoss.org/people/dhopp] created the discussion

"Re: threshold of Memory exceeded alerts"

To view the discussion, visit: http://community.zenoss.org/message/74826#74826

--------------------------------------------------------------
The default (out of box) Process template does not have a threashold defined for Memory so this looks like somebody modified it to attempt to do it. 

To make it a percentage of overall memory you would have to set the threshold to something like here.hw.totalMemory * 0.75

That would make the threshold 75% of the memory in the device. However, the event will still come in not specifying the percentage so you would probably then have to transform the event to display the percentage.

Also, for linux servers that might be misleading becuase I believe here.hw.totalMemory returns the amount of RAM and doesn't account for swap.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74826#74826]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]