MinMaxThreshold not Generating Event

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"MinMaxThreshold not Generating Event"

To view the discussion, visit: http://community.zenoss.org/message/63893#63893

--------------------------------------------------------------
3.2.1 on RHEL 5.5

I have a JMX based datasource with a derive datapoint upon which I have set a MinMaxThreshold of 1 (Just for testing).
I then apply some load to the application in question and I can watch the graph go from 0 to 10 (the amount of load I'm applying)

My problem is that I never see an event generated, and I'm not sure where to go to debug it further.

Here are the settings of the threshold:
          Severity = Error
          Enabled = True
          Minimum Value = Blank/Empty
          Maximum Value = 1
          Event Class = /App/Conn
          Escalate Count = 0

There are no event transforms applied to "/App/Conn" and the device in question is set to "Production" Production State.
What am I missing here?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63893#63893]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: MinMaxThreshold not Generating Event"

To view the discussion, visit: http://community.zenoss.org/message/63894#63894

--------------------------------------------------------------

DEBUG zen.RRDUtil: /opt/zenoss/perf/Devices/mydevice/My DSName Includes Spaces_myDataPoint.rrd: 109810L
DEBUG zen.thresholds: Checking value 13.4525006792 on Devices/mydevice/My DSName Includes Spaces_myDataPoint
DEBUG zen.MinMaxCheck: Checking My DSName Includes Spaces_myDataPoint 13.4525006792 against min None and max 3
DEBUG zen.zenjmx: Queueing event {'manager': 'ZenossServer', 'eventKey': 'My DSName Includes Spaces_myDataPoint', 'device': 'mydevice', 'eventClass': '/App/Conn', 'summary': *'threshold of Max_HTTP_Reqs_Sec exceeded: current value 13.45*', 'component': '', 'monitor': 'localhost', 'agent': 'zenjmx', 'severity': 4}

So it recognizes the fact that it was over the threshold, but still no event. I also checked event history and enabled clears in case for some reason a quick clear or something was causing an event to get cleared right away. Nothing... So I guess the question at this point is once an event is "Queued" where is the next logical place to determine what happened to it?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63894#63894]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: MinMaxThreshold not Generating Event"

To view the discussion, visit: http://community.zenoss.org/message/63904#63904

--------------------------------------------------------------
dpetzel:

Zenhub debug ;)


--Shane (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63904#63904]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: MinMaxThreshold not Generating Event"

To view the discussion, visit: http://community.zenoss.org/message/63917#63917

--------------------------------------------------------------

2012-01-25 12:12:42,486 DEBUG zen.zenjmx: Queueing event {'manager': 'ZenossServer', 'eventKey': 'My DataSource_DataPoint', 'device': 'monitored_node', 'eventClass': '/App/Conn', 'summary': 'threshold of Max_HTTP_Reqs_Sec exceeded: current value 11.61', 'component': '', 'monitor': 'localhost', 'agent': 'zenjmx', 'severity': 4}

Matching up the timestamp in that entry I found the following entries in zenhub.log

2012-01-25 12:12:43,357 DEBUG zen.Events: ===============  incoming event  ===============
2012-01-25 12:12:43,358 DEBUG zen.Events: Event info: {'firstTime': 1327522363.3579791, 'monitor': 'localhost', 'component': '', 'agent': 'zenjmx', 'summary': 'threshold of Max_HTTP_Reqs_Sec exceeded: current value 11.61', 'manager': 'ZenossServer', 'eventKey': 'My DataSource_DataPoint', 'device': 'monitored_node', 'eventClass': '/App/Conn', 'lastTime': 1327522363.3579791, 'message': 'threshold of Max_HTTP_Reqs_Sec exceeded: current value 11.61', 'severity': 4}
2012-01-25 12:12:43,359 DEBUG zen.Events: Found device monitored_node and adding device-specific data
2012-01-25 12:12:43,360 DEBUG zen.Events: Looking for event class named in event: /App/Conn
2012-01-25 12:12:43,362 DEBUG zen.Events: Per transform/mapping, using severity 4, action 'status' and clear classes []
2012-01-25 12:12:43,362 DEBUG zen.Events: *Applying transform/mapping at Event Class /Events*
2012-01-25 12:12:43,362 DEBUG zen.Events: Results after transform: <Products.ZenEvents.Event.Event instance at 0x243f49e0>
2012-01-25 12:12:43,362 DEBUG zen.Events: Dropping event
2012-01-25 12:12:43,362 DEBUG zen.Events: Unable to obtain event -- ignoring.(None)
#BE CAREFUL - This Transform is Applied at the Root
#As such it touches ALL events
    #If a Device is Down suppress all other messages
        #Don't drop the ping event, else we dont know its down
            evt._action = "drop"

Turns out a missing set of parenthesis on getPingStatusString was causing my grief. Oddly enough other events were still showing up (I'll leave that mystery for another day). Changing device.getPingStatusString to device.getPingStatusString() has resolve the issue and I am now seeing my threshold events
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/63917#63917]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]