Customizing Email Notifications

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67941#67941

--------------------------------------------------------------
I'm using Zenoss 4.2 and monitoring AWS instances. All of my devices got imported automatically when I plugged in my AWS credentials to the EC2Manager plugin. However, it made their device id something like this -
| %1,3% Device ID: |
| %1,3% aws-i-252dsf5e |

But their device titles were populated like this: ec2-167-21-180-175.compute-1.amazonaws.com.

All fine and good, except for the fact that the email notifications use %(device) and that references the Device ID, and not teh device title. This is not helpful when I get an alert. I am able to rename the device with the Rename Device menu option in the lower left (by clicking the gear under Overview left hand pane for the device.). This changes the Device ID and then the alert has the name I want. But this seems to break the automatic linking between imported EC2 instances and their respective device objects. I really like that feature, and I'd be fine if I could just tell the email notifications to use something like %(deviceTitle) instead of the device ID. It seems silly that this isn't a notification variable. Also 'device group' should be a notification variable. Device ID's should
be able to be left alone, and the more meaningful, easily modified, and human readable 'Device Title' is what I want to see in my notifications.
Zenoss - please add one more little notification variable, pretty please!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67941#67941]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67979#67979

--------------------------------------------------------------
Jeremy:

You'll need to patch $ZENHOME/ZenModel/actions.py ~line 76 to add deviceTitle to the action context.

Good luck!

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67979#67979]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67995#67995

--------------------------------------------------------------
Thanks Shane. So I'm looking at the block of code that starts on line 76 in /opt/zenoss/Products/ZenModel/actions.py and I'm not really seeing anything that looks related directly to my issue. I'm on version 4.2.0 and this is what I see:

def _signalToContextDict(signal, zopeurl, notification=None, guidManager=None):
    summary = signal.event
    # build basic event context wrapper for notifications
    if signal.clear:
        # aged and closed events have clear == True, but they don't have an associated clear event
        #   spoof a clear event in those cases, so the notification messages contain useful info
        if summary.status == zep_pb2.STATUS_AGED:
            occur = signal.clear_event.occurrence.add()
            occur.summary = "Event aging task aged out the event."
            summary.cleared_by_event_uuid = "Event aging task"
        elif summary.status == zep_pb2.STATUS_CLOSED:
            occur = signal.clear_event.occurrence.add()
            occur.summary = "User '" + summary.current_user_name + "' closed the event in the Zenoss event console."
            summary.cleared_by_event_uuid = "User action"
        data = NotificationEventContextWrapper(summary, signal.clear_event)
    else:
        data = NotificationEventContextWrapper(summary)



I hate be overly needy, but can you give me another hint or 2 as to what exactly I'm looking to do here? Really appreciate it...

thanks,
jeremy

ps - whole file is attached
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67995#67995]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67987#67987

--------------------------------------------------------------
Jeremy:

My mistsake, I was looking at v4.1. For v4.2 it's around line 96. :)

You will see where the other variables are defined in this area.

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67987#67987]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67996#67996

--------------------------------------------------------------
Thanks again Shane. I just poured over that file pretty well, as well as some associated files that it imports from, but it's not jumping out at me how to accomplish this. My python-fu is pretty weak. I'll keep pondering on it though. Thanks.

Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67996#67996]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67998#67998

--------------------------------------------------------------
Are you really seeing "%(device)" somewhere in your Email Notification configuration? In 4.2, I would expect it would be using straight up tales. Can you paste your current notification configuration?


I would expect that you have something like "${evt/device}" in your body format field? If so, I think you could replace that with "${evt/device/title}".


That said, I have not done much of anything with email notifications, I'm basing this on the 4.2 admin guide and the code that Hackman referenced
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67998#67998]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68009#68009

--------------------------------------------------------------
dpetzel:

You're right, %(device} is a throwback to the olden days. *face palm*

Jeremy:
Regular old TALEs expressions will work as suggested. :) Give ${evt/device/title} a try.

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68009#68009]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/67999#67999

--------------------------------------------------------------
Awesome, I'll give that a shot now. I read the part in the admin guide about notification variables, and it really isn't clear that you can use sub-attributes like that. I thought they strictly had to be in the form of ${objectname/objectattribute}. Fingers crossed, thanks!

2.6.2.3.1. Notification Content Variables
Within the body of your email, page, and command notifications, you can specify information about the current
event, in the form:
'${objectname/objectattribute}'
Object names may be evt, evtSummary, or urls; or for clearing event context, clearEvt and clearEventSummary.
For each object name, the following lists show valid attributes (for example, '${evt/DevicePriority}'):
• evt/ and clearEvt/
• DevicePriority
• agent
• clearid
• component
• count
• created
• dedupid
• device
• eventClass
• eventClassKey
• eventGroup
• eventKey
• eventState
• evid
• facility
• firstTime
• ipAddressUsing Zenoss Core
• lastTime
• manager
• message
• ntevid
• ownerid
• priority
• prodState
• severity
• stateChange
• status
• summary
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/67999#67999]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68001#68001

--------------------------------------------------------------
Unfortunately, that didn't do the trick.
Here's what I have for my Message (Subject) Format under Notifications: *${evt/device/title} ${evt/summary}*
And for the notification body format, I have:

*Device: ${evt/device/title}*
*Device ID: ${evt/device}*
*Component: ${evt/component}*
*Severity: ${evt/severity}*
*Time: ${evt/lastTime}*
*Message:*
*${evt/message}*
*<a href="${urls/eventUrl}">Event Detail</a>*
*<a href="${urls/ackUrl}">Acknowledge</a>*
*<a href="${urls/closeUrl}">Close</a>*
*<a href="${urls/eventsUrl}">Device Events</a>*


And I just caused an alert to fire for a device with these parameters:
Device ID: ip-10-157-11-126.ec2.internal
Device Title: db.mysql.cv.02

The Device ID got set automatically when I added the device ( I added it by IP). The Device title was also set automatically and matched the device ID. I edited the Device title to our meaningful format of db.mysql.cv.02, but left the Device ID as is.
I then triggered the alert, and it came in with the subject of:
Ip-10-157-11-126.Ec2.Internal Process not running: /usr/sbin/mysqld

*And a body of:*
Device: Ip-10-157-11-126.Ec2.Internal
Device ID: ip-10-157-11-126.ec2.internal
Component: usr_sbin_mysqld eade5654a687989abac541f643400ebb
Severity: 4
Time: 2012/08/19 10:55:37.000
Message:
Process not running: /usr/sbin/mysqld
   Using regex 'sbin\/mysqld'
   All Processes have stopped since the last model occurred. Last Modification time (2012/08/19 10:43:24)
Event Detail: http://zenoss-01.branchout.com:8080/zport/dmd/Events/viewDetail?evid=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f http://zenoss-01.branchout.com:8080/zport/dmd/Events/viewDetail?evid=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f
Acknowledge: http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_ackEvents?evids=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f&zenScreenName=viewEvents http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_ackEvents?evids=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f&zenScreenName=viewEvents
Close: http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_deleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f&zenScreenName=viewHistoryEvents http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_deleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea2714bd4d1f&zenScreenName=viewHistoryEvents
Device Events: http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail#deviceDetailNav:device_events http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail#deviceDetailNav:device_events

So you can see that the variables
*${evt/device/title}* and *${evt/device}* are producing exactly the same content in the email, except that *${evt/device/title}* appears to camel case the device name, whereas *${evt/device}* leaves it all lower case.

Drat.. Any other ideas?

Thanks,
Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68001#68001]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68010#68010

--------------------------------------------------------------
I can't seem to get to the SVN repo at the moment to take a look at the 4.x code, but looking at the 3.2.1 device.py, can you try *${evt/device/name}*
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68010#68010]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68002#68002

--------------------------------------------------------------
Thanks DP. I tried substituting in *${evt/device/name}* as the variable in the notification subject and it seemed to generate an exception that caused the email alert not to go out. Here's the event that got logged. I've got it switched back for now.. Hoping there is a variable somewhere that can give me that title.. Seems really surprising that it's so hard to find when it should be listed in the admin guide along with the others. This seems like it would be a really common scenario.

Thanks,
Jeremy

Resource:localhostComponent:zenactiondEvent Class: http://zenoss-01.branchout.com:8080/zport/dmd/Events/App/Failed /App/FailedStatus:NewMessage:Traceback (most recent call last): File "/opt/zenoss/Products/ZenModel/actions.py", line 244, in execute self.executeBatch(notification, signal, targets) File "/opt/zenoss/Products/ZenModel/actions.py", line 293, in executeBatch subject = processTalSource(notification.content['subject_format'], **data) File "/opt/zenoss/Products/ZenModel/actions.py", line 73, in processTalSource return talEval(sourceStr, context, kwargs) File "/opt/zenoss/Products/ZenUtils/ZenTales.py", line 78, in talEval return talesEvalStr(expression, context, extra) File "/opt/zenoss/Products/ZenUtils/ZenTales.py", line 28, in talesEvalStr return talesEval('string:%s' % expression, context, extra) File "/opt/zenoss/Products/ZenUtils/ZenTales.py", line 48, in talesEval raise InvalidTalesException(msg) InvalidTalesException: Error when processing tales expression string: ${evt/device/name} ${evt/summary} on context {'evt': <Products.ZenModel.NotificationSubscription.NotificationEventSummaryProxy object at 0x77ea710>, 'trigger': {'name': 'prod'}, 'urls': {'ackUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_ackEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewEvents http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_ackEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewEvents', 'reopenUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_undeleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewEvents http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_undeleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewEvents', 'eventUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Events/viewDetail?evid=12313d1c-44bc-b6f2-11e1-ea3667de4984 http://zenoss-01.branchout.com:8080/zport/dmd/Events/viewDetail?evid=12313d1c-44bc-b6f2-11e1-ea3667de4984', 'baseUrl': ' http://zenoss-01.branchout.com:8080 http://zenoss-01.branchout.com:8080', 'closeUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_deleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewHistoryEvents http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_deleteEvents?evids=12313d1c-44bc-b6f2-11e1-ea3667de4984&zenScreenName=viewHistoryEvents', 'deviceUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail', 'eventsUrl': u' http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail#deviceDetailNav:device_events http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/ip-10-157-11-126.ec2.internal/devicedetail#deviceDetailNav:device_events'}, 'clearEvt': {}, 'notification': {'name': 'email'}, 'clearEventSummary': {}, 'eventSummary': <zenoss.protocols.wrappers.EventSummaryAdapter object at 0x77eaa50>} : Exception Class <class 'zope.location.interfaces.LocationError'> Message: (u'ip-10-157-11-126.ec2.internal', 'name')
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68002#68002]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68003#68003

--------------------------------------------------------------
hmm. I still cant get to the SVN site... Lets see if we can't dig this out from another direction..

SSH into your zenoss server, and become the zenoss user via `su - zenoss`
then type `zendmd` and paste the following:

d = dmd.Devices.findDevice('ip-10-157-11-126.ec2.internal')
print getattr(d, 'id', None)
print getattr(d, 'title', None)
print d.name()

I've not dealt much with the events in 4.x yet, so sorry about the fact finding missing here, but as far as I know you you can traverse the object path in the tales expressions, so evt/device should be a device object, so we should be able to use any valid attributes of the device. I can see name() in the 3.2.1 code, and I would assume its there in 4.x, but I dont have a copy of the code handy to confirm.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68003#68003]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68004#68004

--------------------------------------------------------------
Here's what I got as the output of zendmd:

[***@zenoss-01 ~]$ zendmd
Welcome to the Zenoss dmd command shell!
'dmd' is bound to the DataRoot. 'zhelp()' to get a list of commands.
Use TAB-TAB to see a list of zendmd related commands.
Tab completion also works for objects -- hit tab after an object name and '.'
(eg dmd. + tab-key).

Post by dpetzel
d = dmd.Devices.findDevice('ip-10-157-11-126.ec2.internal')
print getattr(d, 'id', None)

ip-10-157-11-126.ec2.internal

Post by dpetzel
print getattr(d, 'title', None)

db.mysql.cv.02

Post by dpetzel
print d.name()

db.mysql.cv.02
If there's a .py file that I can attach for your evaluation, just let me know. I've attached /opt/zenoss/Products/Zuul/infos/device.py in hopes that it might help.
Thanks much!

Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68004#68004]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68021#68021

--------------------------------------------------------------
Well trac is back up now, but thanks for the attachment. What is interesting is that name and title both return the desired output when run via DMD.....
Its wierd that name produces a traceback, but title works (but returns the wrong data)... I'm a little baffled by why ${evt/device/title} doesnt work....

If this is a test system, we could try some hackery, that I'm almost positive is the *wrong* approach, but *might* work..Its VERRRYY untested..

data['deviceName'] = getattr(dmd.Devices.findDevice(data[device]),'title', "Unknown")

Restart your Zenoss Stack.

Then in your notification configuration replace ${evt/device/title} with ${evt/deviceName/}


I'm pretty sure, there is a way to do this direct in tales without hacking this code, but I'm just not somewhere with a 4.x installation. Later in the week
I can check for realz...
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68021#68021]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68022#68022

--------------------------------------------------------------
It is strange about the dmd output being correct. I'll try using $(evt/device/title) again just to be sure. I also tried adding that snippet at line 103, and it also generated a traceback. That'd be awesome if you're able to have a look at a 4.x install and see something I'm not seeing here. Thanks again.. Here's the traceback from that last try:

| Resource: | localhost |
| Component: | zenactiond |
| Event Class: | /App/Failed (http://zenoss-01.branchout.com:8080/zport/dmd/Events/App/Failed) |
| Status: | New |
| Message: | Traceback (most recent call last): File "/opt/zenoss/Products/ZenModel/actions.py", line 245, in execute self.executeBatch(notification, signal, targets) File "/opt/zenoss/Products/ZenModel/actions.py", line 288, in executeBatch data = _signalToContextDict(signal, self.options.get('zopeurl'), notification, self.guidManager) File "/opt/zenoss/Products/ZenModel/actions.py", line 103, in _signalToContextDict data['deviceName'] = getattr(dmd.Devices.findDevice(data[device]),'title', "Unknown") NameError: global name 'dmd' is not defined |
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68022#68022]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68083#68083

--------------------------------------------------------------
Jeremy:

Are you using $(evt/device/title) or ${evt/device/title}? You must use curly brackets for tales.

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68083#68083]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68085#68085

--------------------------------------------------------------
Shane,
I'm definitely using curly brackets. Thanks.

Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68085#68085]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68094#68094

--------------------------------------------------------------
Jeremy:

Can you post the subject and body of your notification?

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68094#68094]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68077#68077

--------------------------------------------------------------
Gladly. Here is the subject format:
${evt/device/title} ${evt/summary}

And here is the body format:

Device Title: ${evt/device/title}
Device ID: ${evt/device}
Component: ${evt/component}
Severity: ${evt/severity}
Time: ${evt/lastTime}
Message:
${evt/message}
<a href="${urls/eventUrl}">Event Detail</a>
<a href="${urls/ackUrl}">Acknowledge</a>
<a href="${urls/closeUrl}">Close</a>
<a href="${urls/eventsUrl}">Device Events</a>


I only have one trigger and one corresponding notification. Does the above look ok to you? Are you able to get accurate device titles in your notifications?
Thanks again.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68077#68077]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68105#68105

--------------------------------------------------------------
Jeremy:

This is what I used in a test version of v4.2 and it worked okay.

Subject:
[zenoss] ${evt/device/title} ${evt/summary}

Body:
Device: ${evt/device/title}
Component: ${evt/component}
Severity: ${evt/severity}
Time: ${evt/lastTime}
Message:
${evt/message}
<a href="${urls/eventUrl}">Event Detail</a>
<a href="${urls/ackUrl}">Acknowledge</a>
<a href="${urls/closeUrl}">Close</a>
<a href="${urls/eventsUrl}">Device Events</a>

It doesn't differ from what you have hardly at all.

Can you `su zenoss` then `zenactiond debug` and post the log after the notification traceback occurs?

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68105#68105]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68080#68080

--------------------------------------------------------------
Yes, I'll do that and post the results. So just to be clear, you have a
device with a title that differs from your device id and you are able to
use either one and get the correct output? Are there any patches you had to
install, or that I should try installing?
I'll post the output of that log later today. Thanks.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68080#68080]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68108#68108

--------------------------------------------------------------
Jeremy:

Yep. Its v4.2.0-1582.el5.

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68108#68108]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68113#68113

--------------------------------------------------------------
Shane,
Sorry, this is probably an easy one but I'm not seeing it. What's the
easiest way to confirm that I'm on the same version as you? I'm looking
under versions in Advanced, and I just see 4.2.0.

Zenoss < http://www.zenoss.com/ http://www.zenoss.com/>Zenoss 4.2.0OS < http://www.tldp.org/ http://www.tldp.org/>Linux
(x86_64) 2.6.32 (Linux zenoss-01.branchout.com 2.6.32-220.2.1.el6.x86_64 #1
SMP Fri Dec 23 02:21:33 CST 2011 x86_64)Zope < http://www.zope.org/ http://www.zope.org/>Zope
2.13.13Python < http://www.python.org/ http://www.python.org/>Python
2.7.2Database< http://www.mysql.com/ http://www.mysql.com/>MySQL
5.5.27 (Ver 5.5.27)RRD < http://oss.oetiker.ch/rrdtool http://oss.oetiker.ch/rrdtool>RRDtool
1.4.7Twisted< http://twistedmatrix.com/trac http://twistedmatrix.com/trac>Twisted
11.0.0RabbitMQ < http://www.rabbitmq.com/ http://www.rabbitmq.com/>RabbitMQ
2.8.4Erlang< http://www.erlang.org/ http://www.erlang.org/>Erlang
5.8.5NetSnmp < http://net-snmp.sourceforge.net/ http://net-snmp.sourceforge.net/>NetSnmp
5.5.0PyNetSnmp< http://www.zenoss.com/ http://www.zenoss.com/>PyNetSnmp
0.30.3WMI < http://www.zenoss.com/ http://www.zenoss.com/>Wmi 1.3.15
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68113#68113]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68110#68110

--------------------------------------------------------------
Jeremy:

rpm -qa zenoss

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68110#68110]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68126#68126

--------------------------------------------------------------
Thanks Shane. So it looks like I am just slightly ahead in version, but the big difference is that I'm on CentOS 6.2, and it looks like you are on 5.x. You think that could be a culprit? Willing to try anything here, including rebuilding my server on centos 5.x.  Thoughts?

Thanks,
Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68126#68126]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68127#68127

--------------------------------------------------------------
Shane,
OK, I set zenactiond logging to debug and triggered an alert on a box by stopping mysql. The Device ID for this box is: db.mysql.cv.02-id2
And the device title for the same box is: db.mysql.cv.02-title

And here's the email I got from Zenoss:
Subject: Db.Mysql.Cv.02-Id2 Process not running: /usr/sbin/mysqld

Body:
Device Title: Db.Mysql.Cv.02-Id2
Device ID: db.mysql.cv.02-id2
Component: usr_sbin_mysqld eade5654a687989abac541f643400ebb
Severity: 4
Time: 2012/08/23 14:11:49.000
Message:
Process not running: /usr/sbin/mysqld
etc etc etc..

You can see it's using the device ID instead of the title based on the subject and body format I posted earlier. And here's the chunk of the zenactiond.log that correlates withe the event:


2012-08-23 14:13:04,271 DEBUG zen.zenoss.protocols.amqp: Publishing with routing key zenoss.heartbeat.localhost to exchange zenoss.heartbeats
2012-08-23 14:13:04,272 DEBUG zen.maintenance: Rescheduling maintenance in 60s
2012-08-23 14:13:44,223 DEBUG zen.zenactiond: processing message.
2012-08-23 14:13:44,225 DEBUG zen.notifications: Notification is enabled: email
2012-08-23 14:13:44,225 DEBUG zen.notifications: Notification is enabled, but has no windows, it is active.
2012-08-23 14:13:44,225 DEBUG zen.zenactiond: Found matching notification: <NotificationSubscription at email>
2012-08-23 14:13:44,225 DEBUG zen.zenactiond: Found these matching notifications: [<NotificationSubscription at /zport/dmd/NotificationSubscriptions/email>]
2012-08-23 14:13:44,226 DEBUG zen.actions: Executing batch action for targets.
2012-08-23 14:13:44,226 DEBUG zen.actions: Executing Email action for targets: set(['***@branchout.pagerduty.com'])
2012-08-23 14:13:44,229 DEBUG zen.actions: Sending this subject: Db.Mysql.Cv.02-Id2 No performance data from plugin
2012-08-23 14:13:44,229 DEBUG zen.actions: Sending this body:
Device Title: Db.Mysql.Cv.02-Id2
Device ID: db.mysql.cv.02-id2
Component: tcp_03306
Severity: 4
Time: 2012/08/23 14:13:43.000
Message:
MySQL Error: (2003, "Can't connect to MySQL server on '10.157.11.126' (111)")


<a href="http://zenoss-01.branchout.com:8080/zport/dmd/Events/viewDetail?evid=12313d1c-44bc-9699-11e1-ed676b1b7025">Event Detail</a>
<a href="http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_ackEvents?evids=12313d1c-44bc-9699-11e1-ed676b1b7025&zenScreenName=viewEvents">Acknowledge</a>
<a href="http://zenoss-01.branchout.com:8080/zport/dmd/Events/manage_deleteEvents?evids=12313d1c-44bc-9699-11e1-ed676b1b7025&zenScreenName=viewHistoryEvents">Close</a>
<a href="http://zenoss-01.branchout.com:8080/zport/dmd/Devices/Server/Linux/MySQL/devices/db.mysql.cv.02-id2/devicedetail#deviceDetailNav:device_events">Device Events</a>


2012-08-23 14:13:44,279 DEBUG zen.actions: Notification 'email' sent emails to: set(['***@branchout.pagerduty.com'])
2012-08-23 14:13:44,280 INFO zen.zenactiond: Event:'db.mysql.cv.02-id2|tcp_03306|/App/MySQL|mysql|4' Trigger:prod Action:email Status:SUCCESS Target:***@branchout.pagerduty.com Info:<NotificationSubscription Info "email">
2012-08-23 14:13:44,280 DEBUG zen.zenactiond: Done processing signal. (MySQL Error: (2003, "Can't connect to MySQL server on '10.157.11.126' (111)")
)
2012-08-23 14:13:44,280 DEBUG zen.zenactiond: Done processing signal.
2012-08-23 14:13:44,280 DEBUG zen.zenactiond: Acknowledging message. (MySQL Error: (2003, "Can't connect to MySQL server on '10.157.11.126' (111)")



It really looks like a bug to me at this point. I have my notification confingured pretty much exactly like yours, and it just refuses to use the device title. It does treat the ID and title slightly differently in that it camel-cases the device title, but leaves the device ID all lower case. But the string it's using for device title is clearly the device ID. Any other things to try on this? I'm thinking I just just rebuild a box on centos 5 so I can move past this. Seems minor but I really need device title to work, or else I will have a lot of renaming of device ID's to do, which then breaks the link between the actual zenoss device and the EC2Manager instance which is really nice to have. Thanks again..
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68127#68127]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68131#68131

--------------------------------------------------------------
Jeremy:

Yeah this looks like a regression. Can you post it in Jira for the devs to look at?

Best,
--Shane Scott (Hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68131#68131]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Jeremy Koerber]

Jeremy Koerber [http://community.zenoss.org/people/jeremykoerber] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68118#68118

--------------------------------------------------------------
Thanks Shane. I detailed that in Jira here:
http://jira.zenoss.com/jira/browse/ZEN-3191 http://jira.zenoss.com/jira/browse/ZEN-3191

Let me know if there's anything else I can do..

Thanks,
Jeremy
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68118#68118]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Insane_Homer]

Insane_Homer [http://community.zenoss.org/people/Insane_Homer] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/71251#71251

--------------------------------------------------------------
Clean install on 4.2.3 on Centos.

I used a Network Discovery to find most of my kit.

Renamed it via the : *"Device Title"*

Event Notification results the IP address being displayed twice. I don' t really want to rename each device manually again to Change the *Device ID*.

Device Title: ${evt/device/title} Appears to return Device ID or IP not the device title.

Does not return the correct detail required.

what is the syntax to return *"Device Title"*?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71251#71251]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Philip Warren]

Philip Warren [http://community.zenoss.org/people/pwarren] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/71252#71252

--------------------------------------------------------------
Can you try using:

${evtSummary/actor/element_title}

-- Philip
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71252#71252]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[digital-cipher]

digital-cipher [http://community.zenoss.org/people/digital-cipher] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68275#68275

--------------------------------------------------------------
Hi all

I have the same problem when trying to use Device: ${evt/device/title}
I get the device id and not the device title

OS = Centos 6 - 64bit
Zenoss version = 4.2

Device: ${evt/device/title}
Device ID: ${evt/device}
Component: ${evt/component}
Severity: ${evt/severity}
Time: ${evt/lastTime}
Message: ${evt/message}

I would like to get the email notifications to show both the device title and device location

PS: Sorry for jumping in on this post
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68275#68275]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/68291#68291

--------------------------------------------------------------
digital-cipher, can you try the event transform workaround I discussed here: http://community.zenoss.org/message/68226#68226 http://community.zenoss.org/message/68226#68226

I think that will get you what you need until the root issue is resolved.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68291#68291]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Insane_Homer]

Insane_Homer [http://community.zenoss.org/people/Insane_Homer] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/71253#71253

--------------------------------------------------------------
Thanks Philip,

*${eventSummary/actor/element_title}* did the trick!
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71253#71253]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[bukhary ikhwan ismail]

bukhary ikhwan ismail [http://community.zenoss.org/people/bukharyi] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/71355#71355

--------------------------------------------------------------
Thank you very much!. I had this issue when I deployed stock zenoss 4.2. Not sure why, but my previous installation never had this problem.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/71355#71355]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[David Holloway]

David Holloway [http://community.zenoss.org/people/dmrholloway] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/72657#72657

--------------------------------------------------------------
is there way for the email to tell device class in its email notifications.
i have all the switches in the switch class and routers in the router class and firewalls in the firewall class,
they are all being monitored by other each group.
So if the the switch goes down, it only goes the the switch group, same for the other classes
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72657#72657]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Shane Scott]

Shane Scott [http://community.zenoss.org/people/hackman238] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/72659#72659

--------------------------------------------------------------
David,

Using an event transform you can append the devices device class to the evt.message.

Best,
--Shane Scott (hackman238)
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/72659#72659]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[pwinkeler]

pwinkeler [http://community.zenoss.org/people/pwinkeler] created the discussion

"Re: Customizing Email Notifications"

To view the discussion, visit: http://community.zenoss.org/message/74712#74712

--------------------------------------------------------------
Whoa - exactly what I was looking for!  Is this something that got fixed in 4.2.4?
Also, where can one find a list of all other data elements under eventSummary?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/74712#74712]

Start a new discussion in zenoss-users at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]