Travis Balinas
2012-05-10 13:41:24 UTC
Travis Balinas [http://community.zenoss.org/people/tbalinas] created the discussion
"Hardware Monitoring: Syncing Drupal with Zenoss"
To view the discussion, visit: http://community.zenoss.org/message/66304#66304
--------------------------------------------------------------
Hi guys. I came across this post yesterday from some Zenoss alerts I have set up. After checking the forums, I didn't see anything like it so I figured a repost was in order. This is courtesy of Toni WestBrook from his personal site. Cheers!
*Overview*
One of the more daunting tasks of managing a larger network is keeping track of all your devices â both physically, and from a network monitoring perspective. When I arrived on the job 3 years ago, the first major task I laid down for myself was implementing both an asset management system, as well as a network monitoring system, to ensure we always knew what we had, and if it was functioning properly.
I decided almost immediately that http://www.drupal.org/ Drupal was the right candidate for the job of asset management. There are a number of commercial IT/helpdesk systems out there which work great, but they are usually fairly expensive with recurring licensing costs, and my history with them has always been shaky. Plus, I find myself not always using all the functionality I paid for. I knew with my Drupal experience, I could get something comparable up in almost no time â this is not a discredit to IT packages, but moreso the power of the Drupal framework.
*Network Monitoring â Cue Zenoss*
Now that I had the hardware DB taken care of, I needed a NMS for monitoring. Originally I was planning on Nagios, but a contractor who works for us (now friend) had introduced me to http://www.zenoss.org/ Zenoss, another open source alternative. Zenoss is awesome â is absolutely has its quirks, and is not the most intuitive system to learn, but once things are up and running itâs great â and tremendously powerful. So the choice was made.
Now â I had both pieces, but I absolutely hate entering data twice, and the interoperability guy in me loves integrating systems. So I decided to write a script that would sync our Drupal database with Zenoss. Drupal would serve as our master system, and any hardware we entered into it would automatically port over to Zenoss. Any changes or deletions we made (IP address, location, name, etc) would sync over as well.
The below script performs this synchronization. Some warnings up front â Iâm not a Python guy by any means, I specifically learned it for this script, so I apologize for any slopping coding or obvious Python-y mistakes. Iâve tried to thoroughly comment it to document how to use it and how it works. Hopefully it can help some others out as well!
http://www.toniwestbrook.com/archives/638#viewSource view source http://www.toniwestbrook.com/archives/638#printSource print http://www.toniwestbrook.com/archives/638#about ?
001.# Description: Sync devices to be monitored from Drupal to Zenoss
002.#
003.# Setup Work: Create a (or use an existing) content type that houses your hardware items to be monitored.
004.# They should have CCK fields for the IP address of the device, the name, and the type of
005.# device it is. The device type will determine the Zenoss class the script adds it to, and hence
006.# the kind of monitoring it will receive (e.g. Linux server, switch, ping only, etc)
007.#
008.# Additionally, in Zenoss, create a custom property field that will house the nid of the Drupal
009.# node. This serves as the foreign key and will be used to link the item in Drupal to its entry in Zenoss
010.#
011.# Usage: This script should be run from zendmd, and may be run once or periodically. We run it every 20 minutes from
012.# a cron job.
013.# It will create new entries in Zenoss for items not yet imported, delete ones that no longer exist in
014.# Drupal (it will only delete ones that were originally imported from Drupal), and will update ones that have
015.# been updated (type, IP, location, etc).
016.#
017.# Note: Excuse all the extra commits - we experienced some issues with data not being saved, and I threw some extra in
018.# there - they're almost definitely not necessaryimport MySQLdb
019.
020.# Take a taxonomy term from Drupal identifying the type of monitoring to be done,
021.# and convert it to the appropriate Zenoss class path. Update these to whatever terms
022.# and Zenoss class paths that make sense for your environment. We setup ones for
023.# Linux and Windows servers, switches, waps, UPSes, PDUes, etc, as can be seen.
024.def getClassPath(passType):
025.
026.if passType.lower() == "windows":
027.return "/Server/Windows"
028.elif passType.lower() == "linux":
029.return "/Server/Linux"
030.elif passType.lower() == "switch":
031.return "/Network/Switch"
032.elif passType.lower() == "mwap":
033.return "/Network/WAP/Managed"
034.elif passType.lower() == "uwap":
035.return "/Network/WAP/Unmanaged"
036.elif passType.lower() == "ups":
037.return "/Power/UPS"
038.elif passType.lower() == "pdu":
039.return "/Power/PDU"
040.elif passType.lower() == "camera":
041.return "/Camera"
042.elif passType.lower() == "cphone":
043.return "/Network/Telephone/Crash"
044.elif passType.lower() == "sphone":
045.return "/Network/Telephone/Standard"
046.elif passType.lower() == "printer":
047.return "/Printer"
048.elif passType.lower() == "converter":
049.return "/Network/Converter"
050.elif passType.lower() == "ping":
051.return "/Ping"
052.return "/Ping"
053.
054.# Connect to Drupal's MySQL DB (Replace these values with the appropriate ones for your system)
055.imsConn = MySQLdb.connect(DRUPAL_MYSQL_SERVER, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB)
056.imsCursor = imsConn.cursor()
057.
058.# Execute the query to grab all your items to be monitored. In our case, we have a node type called "hardware" that had CCK fields identifying the IP address,
059.# the type of hardware (a taxonomy term that dictated the Zenoss class of the item - see getClassPath above), a physical location, etc.
060.# You'll want to change the specific table/field names, but the inner join will probably stay, as you'll want to grab both the node and CCK fields that belong to it.
061.imsCursor.execute("""
062.SELECT node.nid, content_type_hardware.field_hardware_dns_value, content_type_hardware.field_hardware_location_value, content_type_hardware.field_hardware_ip_value, content_type_hardware.field_hardware_monitor_type_value, content_type_hardware.field_hardware_switchname_value, content_type_hardware.field_hardware_switchport_value
063.FROM node
064.INNER JOIN content_type_hardware ON node.nid = content_type_hardware.nid
065.""")
066.
067.# Loop through all returned records - Check for additions, changes, and removals
068.while (1):
069.#tempRow is our current hardware item record
070.tempRow = imsCursor.fetchone()
071.if tempRow == None:
072.# No more entries, break out of loop
073.break
074.else:
075.# Search Zenoss records for the nid of the hardware item. A custom field will need to be created in Zenoss to serve
076.# as this foreign key. In our case, we used MHTIMSID - but you can use anything you'd like - just be sure to create the field in Zenoss.
077.found = False
078.for d in dmd.Devices.getSubDevices():
079.if d.cMHTIMSID != "":
080.if int(d.cMHTIMSID) == tempRow[0]:
081.found = True
082.break
083.
084.if found == False:
085.# Hardware item not found, add it if it's monitored
086.if tempRow[4] != None:
087.dmd.DeviceLoader.loadDevice(("%s.yourdomain.com" % tempRow[1]).lower(), getClassPath(tempRow[4]),
088."", "", # tag="", serialNumber="",
089."", "", "", # zSnmpCommunity="", zSnmpPort=161, zSnmpVer=None,
090."", 1000, "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6]), # rackSlot=0, productionState=1000, comments="",
091."", "", # hwManufacturer="", hwProductName="" (neither or both),
092."", "", # osManufacturer="", osProductName="" (neither or both).
093."", "", "", #locationPath="",groupPaths=[],systemPaths=[],
094."localhost", # performanceMonitor="localhost',
095."none")
096.tempDevice = find(("%s.yourdomain.com" % tempRow[1]).lower())
097.tempDevice.setManageIp(tempRow[3])
098.commit()
099.# Save nid to Zenoss record (to serve as foreign key) for syncing
100.tempDevice._setProperty("cMHTIMSID","MHTIMS ID","string")
101.tempDevice.cMHTIMSID = tempRow[0];
102.commit()
103.else:
104.# Hardware item found - delete, update, or do nothing
105.if tempRow[4] == None:
106.# Delete if not set to monitor
107.dmd.Devices.removeDevices(d.id)
108.else:
109.# Update DNS and IP to current values
110.if d.getDeviceName() != ("%s.yourdomain.com" % tempRow[1]).lower():
111.d.renameDevice(("%s.yourdomain.com" % tempRow[1]).lower())
112.if d.getManageIp() != tempRow[3]:
113.d.setManageIp(tempRow[3])
114.commit()
115.
116.# Change class if not set to "Manual" (We setup a taxonomy term called "Manual" that would turn off automatic Zenoss class selection during syncing
117.# and allow us to manually specificy the class of the device.
118.if tempRow[4] != "Manual":
119.d.changeDeviceClass(getClassPath(tempRow[4]))
120.commit()
121.
122.# Update comments (location change)
123.d.comments = "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6])
124.commit()
125.
126.# Save any missed changes
127.commit()
128.
129.# Close connection to database
130.imsCursor.close()
131.imsConn.close()
http://www.toniwestbrook.com/archives/638 Original Post
--------------------------------------------------------------
Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/66304#66304]
Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]
"Hardware Monitoring: Syncing Drupal with Zenoss"
To view the discussion, visit: http://community.zenoss.org/message/66304#66304
--------------------------------------------------------------
Hi guys. I came across this post yesterday from some Zenoss alerts I have set up. After checking the forums, I didn't see anything like it so I figured a repost was in order. This is courtesy of Toni WestBrook from his personal site. Cheers!
*Overview*
One of the more daunting tasks of managing a larger network is keeping track of all your devices â both physically, and from a network monitoring perspective. When I arrived on the job 3 years ago, the first major task I laid down for myself was implementing both an asset management system, as well as a network monitoring system, to ensure we always knew what we had, and if it was functioning properly.
I decided almost immediately that http://www.drupal.org/ Drupal was the right candidate for the job of asset management. There are a number of commercial IT/helpdesk systems out there which work great, but they are usually fairly expensive with recurring licensing costs, and my history with them has always been shaky. Plus, I find myself not always using all the functionality I paid for. I knew with my Drupal experience, I could get something comparable up in almost no time â this is not a discredit to IT packages, but moreso the power of the Drupal framework.
*Network Monitoring â Cue Zenoss*
Now that I had the hardware DB taken care of, I needed a NMS for monitoring. Originally I was planning on Nagios, but a contractor who works for us (now friend) had introduced me to http://www.zenoss.org/ Zenoss, another open source alternative. Zenoss is awesome â is absolutely has its quirks, and is not the most intuitive system to learn, but once things are up and running itâs great â and tremendously powerful. So the choice was made.
Now â I had both pieces, but I absolutely hate entering data twice, and the interoperability guy in me loves integrating systems. So I decided to write a script that would sync our Drupal database with Zenoss. Drupal would serve as our master system, and any hardware we entered into it would automatically port over to Zenoss. Any changes or deletions we made (IP address, location, name, etc) would sync over as well.
The below script performs this synchronization. Some warnings up front â Iâm not a Python guy by any means, I specifically learned it for this script, so I apologize for any slopping coding or obvious Python-y mistakes. Iâve tried to thoroughly comment it to document how to use it and how it works. Hopefully it can help some others out as well!
http://www.toniwestbrook.com/archives/638#viewSource view source http://www.toniwestbrook.com/archives/638#printSource print http://www.toniwestbrook.com/archives/638#about ?
001.# Description: Sync devices to be monitored from Drupal to Zenoss
002.#
003.# Setup Work: Create a (or use an existing) content type that houses your hardware items to be monitored.
004.# They should have CCK fields for the IP address of the device, the name, and the type of
005.# device it is. The device type will determine the Zenoss class the script adds it to, and hence
006.# the kind of monitoring it will receive (e.g. Linux server, switch, ping only, etc)
007.#
008.# Additionally, in Zenoss, create a custom property field that will house the nid of the Drupal
009.# node. This serves as the foreign key and will be used to link the item in Drupal to its entry in Zenoss
010.#
011.# Usage: This script should be run from zendmd, and may be run once or periodically. We run it every 20 minutes from
012.# a cron job.
013.# It will create new entries in Zenoss for items not yet imported, delete ones that no longer exist in
014.# Drupal (it will only delete ones that were originally imported from Drupal), and will update ones that have
015.# been updated (type, IP, location, etc).
016.#
017.# Note: Excuse all the extra commits - we experienced some issues with data not being saved, and I threw some extra in
018.# there - they're almost definitely not necessaryimport MySQLdb
019.
020.# Take a taxonomy term from Drupal identifying the type of monitoring to be done,
021.# and convert it to the appropriate Zenoss class path. Update these to whatever terms
022.# and Zenoss class paths that make sense for your environment. We setup ones for
023.# Linux and Windows servers, switches, waps, UPSes, PDUes, etc, as can be seen.
024.def getClassPath(passType):
025.
026.if passType.lower() == "windows":
027.return "/Server/Windows"
028.elif passType.lower() == "linux":
029.return "/Server/Linux"
030.elif passType.lower() == "switch":
031.return "/Network/Switch"
032.elif passType.lower() == "mwap":
033.return "/Network/WAP/Managed"
034.elif passType.lower() == "uwap":
035.return "/Network/WAP/Unmanaged"
036.elif passType.lower() == "ups":
037.return "/Power/UPS"
038.elif passType.lower() == "pdu":
039.return "/Power/PDU"
040.elif passType.lower() == "camera":
041.return "/Camera"
042.elif passType.lower() == "cphone":
043.return "/Network/Telephone/Crash"
044.elif passType.lower() == "sphone":
045.return "/Network/Telephone/Standard"
046.elif passType.lower() == "printer":
047.return "/Printer"
048.elif passType.lower() == "converter":
049.return "/Network/Converter"
050.elif passType.lower() == "ping":
051.return "/Ping"
052.return "/Ping"
053.
054.# Connect to Drupal's MySQL DB (Replace these values with the appropriate ones for your system)
055.imsConn = MySQLdb.connect(DRUPAL_MYSQL_SERVER, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB)
056.imsCursor = imsConn.cursor()
057.
058.# Execute the query to grab all your items to be monitored. In our case, we have a node type called "hardware" that had CCK fields identifying the IP address,
059.# the type of hardware (a taxonomy term that dictated the Zenoss class of the item - see getClassPath above), a physical location, etc.
060.# You'll want to change the specific table/field names, but the inner join will probably stay, as you'll want to grab both the node and CCK fields that belong to it.
061.imsCursor.execute("""
062.SELECT node.nid, content_type_hardware.field_hardware_dns_value, content_type_hardware.field_hardware_location_value, content_type_hardware.field_hardware_ip_value, content_type_hardware.field_hardware_monitor_type_value, content_type_hardware.field_hardware_switchname_value, content_type_hardware.field_hardware_switchport_value
063.FROM node
064.INNER JOIN content_type_hardware ON node.nid = content_type_hardware.nid
065.""")
066.
067.# Loop through all returned records - Check for additions, changes, and removals
068.while (1):
069.#tempRow is our current hardware item record
070.tempRow = imsCursor.fetchone()
071.if tempRow == None:
072.# No more entries, break out of loop
073.break
074.else:
075.# Search Zenoss records for the nid of the hardware item. A custom field will need to be created in Zenoss to serve
076.# as this foreign key. In our case, we used MHTIMSID - but you can use anything you'd like - just be sure to create the field in Zenoss.
077.found = False
078.for d in dmd.Devices.getSubDevices():
079.if d.cMHTIMSID != "":
080.if int(d.cMHTIMSID) == tempRow[0]:
081.found = True
082.break
083.
084.if found == False:
085.# Hardware item not found, add it if it's monitored
086.if tempRow[4] != None:
087.dmd.DeviceLoader.loadDevice(("%s.yourdomain.com" % tempRow[1]).lower(), getClassPath(tempRow[4]),
088."", "", # tag="", serialNumber="",
089."", "", "", # zSnmpCommunity="", zSnmpPort=161, zSnmpVer=None,
090."", 1000, "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6]), # rackSlot=0, productionState=1000, comments="",
091."", "", # hwManufacturer="", hwProductName="" (neither or both),
092."", "", # osManufacturer="", osProductName="" (neither or both).
093."", "", "", #locationPath="",groupPaths=[],systemPaths=[],
094."localhost", # performanceMonitor="localhost',
095."none")
096.tempDevice = find(("%s.yourdomain.com" % tempRow[1]).lower())
097.tempDevice.setManageIp(tempRow[3])
098.commit()
099.# Save nid to Zenoss record (to serve as foreign key) for syncing
100.tempDevice._setProperty("cMHTIMSID","MHTIMS ID","string")
101.tempDevice.cMHTIMSID = tempRow[0];
102.commit()
103.else:
104.# Hardware item found - delete, update, or do nothing
105.if tempRow[4] == None:
106.# Delete if not set to monitor
107.dmd.Devices.removeDevices(d.id)
108.else:
109.# Update DNS and IP to current values
110.if d.getDeviceName() != ("%s.yourdomain.com" % tempRow[1]).lower():
111.d.renameDevice(("%s.yourdomain.com" % tempRow[1]).lower())
112.if d.getManageIp() != tempRow[3]:
113.d.setManageIp(tempRow[3])
114.commit()
115.
116.# Change class if not set to "Manual" (We setup a taxonomy term called "Manual" that would turn off automatic Zenoss class selection during syncing
117.# and allow us to manually specificy the class of the device.
118.if tempRow[4] != "Manual":
119.d.changeDeviceClass(getClassPath(tempRow[4]))
120.commit()
121.
122.# Update comments (location change)
123.d.comments = "%s (%s - %s)" % (tempRow[2], tempRow[5], tempRow[6])
124.commit()
125.
126.# Save any missed changes
127.commit()
128.
129.# Close connection to database
130.imsCursor.close()
131.imsConn.close()
http://www.toniwestbrook.com/archives/638 Original Post
--------------------------------------------------------------
Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/66304#66304]
Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]