zendevicedump won't dump

[mhaswell]

mhaswell [http://community.zenoss.org/people/mhaswell] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62321#62321

--------------------------------------------------------------
I am having the same issue with zendevicedump. It used to work - then we got someone to labour over the zenoss install (v3.03) adding all our systems and now we can't devicedump (which is how we were going to sync the setup to the backup zenoss).

Does anyone have any other way to do this? Despite it being both ubuntu and same version of zenoss a backup (without events or performance history) just freezes the backup system if we try to install it. So zendevicedump wasour preferred way until this came up.

Ok - looking through the forums and googling around I see that no one has answers. Can anyone think of a way to find the rogue, possibly non-UTF-8 character, device?

Matt
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62321#62321]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Ryan Matte]

Ryan Matte [http://community.zenoss.org/people/rmatte] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62361#62361

--------------------------------------------------------------
The last time I had something like this happen it turned out that someone had copied/pasted something in to the "Comments" field of one of my devices from Microsoft Word, and it brought along some unfriendly characters with it as well.  I eventually found it and removed the offending character.  You guys probably had something similar happen somewhere.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62361#62361]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mhaswell]

mhaswell [http://community.zenoss.org/people/mhaswell] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62426#62426

--------------------------------------------------------------
I agree that it's probably what has happened - I bet excel spreadsheets were involved somewhere in this.

Does anyone have any way to dump the database and to look for non UTF-8 characters though? Or an alteration to zendevicedump to list the device it's on for every device so at least I know which one it is crashing on?

Matt
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62426#62426]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Ryan Matte]

Ryan Matte [http://community.zenoss.org/people/rmatte] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62417#62417

--------------------------------------------------------------
As a starting point, edit /usr/lib/python2.4/site-packages/_xmlplus/dom/expatbuilder.py, fine line 223.  Make the following modification.

Change this section of code:

try:
    parser.Parse(string, True)
    self._setup_subset(string)


To look like:

try:
    parser.Parse(string, True)
    self._setup_subset(string)
    print "STRING: %s" % (string)


Then run zendevicedump.  This may give you more info as to exactly what it's failing on.  Make sure you remove the line when you're done.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62417#62417]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mhaswell]

mhaswell [http://community.zenoss.org/people/mhaswell] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62429#62429

--------------------------------------------------------------
Unfortunately our system is a 3.03 system so it uses Python 2.6 ...  I found what I thought was a similar bit but my poor knowledge of python has probably buggered it.

I found the file:
/usr/lib/python2.6/xml/dom/expatbuilder.py

Inside I found the following function:

| | def parseFile(self, file): |
|     | """Parse a document from a file object, returning the document |
|     | node.""" |
|     | parser = self.getParser() |
|     | first_buffer = True |
|     | try: |
|         | while 1: |
|             | buffer = file.read(16*1024) |
|             | if not buffer: |
|                 | break |
|             | parser.Parse(buffer, 0) |
|             | if first_buffer and self.document.documentElement: |
|                 | self._setup_subset(buffer) |
|             | first_buffer = False |
|         | parser.Parse("", True) |
|     | except ParseEscape: |
|         | pass |
|     | doc = self.document |
|     | self.reset() |
|     | self._parser = None |
|     | return doc |


So I added

print "STRING: %s" % (buffer)

after

self._setup_subset(buffer)

but it doesn't do anything. Can anyone with (any) better grasp of python help out?

Outputs:

***@rnln-unixmon01:~$ zendevicedump -o /tmp/t.txt
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 229, in <module>
    ex.export()
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 215, in export
    doc = parseString(buffer.getvalue())
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString
    return expatbuilder.parseString(string)
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 942, in parseString
    return builder.parseString(string)
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 134784, column 0
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62429#62429]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Ryan Matte]

Ryan Matte [http://community.zenoss.org/people/rmatte] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62418#62418

--------------------------------------------------------------
You edited the wrong file, there are multiple expatbuilder.py files on the system.  You need to edit the specific one that I told you to edit since that's the one that was in your traceback.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62418#62418]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mhaswell]

mhaswell [http://community.zenoss.org/people/mhaswell] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62431#62431

--------------------------------------------------------------
I think you are confusing me with "psyber" the first user who posted in this thread - they obviously use a python 2.4 system - so I assume a 2.x zenoss implementation. However our implementation is 3.03 and uses python2.6.

However I missed the fact that the traceback I posted mentioned another expatbuilder at
/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py
so my fault!

        """Parse a document from a string, returning the document node."""
        parser = self.getParser()
*        try:*
*            parser.Parse(string, True)*
*            self._setup_subset(string)*
            pass
        doc = self.document
        self.reset()
        self._parser = None
        return doc
        """Parse a document from a string, returning the document node."""
        parser = self.getParser()
            parser.Parse(string, True)
            self._setup_subset(string)
*            print "STRING: %s" % (string)*
            pass
        doc = self.document
        self.reset()
        self._parser = None
        return doc

So the output is:

a bunch of stuff that looks like it's from the dashboard and help documents and then the last STRING is:

STRING: <?xml version="1.0" encoding="UTF-8"?>
<view>

<!-- Overview buttons -->

  <tooltip>
    <target></target>

    <html>Save</html>
  </tooltip>

  <tooltip>
    <target></target>

    <html>Cancel</html>
  </tooltip>

<!-- Events buttons -->

  <tooltip>
    <target></target>

    <html>Acknowledge event</html>
  </tooltip>


  <tooltip>
    <target></target>

    <html>Close event</html>
  </tooltip>


  <tooltip>
    <target></target>

    <html>Go to event console</html>
  </tooltip>


  <tooltip>
    <target></target>

    <html>View events</html>
  </tooltip>

<!-- bottom of page buttons -->

  <tooltip>
    <target>commands-menu</target>
    <anchor>top</anchor>

    <html>Run commands</html>
  </tooltip>

<tooltip>
    <target>device_configure_menu</target>
    <anchor>top</anchor>

    <html>Actions</html>
</tooltip>

<tooltip>
    <target>component-add-menu</target>
    <anchor>top</anchor>

    <html>Add components</html>
</tooltip>

<!-- Modeler Plugins buttons -->

<tooltip>
   <target></target>

   <html>Save</html>
</tooltip>

<tooltip>
   <target></target>

   <html>Delete</html>
</tooltip>

<!-- Custom buttons -->

<tooltip>
   <target></target>

   <html>Save</html>
</tooltip>

<!-- Monitoring Templates buttons -->

<tooltip>
    <target>datasourceAddButton</target>

    <html>Add data source</html>
</tooltip>

<tooltip>
    <target>datasourceDeleteButton</target>

    <html>Remove data source</html>
</tooltip>

<tooltip>
    <target>datasourceEditButton</target>

    <html>Data source edit options</html>
</tooltip>

<!-- thresholds -->
<tooltip>
    <target>thresholdAddButton</target>

    <html>Add threshold</html>
</tooltip>

<tooltip>
    <target>thresholdDeleteButton</target>

    <html>Delete threshold</html>
</tooltip>

<tooltip>
    <target>thresholdEditButton</target>

    <html>View and edit threshold details </html>
</tooltip>

<!-- graph definition -->
<tooltip>
    <target>addGraphDefinitionButton</target>

    <html>Add graph definition</html>
</tooltip>

<tooltip>
    <target>deleteGraphDefinitionButton</target>

    <html>Delete graph definition</html>
</tooltip>

<tooltip>
    <target>graphDefinitionMenuButton</target>

    <html>Manage graph definition</html>
</tooltip>

<!-- graph points -->
<tooltip>
    <target>addGraphPointButton</target>

    <html>Add graph point</html>
</tooltip>

<tooltip>
    <target>deleteGraphPointButton</target>

    <html>Delete graph point</html>
</tooltip>

<tooltip>
    <target>editGraphPointButton</target>

    <html>View and edit graph point details</html>
</tooltip>



</view>

Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 229, in <module>
    ex.export()
  File "/usr/local/zenoss/zenoss/Products/ZenRelations/ExportDevices.py", line 215, in export
    doc = parseString(buffer.getvalue())
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/minidom.py", line 1925, in parseString
    return expatbuilder.parseString(string)
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 943, in parseString
    return builder.parseString(string)
  File "/usr/local/zenoss/python/lib/python2.6/site-packages/_xmlplus/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 134784, column 0



Does this mean the problem is in the dashboard setup? Or is it in the graphing? Or did it just not get to the nodes?

Matt
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62431#62431]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Ryan Matte]

Ryan Matte [http://community.zenoss.org/people/rmatte] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62419#62419

--------------------------------------------------------------
That's bad logic on my part, the print line should come before the line it's failing on not after, move the print line to directly under the def parseString(self, string): line and give it another go.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62419#62419]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[mhaswell]

mhaswell [http://community.zenoss.org/people/mhaswell] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62466#62466

--------------------------------------------------------------
Thank you so much for the help!

Yes - putting the print line after the "def parseString" line worked well with spitting out devices - unfortunately it then went right through a perfect zendevicedump!

All we can think is that the 1 device we deleted since I tried 4 days ago was, by a lucky coincidence, exactly the problematic device!

Hopefully these comments can help other people with the same issue though,

Many thanks again for the help,

Matt
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62466#62466]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[Ryan Matte]

Ryan Matte [http://community.zenoss.org/people/rmatte] created the discussion

"Re: zendevicedump won't dump"

To view the discussion, visit: http://community.zenoss.org/message/62455#62455

--------------------------------------------------------------
No problem, cheers.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/62455#62455]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]