Accessing event history causing the server to hang

[nilie]

nilie [http://community.zenoss.org/people/nilie] created the discussion

"Accessing event history causing the server to hang"

To view the discussion, visit: http://community.zenoss.org/message/68549#68549

--------------------------------------------------------------
Hello everybody,

I'm running Zenoss Core v3.1.0 rpm install on a CentOS v5.8 64-bit server and I've just discovered that I can no longer access event history. When ever I click the button the page becomes blank (only headers are being displayed) and after a while I get an error message saying  *"Unable to connect to server"*  and after that the whole GUI hangs. It usually takes a Zenoss restart to bring it back to life (I never had the patience to wait more than 10 min to see if it comes back by itself).
I couldn't see any event on the server that might give me a clue, nothing in Zenoss logs and Linux system logs, no error or warning in MySQL. I can access the database and I can see the info is there (over 3 million rows)

Any idea on what should I try next ?
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68549#68549]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[dpetzel]

dpetzel [http://community.zenoss.org/people/dpetzel] created the discussion

"Re: Accessing event history causing the server to hang"

To view the discussion, visit: http://community.zenoss.org/message/68578#68578

--------------------------------------------------------------
I was having a lot of issues with this myself recently. For me I got great results by following the advice of the mysql tuner: https://github.com/rackerhacker/MySQLTuner-perl https://github.com/rackerhacker/MySQLTuner-perl

Not sure if it will help, but might be worth a shot.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68578#68578]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[jcurry]

jcurry [http://community.zenoss.org/people/jcurry] created the discussion

"Re: Accessing event history causing the server to hang"

To view the discussion, visit: http://community.zenoss.org/message/68603#68603

--------------------------------------------------------------
Are you discarding old history events?  Navigate to EVENTS -> Event Manager and examine the parameters in the bottom section.  You should set the "Delete Historical Event s Older Than" parameter (if it is 0 then your history table will just grow forever).

Another utility I use is $ZENHOME/Products/ZenUtils/ZenDeleteHistory.py.  With this, you can be more selective about the events you delete, like for a device or for specific severity.  I tend to run a small cron job at night that backs up Zenoss and clears out history events of severity debug and info - it reduces the history taqble dramatically. Eg.
$ZENHOME/Products/ZenUtils/ZenDeleteHistory.py --numDays=1 \
  --severity=0 --severity=1 --severity=2  \
  --cleanup --optimize

Cheers,
Jane
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68603#68603]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[nilie]

nilie [http://community.zenoss.org/people/nilie] created the discussion

"Re: Accessing event history causing the server to hang"

To view the discussion, visit: http://community.zenoss.org/message/68613#68613

--------------------------------------------------------------
Thank you for your reply, Jane.
I used to keep history events for the last 93 days and reduced it to 63 days when I discovered this problem. I also tried to optimize the MySQL database as dpetzel suggested here but nothing solved my problem. I am now considering to  delete the event history database completely but first I will have to seek approval for this.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68613#68613]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]

[nilie]

nilie [http://community.zenoss.org/people/nilie] created the discussion

"Re: Accessing event history causing the server to hang"

To view the discussion, visit: http://community.zenoss.org/message/68729#68729

--------------------------------------------------------------
Jane,

What would be the command if I want to delete the whole history events from the database ? I tried optimizing MySQL configuration, tried to check the database for errors (there were none) but still no luck. Zenoss hangs everytime I try to access the event history so now I'm starting to consider this possiblity.

Thanks.
--------------------------------------------------------------

Reply to this message by replying to this email -or- go to the discussion on Zenoss Community
[http://community.zenoss.org/message/68729#68729]

Start a new discussion in zenoss-users by email
[discussions-community-forums-zenoss--***@community.zenoss.org] -or- at Zenoss Community
[http://community.zenoss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2003]