November 2015 - Xymon - Xymon mailman web

server-side message pre-processor?
by john.thurston＠alaska.gov 04 Nov '15

04 Nov '15

Is there a mechanism through which I can dump certain host+test combinations to /dev/null ? By way of example, let's hypothetically say I had a set of hosts whose owner steadfastly refused to clean up their data streams. They might be sending thousands of garbage test results to my server. The xymon server might be defined to not propagate the test results to the non-green screen, and there may be no alerts defined for the host+test combinations. But the xymon server is still passing thousands of garbage messages through its parsing, evaluating, graphing, and alerting handlers. Xymongen is still including these ignored messages in the list of "events received" at the bottom of the non-green page. It would be cool if there were a way to create a flush-list for which: This host+test message combination will always be flushed. It will never get past the parsing handler. It won't be recorded. It wont be reported. It will not pass "go". It will not collect $200. It will just disappear. Alternatively, the ability to prevent a host+test combination appearing on the "events received int the past x minutes" portion of the non-green page would be a big help. In the hypothetical situation outlined above, the thousands of messages from those hosts would push every other meaningful message off the bottom of that list of events. -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

6 13

Xymon 4.3.22-beta Release Download
by john.thurston＠alaska.gov 04 Nov '15

04 Nov '15

On 10/29/2015 5:44 PM, J.C. Cleaver wrote: > Hello all! > > Xymon-4.3.22-beta has been posted to Sourceforge, containing bug fixes and > a smattering of new features and improvements since 4.3.21. It's reachable > at > http://sourceforge.net/projects/xymon/files/Xymon/4.3.22/xymon-4.3.22-beta.… I have another 24 behind me on 4.3.22-RC2 on Solaris. It is interacting well with my 4.3.21 (Solaris) and 4.3.17 (CentOS) instances. I've updated some of my production http tests to handle the change in behavior for 301 responses. The no-showemptygroups option was needed on my xymongen to regain the previous behavior. I've deleted some new .sh scripts from cgi-secure. And I'm loving the noflap tag. Has anyone else anyone turned up oddities with 4.3.22-RC2 ? Has anyone else deployed it? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

2 1

"Disable until change"
by john.thurston＠alaska.gov 04 Nov '15

04 Nov '15

We often use "disable until ok", but it was brought to my attention that it has burned us from time to time. For example: Host foo is yellow on disk. But that's ok. We're going to allocate some new storage for it in the next service window. The test is marked "disable until ok". But before the service window arrives, something chews up a whole bunch of disk and the now-red test continues to be blue because the test is not yet ok. We sometimes use "acknowledge" for this function, but the non-green screen can get kind of cluttered this way. Does anyone have a good way to fake "disable while status remains unchanged"? -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

6 8

Anyone got a patch for the 24-hour limit on ACKS? Or a script to invoke acknowledge.sh?
by cleaver＠terabithia.org 03 Nov '15

03 Nov '15

Hi Betsy, There's a possibility this might be fixed in RC2. If you get a chance, would you be able to test it out on your side? If the issue continues to occur, can you add a ACK_COOKIE_EXPIRATION= line to xymonserver.cfg, and set the value to something higher than 86400 (1 day)? I'm not completely convinced it's related, but that's the only magic value relating to cookie codes that's set to 1 day itself now. HTH, -jc On 10/19/2015 10:53 AM, J.C. Cleaver wrote: > Betsy, > > Have you noticed any difference in success depending on how you're > specifying the duration? (eg, "1d" vs "24h", etc.?) > > There's one "magic value" relating to a full day when we're generating the > ack code to begin with, but on walking through the module code I can't see > how it would cause an effect like this. > > > After the acknowledgement, the expiration time of the current one should > be displayed on the status page for that test. Can you confirm that the > pending date in those cases is what you'd expect it to be? > > Sort of grasping at straws here. I'll keep digging in, but so far things > seem to be working correctly for me. > > Regards, > -jc > > > On Mon, October 19, 2015 10:38 am, Novosielski, Ryan wrote: >> My suspicion would be that the test briefly exits the bad state, or you >> have a problem with a clock somewhere. One weird issue that I used to have >> is if I set acknowledgments too long (we are talking like 999 weeks), IRIX >> would expire them almost immediately. But that would not happen under your >> circumstance certainly. >> >> -- >> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences* >> || \\UTGERS |---------------------*O*--------------------- >> ||_// Biomedical | Ryan Novosielski - Senior Technologist >> || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu>- >> 973/972.0922 (2x0922) >> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark >> `' >> >> On Oct 19, 2015, at 13:07, Betsy Schwartz >> <betsy.schwartz at gmail.com<mailto:betsy.schwartz at gmail.com>> wrote: >> >> our acks always always always seem to time out >> what are we doing differently? >> >> we're running xymon 4.3.21 on RHEL6 >> >> >> On Mon, Oct 19, 2015 at 11:18 AM, Ralph Mitchell >> <ralphmitchell at gmail.com<mailto:ralphmitchell at gmail.com>> wrote: >> >> I have ack-by-email set up, and can send multi-day/week acks without a >> problem. >> >> Ralph Mitchell >> >> On Oct 19, 2015 11:02 AM, "Root, Paul T" >> <Paul.Root at centurylink.com<mailto:Paul.Root at centurylink.com>> wrote: >> I was just doing an ack for 1+ days. >> >> It will only get cancelled if the test in question goes green however >> briefly. >> >> A disable will persist if it is set to expire at a certain time. >> >> From: Xymon >> [mailto:xymon-bounces at xymon.com<mailto:xymon-bounces at xymon.com>] On Behalf >> Of Novosielski, Ryan >> Sent: Monday, October 19, 2015 9:59 AM >> To: betsy.schwartz at gmail.com<mailto:betsy.schwartz at gmail.com> >> Cc: xymon at xymon.com<mailto:xymon at xymon.com> >> Subject: Re: [Xymon] Anyone got a patch for the 24-hour limit on ACKS? Or >> a script to invoke acknowledge.sh? >> >> I have never found acks to have a 24 hour limit. Can anyone confirm >> whether this is definitely true or not? >> >> ____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences* >> || \\UTGERS |---------------------*O*--------------------- >> ||_// Biomedical | Ryan Novosielski - Senior Technologist >> || \\ and Health | novosirj at rutgers.edu<mailto:novosirj at rutgers.edu>- >> 973/972.0922<tel:973%2F972.0922> (2x0922) >> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark >> `' >> >> On Oct 19, 2015, at 10:57, Betsy Schwartz >> <betsy.schwartz at gmail.com<mailto:betsy.schwartz at gmail.com>> wrote: >> We've GOT to be able to ack alerts and have the ack survive the weekend. >> (if I can call acknowledge.sh from perl I have a script that can scrape >> the board and re-ack) >> thank you!! >> > > _______________________________________________ > Xymon mailing list > Xymon at xymon.com > http://lists.xymon.com/mailman/listinfo/xymon

1 0

Xymon 4.3.22-beta Release Download
by john.thurston＠alaska.gov 03 Nov '15

03 Nov '15

On 10/29/2015 5:44 PM, J.C. Cleaver wrote: > Hello all! > > Xymon-4.3.22-beta has been posted to Sourceforge, containing bug fixes and > a smattering of new features and improvements since 4.3.21. It's reachable > at > http://sourceforge.net/projects/xymon/files/Xymon/4.3.22/xymon-4.3.22-beta.… > > Some notable changes are at > https://sourceforge.net/projects/xymon/files/Xymon/4.3.22/, while the full > Changelog is at > https://sourceforge.net/p/xymon/code/7707/tree/branches/4.x-master/Changes These look like important corrections and additions. I particularly will like the ability to disable "flap detection". I have been out-of-office and have not yet had time to test this release. I hope to make time today or tomorrow to do so. I am concerned that I see no mention in the changelog of any modification of combo test behavior. The behavior of combo tests (as defined in combo.cfg) is badly broken under Solaris. Failure, test conditions, and a stack excerpt are mentioned in my email to the list: http://lists.xymon.com/archive/2015-October/042382.html It works correctly on my CentOS installation, but not Solaris. I was able to move my combo test to the CentOS server and forward the result to my Solaris server. This gets the job done for me, but should I be hit by a truck it could be a little difficult for the next person to unwind the convoluted path. -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

3 5

Xymon 4.3.22-beta Release Download
by john.thurston＠alaska.gov 02 Nov '15

02 Nov '15

On 10/29/2015 5:44 PM, J.C. Cleaver wrote: > Changelog is at > https://sourceforge.net/p/xymon/code/7707/tree/branches/4.x-master/Changes I suspect the man-page for xymongen has a minor typo for --no-showemptygroups Where it says "When groups are hosts are made", I suspect you mean to say, "When groups of hosts are made" It's a nice option and I don't object to its behavior, but since it reverses the historic behavior of xymongen, I wish it would get a little more prominence in the changelog. -- Do things because you should, not just because you can. John Thurston 907-465-8591 John.Thurston at alaska.gov Enterprise Technology Services Department of Administration State of Alaska

1 0

useradm CGI -- Documentation?
by mvanderw＠nd.edu 02 Nov '15

02 Nov '15

Hello, I noticed that there seems to be a user administration (using htpasswd) functionality hidden away in Xymon. I found the useradm.sh script in /xymon-seccgi/ and there appears to be a CGI file for it too. All the other files to make the menu on the web page work are in the correct places too (just no entry for it in xymonmenu.cfg). However, I do not see any documentation on how to make this work or what needs to be set up for it to work. I don't even see any reference to it in any other man pages or other documentation! I managed to add the appropriate entry in my xymonmenu.cfg file to make it show up on the webpage, but it doesn't appear to be working correctly for me. Is this something that actually works? Or is this something that was only half-implemented (and won't work correctly in it's current form)? If it is supposed to work, can some documentation be created for it? Or can someone point me to some documentation for it? Seems like this goes right along with the chpasswd functionality that will be in the next release... Thanks! -- Matt Vander Werf

3 2

Status, NCV and time stamps
by cleaver＠terabithia.org 02 Nov '15

02 Nov '15

On Sat, October 31, 2015 4:11 am, Matt Vander Werf wrote: > > Would it be possible to also make it so that you can exclude parts of > lines/multiple lines (not just at the end of the message) from being > parsed > using a block form -like method, in addition to what you've already put in > place? > > Something like this: > >  *this should be ignored* </--> ip6_mrt_cache : 0 > ext4_inode_cache  *this should also be ignored* </--> : > 7960000 Hmm. Part of this is doable, but with the way it's structured I'm hesitant to duplicate the searching logic once we've already progressed down to identifying the label on a given line. This might have to run for every single line of each status message that comes in, which means the cycles do add up if we're not careful. For the first case (where the tag as at the beginning of the line, it's not too hard. Please try out the attached patch and let me know if it works for you. It adds a  processor which will move past the text until it sees a "</-->" (or a newline), and will then continue on as normal. It also adds a block-level skip feature, which might be more efficient (since we're just toggling a skip flag)  these : 1 wont : 83 seen : 0  normal : 0 -jc > > Basically, the parsing tool would just see this to parse for the graphing: > > ip6_mrt_cache : 0 > ext4_inode_cache : 7960000 > > Is this something that would be possible to also put in place? I'm not > sure > how the new functionality is done now, so I don't know if this is > something > that's feasible to add in or not. > > If it's not very easy or very feasible or even possible at all (without > major re-work of the code), then that's fine. The existing new > functionality for parsing skipping will be very useful to me as it is! > > But if it could be added, that would be really awesome and even more > useful > for me!! > > If this is something that is possible, but probably won't be ready for the > actual 4.3.22 release, then that's perfectly fine with me as well! I'm > willing to wait until the next release if that's what is necessary! :) > > Thanks as always! > > -- > Matt Vander Werf > > On Sat, Oct 17, 2015 at 10:54 PM, J.C. Cleaver <cleaver at terabithia.org> > wrote: > >> FYI, all, >> >> 4.3.22 will have comment markers that can be used for both skipping a >> line >> and for ending processing. Something along the lines of this will work: >> <pre> >> ip6_dst_cache : 7680 >> ndisc_cache : 7680 >> ip6_mrt_cache : 0 >> ext4_inode_cache : 7960000 >>  >> scsi_sense_cache : 15360 >> </pre> >> >> Not entirely sure if that will render properly :) But essentially: >> pre-pend lines with "ncv_skip" in a regular HTML comment to skip it, and >> a >> bare "ncv_end" in a comment will abort further processing for that >> message. >> >> >> I'd considered adding regex capabilities directly into the NCV module, >> but >> it's a fairly time sensitive operation on more heavily-loaded servers, >> so >> it's important to keep existing modules like that as efficient as >> possible. >> >> The process of RRD module writing here is pretty simple though (simpler >> than I'd recalled it), and it shouldn't be too hard to add other types >> of >> automated/regular processing besides NCV that could be called up. >> >> >> HTH, >> >> -jc >> >> >> On Tue, October 6, 2015 7:42 am, Matt Vander Werf wrote: >> > (Sending this again to the list, as I sent it using the wrong e-mail >> > previously (non-subscriber e-mail)) >> > >> > I second this request for enhancement. It would be nice to be able to >> send >> > more data from the client to be displayed on the column/test page, but >> not >> > get parsed to be included in the RRD graph. Only parsing certain lines >> > that >> > are in the NCV format would be a nice feature. Something like what >> J.C. >> > mentioned ("a data parser that looks for a hidden HTML comment marker >> for >> > lines to process (and ignores the rest)") would work well for this, >> IMO. >> > >> > Or is there a way to do this already? (even through a hack?) >> > >> > Nothing urgent, just something that would be nice to be included in >> some >> > future Xymon release. >> > >> > Thanks. >> > >> > -- >> > Matt Vander Werf >> > >> > On Tue, Oct 6, 2015 at 10:39 AM, Matt Vander Werf <matt1299 at gmail.com> >> > wrote: >> > >> >> I second this request for enhancement. It would be nice to be able to >> >> send >> >> more data from the client to be displayed on the column/test page, >> but >> >> not >> >> get parsed to be included in the RRD graph. Only parsing certain >> lines >> >> that >> >> are in the NCV format would be a nice feature. Something like what >> J.C. >> >> mentioned ("a data parser that looks for a hidden HTML comment marker >> >> for >> >> lines to process (and ignores the rest)") would work well for this, >> IMO. >> >> >> >> Or is there a way to do this already? (even through a hack?) >> >> >> >> Nothing urgent, just something that would be nice to be included in >> some >> >> future Xymon release. >> >> >> >> Thanks. >> >> >> >> -- >> >> Matt Vander Werf >> >> >> >> On Mon, Oct 5, 2015 at 7:48 PM, Boldt, David <dboldt at usgs.gov> wrote: >> >> >> >>> Thanks for these leads. Will looks at both the filter and "trends" >> >>> message options. >> >>> >> >>> As a request for enhancement, I think it would be useful to have an >> >>> option to use a regexp to extract data from the status, one for each >> >>> variable. >> >>> >> >>> >> >>> >> >>> On Mon, Oct 5, 2015 at 7:36 PM, J.C. Cleaver >> <cleaver at terabithia.org> >> >>> wrote: >> >>> >> >>>> On Mon, October 5, 2015 3:52 pm, Boldt, David wrote: >> >>>> > I'm in the process of converting Big Brother tests to XYMon and >> >>>> would >> >>>> like >> >>>> > to enable RRD/graphics for these tests. >> >>>> > >> >>>> > Several of these tests present date/time information which is >> >>>> important. >> >>>> > >> >>>> > As a workaround for one test I have replaced the colon on a >> HH:MM:DD >> >>>> > formatted date with a dot, but it doesn't really look like a time >> >>>> anymore. >> >>>> > >> >>>> > * Is there a way to have NCV ignore colons (and only use "=")? >> >>>> > >> >>>> > * Is there a way to have NCV ignore a line (which might contain >> >>>> colons)? >> >>>> >> >>>> >> >>>> Unfortunately, there isn't a way to do either of these at the >> moment. >> >>>> I'd >> >>>> considered a data parser that looks for a hidden HTML comment >> marker >> >>>> for >> >>>> lines to process (and ignores the rest), but it's not quite present >> >>>> yet. >> >>>> >> >>>> >> >>>> > >> >>>> > * Can I use Data messages in such a way that the corresponding >> >>>> Status >> >>>> > message is not parsed? >> >>>> >> >>>> The easiest way to do this would be to add a --filter= option to >> the >> >>>> xymond_channel command line for the xymond_rrd process that's >> handling >> >>>> the >> >>>> *status* channel, but not the data channel. >> >>>> >> >>>> There's a CPU load hit for doing so (since now you're doing a PCRE >> on >> >>>> each >> >>>> message), however it may not have much of an impact depending on >> your >> >>>> message volume. >> >>>> >> >>>> >> >>>> > * Might there be some other mechanism entirely? >> >>>> >> >>>> Generally, speaking for more advanced RRD submission, the 'trends' >> >>>> message >> >>>> can be a useful data payload which is read directly by xymond_rrd. >> The >> >>>> advantage there (aside from more direct control) is the ability to >> >>>> send >> >>>> data points for multiple graphs at once, which could outweigh the >> >>>> expense >> >>>> of another message transmission. >> >>>> >> >>>> >> >>>> HTH, >> >>>> >> >>>> -jc >> >>>> >> >>>> >> >>>> >> >>> >> >>> >> >>> -- >> >>> -- David Boldt >> >>> <dboldt at usgs.gov> >> >>> >> >>> >> >>> "Ah, this is obviously some strange usage of the word 'safe' that >> I >> >>> wasn't previously aware of." >> >>> --Douglas Adams (1952 - 2001), Arthur Dent in "The Hitchhikers >> >>> Guide >> >>> to the Galaxy" >> >>> >> >>> >> >>> _______________________________________________ >> >>> Xymon mailing list >> >>> Xymon at xymon.com >> >>> http://lists.xymon.com/mailman/listinfo/xymon >> >>> >> >>> >> >> >> > _______________________________________________ >> > Xymon mailing list >> > Xymon at xymon.com >> > http://lists.xymon.com/mailman/listinfo/xymon >> > >> >> >> >

1 0

Xymon version 5
by cleaver＠terabithia.org 02 Nov '15

02 Nov '15

On Sat, October 31, 2015 10:52 am, Novosielski, Ryan wrote: > Check the list archives. There was a discussion not too long ago about it, > as I recall. > I believe http://lists.xymon.com/archive/2014-January/038897.html is most recent. > > On Oct 31, 2015, at 11:45, James Louis > <jglouisjr at gmail.com<mailto:jglouisjr at gmail.com>> wrote: > > Hi all, > > Maybe I'm out of the loop on this but is there going to be a version 5 of > Xymon? Is there a roadmap page? I was actually planning on a roadmap update during the 4.3.22 announcement this week, but might as well post now. :) After 4.3.22 is released, I'll be opening up the 4.4 release. This is primarily focused on high performance and core features, including a back-port of the compression support from Xymon 5 (a/k/a "trunk"). It's a fairly large patch set that incorporates a lot of the work that had been done in the Terabithia RPMs. The potential for migration gotchas combined with the sheer size of the delta warrants a minor version bump, but it's still identifiably "Xymon 4". Xymon 5 is the trunk update Henrik has been working on for a while now. The biggest core features are compression, sqlite as a data store, IPv6 and direct SSL communication support throughout, a new poller (xymonnet2) and page generator, and more streamlined client->RRD processing. Development on it's been somewhat stalled with Henrik being busy at his new position, but it's still definitely "on the roadmap". For a more specific ETA on it, I'd have to defer to him. Performance-wise, there's a lot of improvement in 4.4 which gives continued flexibility to the "roll your own" solutions for both DB-type/automated access and SSL wrapping of payloads, which are probably the two largest fundamental core features (outside of IPv6)... As mentioned, compression will be backported into 4.4 -- with support for lzo and lz4 on top of the original zlib -- but the rest hasn't made it in in a testable form. My plan is to get a 4.4 beta release out by the end of December. I'm also looking to complete full removal of display HTML and an easier way to deploy "themes" at the front end (which can, of course, link in to whatever custom display solutions are desired). Some of that might be 4.4, some might be 4.5, but IMO it's additional ways of visualizing / browsing data at the web layer that would most help the project's reach. Regards, -jc

4 6