On Thu, Feb 5, 2015, at 11:47, Bill Arlofski wrote:

On 02/05/2015 08:40 AM, Mark Felder wrote:

...

I warn you this (probably) isn't exactly what you're looking for, but I've stopped using the SMS email gateways due to reliability issues and when things are really going bad -- rate limiting. Instead I've moved to sending my notifications through Pushover.net which sends (fast!!) push notifications to phones.

You can send messages two ways: their email gateway, or write a notification script that uses their REST api. It's very simple. I wrote this in a few minutes and haven't looked back.

Hi Mark... Thanks for that recommendation re: pushover.net

I have suffered for _years_ with the AT&T (now Frontier) email-to-sms gateway. Using the AT&T email gateway, Xymon notifications get mixed in with other txt messages because each message sent via their gateway is

From: ${SomeEverIncreasingNumber}

So I can not set a filter on a phone number, or username and have a different notification tone for Xymon messages.

Pushover currently has a limited number of notification tones, but that is OK because I can have my xymon alert tell Pushover which to send so I can always know if I got a txt or a Xymon alert now.

Still waiting on my first Xymon->Pushover alert, but this looks promising to me and my sanity. lol

It's cheap, too. If you go over your free 7500 messages per month you can buy more -- and it will only pull from that pool if you go over your limit each month. The extra messages you buy don't ever expire.

FYI my alerts.cfg looks like this:

$ALERT_ME= SCRIPT /scripts/xymon-pushover.sh pushover REPEAT=1h
FORMAT=SCRIPT

Due to the fact that a SCRIPT needs an argument, I give it "pushover" even though it does nothing. I think that value is only used in the INFO page under "recipient". You could go crazy and set recipients to human readable names that are mapped to different USERKEY values if you wanted.

Btw, if you go over your 7500 messages per month I pity your sanity, or you must have a very large environment and lots of techs on-call. :-)

I'm glad for the info about Pushover, because it seems it could be implemented very easily with xymon. At this point, I'm not having problems with SMS, but I'm keeping Pushover as a possibility.

One possible downside of using Pushover, is that it would require Internet connectivity on the cell phone (SMS/MMS doesn't).

Someone really should come up with an affordable true SMS WebService (not email) gateway.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Benjamin Smith Sent: Thursday, February 05, 2015 11:59 To: xymon at xymon.com Subject: Re: [Xymon] Xymon notifcations disappear due to base64 encoding

On Thursday, February 05, 2015 07:40:13 AM Mark Felder wrote:

On Wed, Feb 4, 2015, at 19:05, Benjamin Smith wrote:

...

We've wasted time playing with a wrapper for the mail command that would strip out the "\r" in the alerts but this has, so far, proven fruitless. Messages sent via the Sprint SMS gateway go through just fine. Wondering if anybody else has seen this issue, or has any idea what the best way to handle it might be?

I warn you this (probably) isn't exactly what you're looking for, but I've stopped using the SMS email gateways due to reliability issues and when things are really going bad -- rate limiting. Instead I've moved to sending my notifications through Pushover.net which sends (fast!!) push notifications to phones.

You can send messages two ways: their email gateway, or write a notification script that uses their REST api. It's very simple. I wrote this in a few minutes and haven't looked back.

I will be evaluating this alongside the other fix in this thread.

Glauber presented a simple solution that solves our immediate problem. Otherwise, we've had no issue with SMS messages for years. (Verizon) I'm testing both solutions.

Thanks!

Ben

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

More details:

In xymonserver.cfg, I have

#MAILC="mail"
MAILC="/usr/local/hobbit/server/mail_nocr.sh" # Command used to send an e-mail with no subject MAIL="$MAILC -s" # Command used to send an e-mail with a subject

The mail_nocr.sh script is like this:

#! /bin/bash

20130104 - glauber - get rid of CR in emails

exec /usr/bin/tr -d '\015' | /bin/mail "$@"

That's all, folks!

The occasional CR in the emails not only mess up the SMS gateways, they also make Exchange turn a simple message into an email with a binary attachment.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Ribeiro, Glauber Sent: Thursday, February 05, 2015 09:37 To: Benjamin Smith; xymon at xymon.com Subject: Re: [Xymon] Xymon notifcations disappear due to base64 encoding

I've always had this problem. I ended up setting the mail client in xymon server (via the configuration file) to be a script that pipes the messages through tr to delete the CRs, then pipes then to mail.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Benjamin Smith Sent: Wednesday, February 04, 2015 19:06 To: xymon at xymon.com Subject: [Xymon] Xymon notifcations disappear due to base64 encoding

This morning, one of our servers was down and we (the admins) didn't immediately know because of a long chain of events. After spending the morning drilling down step by step, the issue was that Verizon's SMS email gateway doesn't accept base64 encoded messages, and the message being sent was base64 encoded because of <CR> (aka "\r" or ASCII 13) in the message.

We could reproduce this issue with the following PHP script used at the command line as a Verizon customer. I don't know if this is a recent change at Verizon, and don't really have a way to find out.

// GENERAL ENVIRONMENT CentOS 6, recently updated. Postfix is the MTA PHP5

// THIS WON'T WORK Note the "\r" that causes postfix to base64 encode the message. file.php's contents, 1 line: <?php echo "Test of bad message\r\n"; ?> $ php file.php | mail -s test yournumber at vtext.php

// THIS WORKS FINE, Note the lack of "\r", postfix sends this as text/plain, 7 bit encoding. file.php's contents, 1 line: <?php echo "Test of bad message\n"; ?> $ php file.php | mail -s test yournumber at vtext.php

We've wasted time playing with a wrapper for the mail command that would strip out the "\r" in the alerts but this has, so far, proven fruitless. Messages sent via the Sprint SMS gateway go through just fine. Wondering if anybody else has seen this issue, or has any idea what the best way to handle it might be?

Thanks,

Ben

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

I'm not sure the problem is internally in xymon, but rather in the diversity of sources of the text that ends up in email bodies.

I think xymon should do some simple sanitization of alert texts (maybe a configuration item), to eliminate CRs (for benefit of email), but also characters that are not safe for HTML (for benefit of the browser). I wonder is a crafty person would be able to trick xymon into putting Javascript in an alert, by manipulating error messages.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Mark Felder Sent: Friday, February 06, 2015 09:21 To: xymon at xymon.com Subject: Re: [Xymon] Xymon notifcations disappear due to base64 encoding

On Thu, Feb 5, 2015, at 10:23, Ribeiro, Glauber wrote:

More details:

In xymonserver.cfg, I have

#MAILC="mail"
MAILC="/usr/local/hobbit/server/mail_nocr.sh" # Command used to send an e-mail with no subject MAIL="$MAILC -s" # Command used to send an e-mail with a subject

The mail_nocr.sh script is like this:

#! /bin/bash

20130104 - glauber - get rid of CR in emails

exec /usr/bin/tr -d '\015' | /bin/mail "$@"

Let's kill this in the code instead of resorting to hacks like this.

One or multiple of these instances of \r is bound to be the cause:

~/svn/xymon-code/xymond > grep '\\r' * xymond.c: if (sourcename) cause = strtok(NULL, "\r\n"); xymond.c: txtstart += strspn(txtstart, " \t\r\n"); xymond.c: tok = strtok(buf, " \t\r\n"); xymond.c: if (tok) tok = strtok(NULL, " \t\r\n"); xymond.c: tok = strtok(NULL, " \t\r\n"); xymond.c: ehost = bhost + strcspn(bhost, " \t\r\n"); xymond.c: bid = msg->buf + strcspn(msg->buf, " \t\r\n"); bid += strspn(bid, " \t"); xymond.c: eid = bid + strcspn(bid, " \t\r\n"); xymond.c: p = strtok(p, " \t\r\n"); xymond.c: p = strtok(p, " \t\r\n"); xymond.c: p = msg->buf + strcspn(msg->buf, "\r\n"); xymond.c: if ((*p == '\r') || (*p == '\n')) { xymond_channel.c: hostname = inbuf + strcspn(inbuf, "/|\r\n"); xymond_channel.c: hostend = hostname + strcspn(hostname, "|\r\n"); xymond_client.c: p = s + strcspn(s, "\r\n"); xymond_locator.c: const char *delims = "|\r\n\t ";

JC, Henrik -- any ideas?

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

On Fri, February 6, 2015 9:08 am, Mark Felder wrote:

On Fri, Feb 6, 2015, at 10:26, Ribeiro, Glauber wrote:

...

I'm not sure the problem is internally in xymon, but rather in the diversity of sources of the text that ends up in email bodies.

Internally Xymon is using \r (CR) in a lot of places. The info that is put in alerts is fairly straightforward -- just pieced together from various ENVs related to the event. As far as I can tell there is no data sourced directly from the clients in the alerts unless you include BBALPHAMSG in your alert message, which is going to be a huge amount of information.

A simple sanitation routine in do_alert.c would probably take care of this, but if we can remove the \r from being used internally the problem should solve itself. I think a sanitation routine is probably more treating the symptom and not the problem, as well as unlikely to solve all possibilities of dangerous data which shouldn't be getting this far in the first place.

A simple removal of "\r" characters wouldn't be difficult. But line breaks can be tricky to get correct for all of the various needs. I'm sure there are precious few class Mac OS systems out there we're dealing with, but I also how Windows vs. Unix clients might be affected at display time.

...

I think xymon should do some simple sanitization of alert texts (maybe a configuration item), to eliminate CRs (for benefit of email), but also characters that are not safe for HTML (for benefit of the browser). I wonder is a crafty person would be able to trick xymon into putting Javascript in an alert, by manipulating error messages.

If you use the full BBALPHAMSG in your alert message and the client sends some nasty data I am certain it could do some unsavory things. I also bet you could -- from a compromised client -- send a crafted alert message to a new or existing column and attack the admin when he views the alert in his browser. Your saving grace might be the fact that alert messages are limited to 4096 bytes:

#define MAX_ALERTMSG_SCRIPTS 4096

That will help you wrt. email somewhat, although one would hope that that would make an incoming alert message no more or less a possible source of mal-data than any other potentially untrusted email source.

Hopefully Xymon 5 brings us encrypted and authenticated transport between the client and server as that will help prevent this type of attack, as well as protect your sensitive info in transit :-)

This is really the solution -- end-to-end encoding using key trust; right now the most client security that you have is IP-based. But even if your transport mechanism is over an stunnel, you're really still at the mercy of the original source. A local user could execute a script placing a specially crafted message in $0, which would show up in the 'ps' output and might survive <PRE> wrapping in the 'procs' test to cause a browser problem, for example.

Regards,

-jc

We have had that problem for a long time and tried a similar method as Ribeiro to strip out charactes but endup using mutt instead. Since then haven't had any issues:

#MAILC="mail" MAILC="/usr/local/bin/mutt"

On Thu, Feb 5, 2015 at 11:23 AM, Ribeiro, Glauber < glauber.ribeiro at experian.com> wrote:

More details:

In xymonserver.cfg, I have

#MAILC="mail" MAILC="/usr/local/hobbit/server/mail_nocr.sh" # Command used to send an e-mail with no subject MAIL="$MAILC -s" # Command used to send an e-mail with a subject

The mail_nocr.sh script is like this:

#! /bin/bash

20130104 - glauber - get rid of CR in emails

exec /usr/bin/tr -d '\015' | /bin/mail "$@"

That's all, folks!

The occasional CR in the emails not only mess up the SMS gateways, they also make Exchange turn a simple message into an email with a binary attachment.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Ribeiro, Glauber Sent: Thursday, February 05, 2015 09:37 To: Benjamin Smith; xymon at xymon.com Subject: Re: [Xymon] Xymon notifcations disappear due to base64 encoding

I've always had this problem. I ended up setting the mail client in xymon server (via the configuration file) to be a script that pipes the messages through tr to delete the CRs, then pipes then to mail.

g

-----Original Message----- From: Xymon [mailto:xymon-bounces at xymon.com] On Behalf Of Benjamin Smith Sent: Wednesday, February 04, 2015 19:06 To: xymon at xymon.com Subject: [Xymon] Xymon notifcations disappear due to base64 encoding

This morning, one of our servers was down and we (the admins) didn't immediately know because of a long chain of events. After spending the morning drilling down step by step, the issue was that Verizon's SMS email gateway doesn't accept base64 encoded messages, and the message being sent was base64 encoded because of <CR> (aka "\r" or ASCII 13) in the message.

We could reproduce this issue with the following PHP script used at the command line as a Verizon customer. I don't know if this is a recent change at Verizon, and don't really have a way to find out.

// GENERAL ENVIRONMENT CentOS 6, recently updated. Postfix is the MTA PHP5

// THIS WON'T WORK Note the "\r" that causes postfix to base64 encode the message. file.php's contents, 1 line: <?php echo "Test of bad message\r\n"; ?> $ php file.php | mail -s test yournumber at vtext.php

// THIS WORKS FINE, Note the lack of "\r", postfix sends this as text/plain, 7 bit encoding. file.php's contents, 1 line: <?php echo "Test of bad message\n"; ?> $ php file.php | mail -s test yournumber at vtext.php

We've wasted time playing with a wrapper for the mail command that would strip out the "\r" in the alerts but this has, so far, proven fruitless. Messages sent via the Sprint SMS gateway go through just fine. Wondering if anybody else has seen this issue, or has any idea what the best way to handle it might be?

Thanks,

Ben

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon

---

Xymon mailing list Xymon at xymon.com http://lists.xymon.com/mailman/listinfo/xymon