On Sat, 26 Aug 2023 at 17:53, J.C. Cleaver <cleaver at terabithia.org> wrote:
Jeremy: Might you be able to try adjusting a file name as +/- one hour? I'm wondering if DST calculations might be throwing the lookup off here.
OK, so mystery solved. This was a self-inflicted injury. Sorry to waste peoples' time.
It turns out that some of the binaries installed from the Terabithia package had been replaced (but not by me) by binaries of the same code release (4.3.30) but compiled from source without any of the patches provided in the Terabithia .src.rpm file.
While I haven't yet reverted the binaries to the original, I have confirmed that this would likely fix the issue I was having. If I execute the original CGI binary (which was renamed to *.bak) with suitable QUERY_STRING then I get the correct results instead of the error message. And in particular, if I set the TIMEBUF to be the penultimate entry in the hist/<hostname>.<servicename> file, and run the svcstatus.cgi.bak file under strace, then I can see that it's trying the new format, and then reverting to the old format, in a way that matches the old-style filename in histlogs, which have names relative to my local timezone.
$ SCRIPT_NAME= REQUEST_METHOD=GET QUERY_STRING="HOST=$HOST&SERVICE=$SERVICE&TIMEBUF=$TIMEBUF" strace -f /usr/share/xymon/bin/svcstatus.cgi.bak --no-svcid --historical 2>&1 | grep histlogs stat("/var/lib/xymon/histlogs/<hostname-redacted>/procs/1693158425", 0x7ffcade28c80) = -1 ENOENT (No such file or directory) stat("/var/lib/xymon/histlogs/<hostname-redacted>/procs/Mon_Aug_28_03:47:05_2023", {st_mode=S_IFREG|0644, st_size=48102, ...}) = 0 open("/var/lib/xymon/histlogs/<hostname-redacted>/procs/Mon_Aug_28_03:47:05_2023", O_RDONLY) = 3
(Side note: if I don't define SCRIPT_NAME when running historylog.sh, I get a segfault in svcstatus.c/parse_query() line 154 at the SBUF_MALLOC.)
I wasn't aware of the substituted binary. Apparently, the reason for this was to work around a seg fault in do_request() that occurred when debugging was enabled. The work-around was to double the memory allocated for "clienturi" when history.sh creates its links to svcstatus.sh (from 1024 to 2048). I'll investigate this a bit more and might have a patch to apply one day soon...
Again, sorry for the false alarm, and thanks for helping me to narrow down the search.
J