From c.schwarz at funknetz.at Tue Mar 12 15:45:48 2013 From: c.schwarz at funknetz.at (Christoph Schwarz) Date: Tue, 12 Mar 2013 14:45:48 +0000 Subject: [smokeping-users] FPing problem Message-ID: <97E3C2C919ADBE4BAAEFEF08EA704B9A62BEB842@sharepoint> Hello List, I experienced some interrupts in all our graphs at the same time, which is very unusal and tends to be a general problem of smokeping or the server where it is running on. The log file (/var/log/messages) logs: ... Mar 12 13:57:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. Mar 12 14:07:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 14:17:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 14:27:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 14:37:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. Mar 12 14:47:50 monitoring smokeping[31332]: FPing: NOTE: smokeping took 299 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 14:52:50 monitoring smokeping[31332]: FPing: NOTE: smokeping took 299 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 14:57:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). Mar 12 15:07:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. ... Environment: CPU: 2x Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz Mem.: 2048M Debian Squeeze (6.0.7) Linux 2.6.32-5-amd64 #1 SMP Mon Jan 16 16:22:28 UTC 2012 x86_64 GNU/Linux Smokeping Version: 2.003006 Probes: 1057 I tried to increase the step from 300 to 400 but I got an error because the old RRDs have a step of 300. After removing all unresponsive devices I got still those log messages. Do I need to increase the step and how can I achieve that (I still need the graphs)? Thanks. Regards Christoph From alter3d at alter3d.ca Tue Mar 12 16:02:23 2013 From: alter3d at alter3d.ca (Peter Kristolaitis) Date: Tue, 12 Mar 2013 11:02:23 -0400 Subject: [smokeping-users] FPing problem In-Reply-To: <97E3C2C919ADBE4BAAEFEF08EA704B9A62BEB842@sharepoint> References: <97E3C2C919ADBE4BAAEFEF08EA704B9A62BEB842@sharepoint> Message-ID: <513F437F.3090600@alter3d.ca> I'm assuming based on the errors and your configuration that you're using a single instance of an FPing probe? A quick and easy way, network capacity permitting, to fix this is to use multiple probes and split up your targets among them. This way instead of trying to poll >1000 devices in 5 minutes, you're only polling, say, 200 in 5 minutes, but you're doing 5 batches of 200 concurrently. Details on how to set this up are in the Smokeping docs, and in previous threads on this mailing list. - Pete On 03/12/2013 10:45 AM, Christoph Schwarz wrote: > Hello List, > > I experienced some interrupts in all our graphs at the same time, which is very unusal and tends to be a general problem of smokeping or the server where it is running on. > > The log file (/var/log/messages) logs: > > ... > Mar 12 13:57:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. > Mar 12 14:07:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 14:17:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 14:27:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 14:37:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. > Mar 12 14:47:50 monitoring smokeping[31332]: FPing: NOTE: smokeping took 299 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 14:52:50 monitoring smokeping[31332]: FPing: NOTE: smokeping took 299 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 14:57:51 monitoring smokeping[31332]: FPing: NOTE: smokeping took 300 seconds to complete 1 round of polling. This is over 80%% of the max time available for a polling cycle (300 seconds). > Mar 12 15:07:52 monitoring smokeping[31332]: FPing: WARNING: smokeping took 301 seconds to complete 1 round of polling. It should complete polling in 300 seconds. You may have unresponsive devices in your setup. > ... > > Environment: > CPU: 2x Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz > Mem.: 2048M > > Debian Squeeze (6.0.7) > Linux 2.6.32-5-amd64 #1 SMP Mon Jan 16 16:22:28 UTC 2012 x86_64 GNU/Linux > Smokeping Version: 2.003006 > Probes: 1057 > > I tried to increase the step from 300 to 400 but I got an error because the old RRDs have a step of 300. > After removing all unresponsive devices I got still those log messages. Do I need to increase the step and how can I achieve that (I still need the graphs)? > > Thanks. > > Regards > Christoph > > _______________________________________________ > smokeping-users mailing list > smokeping-users at lists.oetiker.ch > https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users From gregs at sloop.net Wed Mar 20 05:24:11 2013 From: gregs at sloop.net (Gregory Sloop) Date: Tue, 19 Mar 2013 21:24:11 -0700 Subject: [smokeping-users] Slave-Master security Message-ID: <1939047810.20130319212411@sloop.net> Ok, so I've setup a test master-slave config, and the basic config looks good. So, I suppose this is essentially off-topic, but I'm wondering about hardening the communications between a master and a slave. In my case, I'm thinking of having slaves that communicate over an un-secure net [say the internet] back to the master. I know the shared secret [PSK] for the slave-master protect [kinda] so that an attacker can't stuff data into the SP master - but that doesnt' address someone finding a hole in the CGI etc. Essentially, if I let the world hit the smokeping.cgi, but only prevent writes, that does noting to prevent others from looking at my smokeping data [which I may not want to allow] or worse, attacking the smokeping.cgi in an attempt to crack the master machine. [And from what I can see, I can't easily use .htaccess files over https to limit access, because the slaves don't grok that.] This is obviously bad. I've considered building VPN's or SSH tunnels between the slave(s) and masters - but does anyone have any tried-and-true methods that are perhaps less cumbersome - that I haven't considered? -Greg From tobi at oetiker.ch Wed Mar 20 07:15:30 2013 From: tobi at oetiker.ch (Tobias Oetiker) Date: Wed, 20 Mar 2013 07:15:30 +0100 (CET) Subject: [smokeping-users] Slave-Master security In-Reply-To: <1939047810.20130319212411@sloop.net> References: <1939047810.20130319212411@sloop.net> Message-ID: Yesterday Gregory Sloop wrote: > Ok, so I've setup a test master-slave config, and the basic config > looks good. > > So, I suppose this is essentially off-topic, but I'm wondering about > hardening the communications between a master and a slave. > > In my case, I'm thinking of having slaves that communicate over an > un-secure net [say the internet] back to the master. > > I know the shared secret [PSK] for the slave-master protect [kinda] so > that an attacker can't stuff data into the SP master - but that > doesnt' address someone finding a hole in the CGI etc. > > Essentially, if I let the world hit the smokeping.cgi, but only > prevent writes, that does noting to prevent others from looking at my > smokeping data [which I may not want to allow] or worse, attacking the > smokeping.cgi in an attempt to crack the master machine. [And from > what I can see, I can't easily use .htaccess files over https to > limit access, because the slaves don't grok that.] > basic auth would be quite simple to add to slaves I guess ... otoh, you could also teach the slaves to use client certificates http://stackoverflow.com/questions/12697450/using-lwp-with-ssl-and-client-certificates you could further limit access by IP address on the server cheers tobi > This is obviously bad. > > I've considered building VPN's or SSH tunnels between the slave(s) and > masters - but does anyone have any tried-and-true methods that are > perhaps less cumbersome - that I haven't considered? > > -Greg > > _______________________________________________ > smokeping-users mailing list > smokeping-users at lists.oetiker.ch > https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900 From ged at jubileegroup.co.uk Wed Mar 20 11:51:00 2013 From: ged at jubileegroup.co.uk (G.W. Haywood) Date: Wed, 20 Mar 2013 10:51:00 +0000 (GMT) Subject: [smokeping-users] Slave-Master security In-Reply-To: <1939047810.20130319212411@sloop.net> References: <1939047810.20130319212411@sloop.net> Message-ID: Hi there, On Tue, 19 Mar 2013, Gregory Sloop wrote: > So, I suppose this is essentially off-topic, but I'm wondering about > hardening the communications between a master and a slave. Not off-topic at all. > In my case, I'm thinking of having slaves that communicate over an > un-secure net [say the internet] back to the master. > ... > I've considered building VPN's or SSH tunnels between the slave(s) and > masters - but does anyone have any tried-and-true methods that are > perhaps less cumbersome - that I haven't considered? Those are the best methods available. They aren't particularly cumbersome, and once you get used to them they're fairly trivial. I even use VPNs to run Smokeping over private wireless links, so if somebody does crack the wireless encryption all they've found is the encrypted VPN behind it. You can use the security features of Apache (for example denying access to the Smokeping CGI to all but a small range of IPs) but it's never going to be as secure as the encrypted methods you've already identified. -- 73, Ged. From tobi at oetiker.ch Thu Mar 21 09:43:33 2013 From: tobi at oetiker.ch (Tobias Oetiker) Date: Thu, 21 Mar 2013 09:43:33 +0100 (CET) Subject: [smokeping-users] smokeping security update Message-ID: Folks, Yesterday a debian has published a cross site scripting vulnerability advisory on smokeping. (see below). On march 4th I have release Smokeping 2.6.9, fixing this and a bunch of other issues. See http://www.smokeping.org/pub/CHANGES for details. The latest Smokeping is always available from http://www.smokeping.org/pub cheers tobi ---------- Forwarded message ---------- Date: Wed, 20 Mar 2013 23:15:24 +0100 From: Salvatore Bonaccorso To: debian-security-announce at lists.debian.org Subject: [SECURITY] [DSA 2651-1] smokeping security update -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - ------------------------------------------------------------------------- Debian Security Advisory DSA-2651-1 security at debian.org http://www.debian.org/security/ Salvatore Bonaccorso March 20, 2013 http://www.debian.org/security/faq - ------------------------------------------------------------------------- Package : smokeping Vulnerability : cross-site scripting vulnerability Problem type : remote Debian-specific: no CVE ID : CVE-2012-0790 Debian Bug : 659899 A cross-site scripting vulnerability was discovered in smokeping, a latency logging and graphing system. Input passed to the "displaymode" parameter was not properly sanitized. An attacker could use this flaw to execute arbitrary HTML and script code in a user's browser session in the context of an affected site. For the stable distribution (squeeze), this problem has been fixed in version 2.3.6-5+squeeze1. For the testing distribution (wheezy), this problem has been fixed in version 2.6.7-1. For the unstable distribution (sid), this problem has been fixed in version 2.6.7-1. We recommend that you upgrade your smokeping packages. Further information about Debian Security Advisories, how to apply these updates to your system and frequently asked questions can be found at: http://www.debian.org/security/ Mailing list: debian-security-announce at lists.debian.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlFKNLQACgkQXm3vHE4uylq7CwCgye7/+ER5c0HpU2/5dOBdZuSm l4gAoKI6RrCumcP3rJDtlDO9mJmdYZUB =aqLM -----END PGP SIGNATURE----- From gregs at sloop.net Fri Mar 29 19:26:08 2013 From: gregs at sloop.net (Gregory Sloop) Date: Fri, 29 Mar 2013 11:26:08 -0700 Subject: [smokeping-users] Rasp-Pi smokeping slaves Message-ID: <365552064.20130329112608@sloop.net> Perhaps slightly off-topic, but I thought it might be interesting to others and a starter for ideas for others... --- Building on the concept of using a Raspberry Pi as a smokeping slave [Thanks to Ioannis Theodoridis] for the push!] I've completed one. As a FYI: I'm a consultant and need to monitor both external facing connects as well as internal systems for multiple clients. I've played with running the Pi's as full masters, with MRTG and Nagios running. They can do it, at least for smaller installations. [Over-clocking them helps some] But the Pi's are pretty CPU maxed. If you drop Nagios, it gets much better... --- However, running smokeping as slave, and using SSH tunnels back to the "mother-ship" allows me to see all the sites in one central location, and secure the communication channel easily. [SSH is easier than building OpenVPN pipes or something.] I also use Nagios on the main master system to do alerts should smokeping results fall into criteria that need attention. [I like Nagios' reporting much better than SP. This isn't a knock to SP, just that it was never intended to be a sophisticated alerting/escalation tool and it shows.] This also allows me to use the slaves to hit the external facing portions of things from multiple viewpoints - which is valuable because I'm seeing more and more subtle routing and latency issues that are not universal. [i.e. RTT or packet loss occurs only between POP's in the same ISP's network, or only to traffic that is peered with specific other networks, etc.] So, having the additional views of the same end-point from other networks, via other peers etc, can be valuable in troubleshooting issues. I'm not completely done with the Slave Pi image, but if anyone would like to snag a copy, I'd be glad to give it up when I'm done. It's not like doing to work yourself is hard, but it's still a fair bit of work, and time. Starting from an image that already has all the packages installed and configured is nice. --- I also have MRTG + routers2 [HT Shipway] loaded but set not to run at startup. Doing moderate MRTG polls [say, less than a few hundred targets] is very do-able on the Pi. MRTG operates as a stand-alone setup, obviously - since there's no other option. CPU use is obviously higher when actually cranking routers2 data, than when polling, but still very reasonable. I've configured it to be as secure as I know how, using IPTables lock-down as well as user auth with the MRTG/Apache pages, among other things. --- Glad to discuss if anyone is interested. For the price, power use, solid-state design and simplicity, I'm seriously impressed with a Raspberry Pi as a slave - or even a small master smokeping+MRTG setup. -Greg From jester2.0 at gmail.com Sat Mar 30 21:21:45 2013 From: jester2.0 at gmail.com (Jester 2.0) Date: Sat, 30 Mar 2013 16:21:45 -0400 Subject: [smokeping-users] Slave to Master connection over IPv6 Message-ID: I am setting up SmokePing with a master and 2 slaves. During my configuration, I ran into an issue that looks a bug. I configured smokeping to connect to my master that has only an IPv6 address. At startup, I get this error: WARNING Master said 500 Can't connect to gold.example.net:80 (Bad hostname) I then configured Slave to connect to a master that has both v4 and v6 address and the connection worked. Watching the connection, I see that it connected over v4, not v6 as it should prefer to. Anyone else able to confirm this? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.oetiker.ch/pipermail/smokeping-users/attachments/20130330/8a795775/attachment.htm From jean at wedebugyou.com Sun Mar 31 13:19:08 2013 From: jean at wedebugyou.com (Jean) Date: Sun, 31 Mar 2013 07:19:08 -0400 Subject: [smokeping-users] Slave to Master connection over IPv6 In-Reply-To: References: Message-ID: <3d6a87510f3d5f94f80c8338c05e5606@wedebugyou.com> Hello Jester, I confirm this problem too. If anyone find a solution, I am willing to hear it. In my setup the master runs on debian and the slaves runs on FreeBSD and CentOS. Cheers Jean On 2013-03-30 16:21, Jester 2.0 wrote: > I am setting up SmokePing with a master and 2 slaves. ?During my > configuration, I ran into an issue that looks a bug. > > I configured smokeping to connect to my master that has only an IPv6 > address. ?At startup, I get this error: > > WARNING Master said 500 Can't connect to gold.example.net:80 [1] (Bad > hostname) > > I then configured Slave to connect to a master that has both v4 and > v6 address and the connection worked. ?Watching the connection, I see > that it connected over v4, not v6 as it should prefer to. > > Anyone else able to confirm this? > > Links: > ------ > [1] http://gold.example.net:80 > > _______________________________________________ > smokeping-users mailing list > smokeping-users at lists.oetiker.ch > https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users From gert at space.net Sun Mar 31 13:58:56 2013 From: gert at space.net (Gert Doering) Date: Sun, 31 Mar 2013 13:58:56 +0200 Subject: [smokeping-users] Slave to Master connection over IPv6 In-Reply-To: References: Message-ID: <20130331115856.GS51699@Space.Net> Hi, On Sat, Mar 30, 2013 at 04:21:45PM -0400, Jester 2.0 wrote: > WARNING Master said 500 Can't connect to gold.example.net:80 (Bad hostname) > > I then configured Slave to connect to a master that has both v4 and v6 > address and the connection worked. Watching the connection, I see that it > connected over v4, not v6 as it should prefer to. Perl itself isn't really dual-stack friendly (built-in modules only do IPv4, you need to install "Socket6" and explicitely use IO::Socket::INET6 to get dual-stack sockets). The DNS probe in smokeping suffers from the same thing, you can't test IPv6 servers with it either. It *should* be fairly easy to fix, though (for AnotherDNS, it's just replacing one "INET" with "INET6" in the .pm)... Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (89) 32356-444 USt-IdNr.: DE813185279