Passive checks

 

Making passive checks over HTTP

This How-To covers how to configure SmartSNO for accepting passive checks over HTTP and then sending said checks with cURL.
Our running example will be a linux device on which we will collect number of the logged in users. You can use SmartSNO server itself for example.

Configuring SmartSNO

  1. We start by adding a new Probe (Settings->Supported devices->Probes). We will name this probe Users Logged In.
    For protocol we choose HTTP Passive and for format we select Number, this way we can chart the data.
    For handle we input users_logged_in. Handle will be important later when we will construct URL to post data to.
    Enable monitoring and charting of the data while we are at it.
  2. Next thing we want to do is add newly created Probe to a Probe Group which we will add to a device.
    We add a new Probe Group (Settings->Supported devices->Probe Groups) named Linux Metrics.
    We add Users Logged In probe to it and set check interval to 5 minutes.
  3. After we have Probe Group set up we choose a device which represents Linux server.
    I have one with IP 10.100.100.10, but you can also choose to use SmartSNO server itself which should be on 127.0.0.1.
    Select the device and under setup add our Linux Metrics probe group to it.
    Also take note of which access credentials the device is using, we will need it in the next step.
  4. The last thing we need to configure is our access credentials (Settings->Credentials) that the device is using.
    We enable HTTP passive checks and set up username and password to admin and password123.

Once we have that set up we are ready to start collecting passive checks on our probe.
The starting state is UNKNOWN and it stays this way until first check has been received.

Passive checks API

Before we make a passive check let us get acquainted with passive checks API.

Endpoint for receiving passive checks takes ip and probehandle arguments which are taken from URL.
ip is an IP of a device which is configured in SmartSNO.
probehandle is the name of the probe which must be created in SmartSNO and then added to the device.
You authenticate via HTTP basic auth. Username and password must be configured in the credentials that device uses.

Example URL: https://admin:password123@smartsno.company.com/pchecks/10.100.100.10/mysql_health

In the example URL we are accessing device that we have added in SmartSNO with local IP 10.100.100.10.
This device has also been configured that it has probe with handle mysql_health.
admin and password123 should match to that what was configured in device’s credentials.

Once you have the URL you can perform POST requests on it with two pieces of data:
status should be one of the following strings representing statuses in SmartSNO: [ok, warn, crit, unknown, fail]
value can be string or a numerical value (integer, float), depending on how you configured the probe.

If probe is numerical you must provide a numerical value. You also have an option of charting numerical values.
If it is a string, the value is optional.

Example of making a POST with curl:

$ curl https://admin:password123@smartsno.company.com/pchecks/10.100.100.10/mysql_health -d "status=ok&value=70" -k

Posting passive checks results over HTTP

For this step we will use tool called cURL.

If it is not available on your machine you can usually install it via a package manager. For example: $ apt-get install curl

Passive checks API resides at https://smartsno-url/pchecks but because we need to authenticate, our url looks more like
https://admin:password123@smartsno-url/pchecks. We also need to address our device and probe to which we want post passive checks.
With settings as described in Configuring SmartSNO section our full url looks like this:
https://admin:password123@smartsno-url/pchecks/10.100.100.10/users_logged_in

We need to execute HTTP POST request with two pieces of information namely status and value.
We can check how many users are logged in on our machine with who | wc -l. In my case there are four.
So let us push that information to our passive check:

$ curl https://admin:password123@smartsno-url/pchecks/10.100.100.10/users_logged_in -d "status=ok&value=4" -k

This example calls curl with -k option which means that it does not verify the host and is thus susceptible to man in the middle attacks.
Consult curl’s documentation especially –cacert option on how to include certificates to verify the host.

If we now check the status of the probe on our device in SmartSNO (open up device’s dashboard and click on Monitoring status and Charts tab)
we can see that it has status OK and value 4 was charted on the graph.

The next step would be to create a script that finds out how many users are logged in and then decides based on that number which status to send.
For example we may send WARN if there are over 100 users logged in or if we detect user root logged in.
This script then sends passive check with appropriate values automatically. All we have to do is add that script to crontab so it executes every few minutes.
If probe does not get a passive check in time span specified in the Probe Group its status will change to FAIL.