Syncplicity Support

Search our knowledgebase to get the help you need, today

Submit a request

Related articles
Follow

How to configure Storage Connector health metrics using Graphite

This article will demonstrate how to configure the Syncplicity Storage Connector to send its health metrics data feed to Graphite, and how to set up some basic graphs which can be used to monitor the performance and health of the Storage Connector.

Overview

Graphite is an enterprise-scale monitoring tool that runs well on commodity hardware. It is free and released under the open source Apache 2.0 license. Graphite is comprised of:

  • Carbon - A twisted daemon that listens for time-series data
  • Whisper - A simple database library for storing time-series data (similar in design to RRD)
  • Graphite - A Django webapp that renders graphs on-demand using Cairo

Storage Connectors can send Health Metrics directly to Graphite/Carbon via TCP. Today Syncplicity only supports Graphite plain text format which is very simple and versatile enough to work with most popular monitoring tools and metrics collectors like statsd, collectd, or logstash.

Set-up

We assume that you have already installed and configured your Storage Connectors and they are running in your data center (whether in a Dev & Test environment or in Production), and at this point you are ready to connect it to your monitoring tool.

First begin by installing, configuring and starting Graphite. Refer to the Graphite installation guide for detailed steps on how to do this. Our examples below assume that Carbon is set-up on the host graphite and it is listening to metrics on TCP port 2003 (default) and the Graphite interface is located on the host graphite port 80.

Storage Connector configuration

The first action is to add several new configuration settings in the Storage Connector config file. The default location of this config file is /etc/syncp-storage/syncp-storage.conf. The file is written in HOCON format; please see Using HOCON for reference.

You will need to add the following 5 lines to your config file:

syncplicity.health.enable
syncplicity.health.external.enable
syncplicity.health.external.host
syncplicity.health.external.port
syncplicity.health.external.prefix

It is recommended that you add these to the very end of the file.

The proceeding steps will go through how to input the values for each setting.

Step 1

On the Storage Connector node, open the config file for editing.

Step 2

Ensure that health monitoring is enabled.

syncplicity.health.enable = true

Step 3

Enable health metrics to be exported to an external port.

syncplicity.health.external.enable = true

Step 4

Specify the Carbon daemon host URL.

syncplicity.health.external.host = graphite.internaldomain.com

Step 5

Confirm that the Carbon daemon TCP port is specified (the default is port 2300).

syncplicity.health.external.port = 2300

Step 6

Set a prefix for the the health metrics. When an additional prefix is specified, such as "myCluster.myNode", all metrics roll up under the myCluster.myNode.syncp.compute.v1.* path.

syncplicity.health.external.prefix = myCluster.myNode

Alternatively you can use UNIX environment variables to set the prefix. For example, if you have these environment variables set:

HOST=$(hostname -s)
CLUSTER="myCluster"

Then you can configure this setting in the config file as:

syncplicity.health.external.prefix = $CLUSTER.$HOST

Step 7

Save your changes to the config file and restart Storage Connector.

To verify that the Carbon daemon has started to receive metrics, navigate to the Graphite interface and click the dropdown list box on the left. The application will display a drilldown menu that matches the prefix specified in the "syncplicity.health.external.prefix" or the hostname of the Storage Connector if the prefix is not specified. If you open this drilldown further you will see a number of metrics under the *.syncp.compute.v1 path.

At this point Graphite should be receiving data and automatically create corresponding buckets for each of the metrics. No additional configuration on the Graphite site is required to connect Storage Connector to Graphite. Next we will demonstrate how to graph a few of the interesting Storage Connector metrics using Graphite.

Using Graphite to display health metrics

As in our earlier example, the Graphite render API is available on the host graphite on port 80.

The following examples will demonstrate how to construct queries to render the data and generate graphs for the next three scenarios:

  • The total number of requests (uploads + downloads)
  • The total amount of traffic (number of bytes uploaded and downloaded)
  • The JVM heap status

Note: Each query is listed above the graph shown.

Example 1 - Monitor the total number of requests (uploads + downloads)

http://graphite/render/?width=586&height=308&_salt=1459170462.524&target=*.*.syncp.compute.v1.
syncRequests.currentDownloadRequestsCounter&target=*.*.syncp.compute.v1.syncRequests.
currentUploadRequestsCounter&from=-2hours

Example 2 - Monitor the total traffic (number of bytes uploaded and downloaded)

http://graphite/render/?width=586&height=307&_salt=1459170692.851&from=-2hours&lineMode=staircase&
connectedLimit=&areaAlpha=0.7&areaMode=stacked&colorList=%23455A64%2C%23CFD8DC&target=*.*.
syncp.compute.v1.transfer.in.bytes&target=*.*.syncp.compute.v1.transfer.out.bytes

Example 3 - JVM heap status

http://graphite/render/?width=586&height=308&_salt=1459170786.991&target=*.*.syncp.compute.v1.jvm.
heapUsed&from=-2hours&lineMode=staircase
Powered by Zendesk