NYM Mixnode monitoring with Prometheus, Telegraf and Grafana

NYM Mixnode monitoring with Prometheus, Telegraf and Grafana

First of all, I would like to express my gratitude to RoomIT NYM mixnode team, this guide is largely based on their article which is published here.

To begin with, make sure you have got a clean server with NYM Mixnode running on it without any errors, crashes etc. We are going to install everything on the host machine without using Docker. My assumption is that you are using Ubuntu 20 or 22 (or at least Debian-based Linux) and running a non-root shell.

  1. Make sure all packets are up-to-date -
sudo apt update && sudo apt upgrade
  1. Let's start with Grafana installation, this is not mandatory, you can start from any app
sudo apt-get install -y apt-transport-https software-properties-common wget
  1. Let's pull Grafana GPG key
sudo mkdir -p /etc/apt/keyrings/ && wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
  1. Let's now add Grafana's repository
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
  1. Now we are going to install latest Grafana version
sudo apt-get update && sudo apt-get install grafana
  1. Enable Grafana Service
sudo systemctl enable grafana-server && sudo systemctl restart grafana-server

Make sure the service is running

sudo systemctl status grafana-server
Make sure the service is active (running)
  1. Now we are going to install certbot in order to obtain a free SSL certificate. It largely depends on which web server you are using. I presume this is nginx, so...
sudo snap install --classic certbot && sudo ln -s /snap/bin/certbot /usr/bin/certbot

Make sure you have got nginx config file set up with your domain of choice. As an example, I am going to use example.com domain, please change it to your domain. Post installation of nginx (which is not covered by this guide), you will have a default config in /etc/nginx/conf.d, make sure you edit it and change server_name to your domain you want to use with Grafana. Restart nginx.

You will also need to add A record (or AAA records for IPv6 addresses) to your domain to make sure it points to your server, read documents of your provider to learn how to do it. Now we can run certbot.

sudo certbot --nginx

If this is the first time you run certbot, it will ask you to confirm your email (to receive notification about certificate expiry), confirm certain choices and finally you will see a choice of all the domains, just type a digit which corresponds to the domain. Do not forget that if there is firewall in place, ports 80 and 443 must be open. There is also a per hour limit to request certificates, so do not probe it too often if you see an error.

Once you see that certificate has been deployed, you can change your default.conf of your nginx to something like this:

# this is required to proxy Grafana Live WebSocket connections.
map $http_upgrade $connection_upgrade {
  default upgrade;
  '' close;
}

upstream grafana {
  server localhost:3000;
}

server {
  root /usr/share/nginx/html;
  index index.html index.htm;
  server_name example.com;
  server_tokens off;

  location / {
    proxy_set_header Host $http_host;
    proxy_pass http://grafana;
    proxy_hide_header X-Frame-Options;
  }

# Proxy Grafana Live WebSocket connections.
  location /api/live/ {
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_set_header Host $http_host;
    proxy_pass http://grafana;
    proxy_hide_header X-Frame-Options;
  }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
    if ($host = example.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


  listen 80;
  server_name example.com;
    return 404; # managed by Certbot


}

by default, Grafana runs on port 3000, so if you change it for any reason, make sure this is reflected in your nginx conf file. Restart nginx.

  1. If everything was done successfully, your grafana UI should be available at http://example.com Default username is admin and password is admin. Please make sure you change it to something more complex. We are going to come back to Grafana later.
  2. Let's install telegraf
curl -s https://repos.influxdata.com/influxdata-archive.key > influxdata-archive.key

echo '943666881a1b8d9b849b74caebf02d3465d6beb716510d86a39f6c8e8dac7515 influxdata-archive.key' | sha256sum -c && cat influxdata-archive.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive.gpg > /dev/null

echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list

sudo apt-get update && sudo apt-get install telegraf
  1. Add the following to your /etc/telegraf/telegraf.conf
[global_tags]

[agent]
  interval = "1m"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = "0s"


[[outputs.prometheus_client]]
  listen = "0.0.0.0:9273"
  path = "/metrics"
  metric_version = 2
  expiration_interval = "60s"
  1. Add the following to your /etc/telegraf/telegraf.d/resource.conf
[[inputs.socketstat]]
    protocols = [ "tcp", "udp" ]
[[inputs.cpu]]
    percpu = true
    totalcpu = true
    collect_cpu_time = false
    report_active = false
[[inputs.disk]]
    ignore_fs = ["tmpfs", "devtmpfs", "devfs"]
[[inputs.diskio]]
[[inputs.mem]]
[[inputs.net]]
interfaces = ["eth*", "enp0s[0-1]", "ens*", "lo","wg0","enp0s31f6.4000"]
[[inputs.system]]
[[inputs.swap]]
[[inputs.netstat]]
[[inputs.processes]]
[[inputs.kernel]]
  1. Add the following to your /etc/telegraf/telegraf.d/nym.conf
#### STATUS SERVER
[[inputs.http]]
    urls = ["http://YOUR_HOSTS:8000/stats"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [inputs.http.tags]
      mixnode = "YOUR_IDENTITY_KEY"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "received_since_startup"
            rename = "packets_received_since_startup"
            type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "sent_since_startup"
            rename = "packets_sent_since_startup"
            type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "dropped_since_startup"
            rename = "packets_explicitly_dropped_since_startup"
            type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "received_since_last_update"
            rename = "packets_received_since_last_update"
             type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "sent_since_last_update"
            rename = "packets_sent_since_last_update"
             type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "dropped_since_last_update"
            rename = "packets_explicitly_dropped_since_last_update"
             type="int"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "update_time"
            rename = "update_time"


[[inputs.http]]
    urls = ["https://validator.nymtech.net/api/v1/status/mixnode/YOUR_ID_NODE/reward-estimation"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [inputs.http.tags]
      mixnode = "YOUR_IDENTITY_KEY"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "estimation.total_node_reward"
            rename = "estimated_total_node_reward"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "estimation.operator"
            rename = "estimated_operator_reward"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "estimation.delegates"
            rename ="estimated_delegators_reward"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.reward_pool"
            rename ="rewardparams_interval_reward_pool"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.staking_supply"
            rename ="rewardparams_interval_staking_supply"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.staking_supply_scale_factor"
            rename ="rewardparams_interval_staking_supply_scale_factor"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.epoch_reward_budget"
            rename ="rewardparams_interval_epoch_reward_budget"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.sybil_resistance"
            rename ="rewardparams_interval_sybil_resistance"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.active_set_work_factor"
            rename ="rewardparams_interval_active_set_work_factor"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "reward_params.interval.interval_pool_emission"
            rename ="rewardparams_interval_interval_pool_emission"
            type = "float"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "epoch.id"
            rename ="epoch_id"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "epoch.epochs_in_interval"
            rename ="epoch_epochs_in_interval"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "epoch.current_epoch_start"
            rename ="epoch_current_epoch_start"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "epoch.current_epoch_id"
            rename ="epoch_current_epoch_id"



[[inputs.http]]
    urls = ["https://validator.nymtech.net/api/v1/status/mixnode/YOUR_ID_NODE/report"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [inputs.http.tags]
      mixnode = "YOUR_IDENTITY_KEY"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "most_recent"
            rename = "most_recent"
            type = "int"
         [[inputs.http.json_v2.field]]
            path = "last_hour"
            rename = "last_hour"
            type = "int"
         [[inputs.http.json_v2.field]]
            path = "last_day"
            rename = "last_day"
            type = "int"



[[inputs.http]]
    urls = ["https://validator.nymtech.net/api/v1/status/mixnode/YOUR_ID_NODE/avg_uptime"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [inputs.http.tags]
      mixnode = "YOUR_IDENTITY_KEY"
    [[inputs.http.json_v2]]
      [[inputs.http.json_v2.field]]
         path = "avg_uptime"
         rename = "node_uptime"


[[inputs.http]]
    urls = ["https://validator.nymtech.net/api/v1/status/mixnode/YOUR_ID_NODE/status"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [inputs.http.tags]
       mixnode = "YOUR_IDENTITY_KEY"
    [[inputs.http.json_v2]]
      [[inputs.http.json_v2.field]]
        path = "status"
        rename = "node_status"


[[inputs.http]]
    urls = ["https://mixnet.api.explorers.guru/api/mixnodes/YOUR_IDENTITY_KEY"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.host"
            rename = "mixnode_host"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.mix_port"
            rename = "mixnode_mix_port"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.verloc_port"
            rename = "mixnode_verloc_port"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.http_api_port"
            rename = "mixnode_http_api_port"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.sphinx_key"
            rename = "mixnode_sphinx_key"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.identity_key"
            rename = "mixnode_identity_key"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.version"
            rename = "mixnode_version"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.mix_id"
            rename = "mixnode_mix_id"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.owner"
            rename = "mixnode_owner"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.layer"
            rename = "mixnode_layer"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.proxy"
            rename = "mixnode_proxy"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.bonding_height"
            rename = "mixnode_bonding_height"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.is_unbonding"
            rename = "mixnode_is_unbonding"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.in_active"
            rename = "mixnode_in_active"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.in_reserve"
            rename = "mixnode_in_reserve"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.uptime"
            rename = "mixnode_uptime"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.status"
            rename = "mixnode_status"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.saturation"
            rename = "mixnode_saturation"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.profit_margin"
            rename = "mixnode_profit_margin"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.pledge_amount.amount"
            rename = "mixnode_pledge_amount"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.total_delegation.amount"
            rename = "mixnode_total_delegation"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.operating_cost.amount"
            rename = "mixnode_operating_cost"
    [[inputs.http.json_v2]]
         [[inputs.http.json_v2.field]]
            path = "mixnode.unique_delegations"
            rename = "mixnode_unique_delegations"

[[inputs.http]]
    urls = ["https://validator.nymtech.net/api/v1/status/mixnode/YOUR_ID_NODE/stake-saturation"]
    interval = "1m"
    data_format = "json_v2"
    name_suffix = "_nymcollector"
    [[inputs.http.json_v2]]
        [[inputs.http.json_v2.field]]
          path = "saturation"
          rename = "saturation"
          type = "float"

Make sure to replace the following variables:

YOUR_HOSTS with your Public IP
YOUR_IDENTITY_KEY with Identity Key (you can find it out here explorer.nymtech.net)
YOUR_ID_NODE with YOUR MIX ID like 248 (you can find it out here explorer.nymtech.net)

  1. Make sure to test all configs
telegraf --config /etc/telegraf/telegraf.conf --test
telegraf --config /etc/telegraf/telegraf.d/nym.conf --test
telegraf --config /etc/telegraf/telegraf.d/resource.conf --test

You may see some minor issues here but make sure that these tests produce some API request output, it means that everything is working.

  1. Restart telegraf
systemctl restart telegraf
  1. Make sure you install Prometheus. As of the time of writing, current LTS version is 2.45.2. Make sure you check the latest version here
sudo groupadd --system prometheus
sudo useradd -s /sbin/nologin --system -g prometheus prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.45.2/prometheus-2.45.2.linux-amd64.tar.gz
tar vxf prometheus*.tar.gz
cd prometheus*/
sudo mv prometheus /usr/local/bin
sudo mv promtool /usr/local/bin
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
sudo mv consoles /etc/prometheus
sudo mv console_libraries /etc/prometheus
sudo mv prometheus.yml /etc/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries
sudo chown -R prometheus:prometheus /var/lib/prometheus

Let's create a service file for Prometheus

sudo nano /etc/systemd/system/prometheus.service

Make sure to add the following

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Let's enable the service

sudo systemctl daemon-reload
sudo systemctl enable prometheus
sudo systemctl start prometheus
sudo systemctl status prometheus

If you see something along the lines above, it means Prometheus is running.

  1. Let's slightly adjust Prometheus settings, we only need the following lines in our /etc/prometheus/prometheus.yml config
scrape_configs:
  - job_name: "NYM mixnode"
    metrics_path: '/metrics'
    static_configs:
      - targets:
        - "0.0.0.0:9273"
  1. We are now ready to add dashboard to our Grafana. Make sure you log in and click on the following button

Select Prometheus

Make sure you add a correct URL, the default URL for Prometheus is http://localhost:9090

Scroll down and click Save & Test. Once you get a green tick, everything is good and you can simply click on Home at the top. If you get an error, make sure you check you entered a correct URL (For example, you are running Grafana on Docker and Prometheus outside of Docker, it create an extra issue. Or maybe the port is wrong).

  1. Let's create a dashboard
  1. We need to import it

Make sure you enter the following in this empty box. CAREFUL, LONG FILE

{
  "__inputs": [
    {
      "name": "DS_PROMETHEUS",
      "label": "Prometheus",
      "description": "",
      "type": "datasource",
      "pluginId": "prometheus",
      "pluginName": "Prometheus"
    }
  ],
  "__elements": {},
  "__requires": [
    {
      "type": "panel",
      "id": "annolist",
      "name": "Annotations list",
      "version": ""
    },
    {
      "type": "panel",
      "id": "gauge",
      "name": "Gauge",
      "version": ""
    },
    {
      "type": "grafana",
      "id": "grafana",
      "name": "Grafana",
      "version": "10.2.2"
    },
    {
      "type": "datasource",
      "id": "prometheus",
      "name": "Prometheus",
      "version": "1.0.0"
    },
    {
      "type": "panel",
      "id": "stat",
      "name": "Stat",
      "version": ""
    },
    {
      "type": "panel",
      "id": "timeseries",
      "name": "Time series",
      "version": ""
    }
  ],
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": null,
  "links": [
    {
      "asDropdown": true,
      "icon": "external link",
      "includeVars": false,
      "keepTime": false,
      "tags": [],
      "targetBlank": false,
      "title": "Other Dashboard",
      "tooltip": "",
      "type": "dashboards",
      "url": ""
    }
  ],
  "liveNow": false,
  "panels": [
    {
      "collapsed": false,
      "gridPos": {
        "h": 1,
        "w": 24,
        "x": 0,
        "y": 0
      },
      "id": 8,
      "panels": [],
      "title": "NYM Blockchain",
      "type": "row"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-BlYlRd"
          },
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "dark-red",
                "value": 0
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 0,
        "y": 1
      },
      "id": 20,
      "options": {
        "minVizHeight": 75,
        "minVizWidth": 75,
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "showThresholdLabels": false,
        "showThresholdMarkers": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "code",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_uptime)",
          "legendFormat": "__auto",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Uptime Packets",
      "type": "gauge"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [
            {
              "options": {
                "0": {
                  "color": "dark-purple",
                  "index": 1,
                  "text": "Active"
                },
                "1": {
                  "color": "dark-red",
                  "index": 0,
                  "text": "Deactive"
                }
              },
              "type": "value"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "semi-dark-blue",
                "value": 1
              },
              {
                "color": "blue",
                "value": 2
              },
              {
                "color": "light-blue",
                "value": 3
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 2,
        "y": 1
      },
      "id": 48,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 50,
          "valueSize": 38
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "http_nymcollector_epoch_current_epoch_id",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Epoch",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [
            {
              "options": {
                "0": {
                  "color": "dark-red",
                  "index": 0,
                  "text": "Inactive"
                }
              },
              "type": "value"
            },
            {
              "options": {
                "from": 1,
                "result": {
                  "color": "dark-blue",
                  "index": 1,
                  "text": "Active"
                },
                "to": 1000000
              },
              "type": "range"
            }
          ],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 4,
        "y": 1
      },
      "id": 42,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "http_nymcollector_packets_received_since_last_update",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Status",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-YlBl"
          },
          "mappings": [],
          "max": 3,
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "#ff00a0",
                "value": 1
              },
              {
                "color": "blue",
                "value": 2
              },
              {
                "color": "light-blue",
                "value": 3
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 6,
        "y": 1
      },
      "id": 41,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_layer)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Layer",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-YlBl"
          },
          "mappings": [],
          "max": 3,
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "#ff00a0",
                "value": 1
              },
              {
                "color": "blue",
                "value": 2
              },
              {
                "color": "light-blue",
                "value": 3
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 8,
        "y": 1
      },
      "id": 50,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_mix_id)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Id",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [
            {
              "options": {
                "0": {
                  "color": "dark-green",
                  "index": 0,
                  "text": "OK"
                }
              },
              "type": "value"
            },
            {
              "options": {
                "from": 1,
                "result": {
                  "color": "dark-red",
                  "index": 1,
                  "text": "NOK"
                },
                "to": 10000
              },
              "type": "range"
            }
          ],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 3,
        "x": 10,
        "y": 1
      },
      "id": 24,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 9,
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_explicitly_dropped_since_last_update)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packets Last Update Drop Packet",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-BlYlRd"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 12,
            "gradientMode": "hue",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineStyle": {
              "fill": "solid"
            },
            "lineWidth": 2,
            "pointSize": 8,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": [
          {
            "matcher": {
              "id": "byName",
              "options": "sum by(mixnode) (http_nymcollector_mixnode_uptime{group=\"nym\"})"
            },
            "properties": [
              {
                "id": "displayName",
                "value": "Uptime Packet"
              }
            ]
          }
        ]
      },
      "gridPos": {
        "h": 8,
        "w": 4,
        "x": 13,
        "y": 1
      },
      "id": 25,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.0.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_uptime)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Uptime Packets Current",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-BlYlRd"
          },
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 17,
        "y": 1
      },
      "id": 51,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 30,
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_last_day)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "1 Days Packet",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "palette-classic"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "series",
            "axisLabel": "Rewards",
            "axisPlacement": "right",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 12,
            "gradientMode": "hue",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "smooth",
            "lineStyle": {
              "fill": "solid"
            },
            "lineWidth": 2,
            "pointSize": 8,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 5,
        "x": 19,
        "y": 1
      },
      "id": 49,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": false
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.2.6",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "http_nymcollector_estimated_total_node_reward / 1000000",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "builder",
          "expr": "http_nymcollector_estimated_operator_reward{group=\"nym\", rewards=\"rewards-estimation\"} / 1000000",
          "hide": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "B"
        },
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "builder",
          "expr": "http_nymcollector_estimated_delegators_reward{group=\"nym\", rewards=\"rewards-estimation\"} / 1000000",
          "hide": false,
          "legendFormat": "__auto",
          "range": true,
          "refId": "C"
        }
      ],
      "title": "Reward Live",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "gridPos": {
        "h": 1,
        "w": 6,
        "x": 0,
        "y": 5
      },
      "id": 44,
      "options": {
        "limit": 10,
        "navigateAfter": "10m",
        "navigateBefore": "10m",
        "navigateToPanel": true,
        "onlyFromThisDashboard": false,
        "onlyInTimeRange": false,
        "showTags": true,
        "showTime": true,
        "showUser": true
      },
      "title": "PORT",
      "type": "annolist"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "#00ffd77a",
                "value": 0
              },
              {
                "color": "dark-green",
                "value": 90
              },
              {
                "color": "dark-red",
                "value": 100
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 3,
        "x": 6,
        "y": 5
      },
      "id": 36,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 1
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_saturation) * 100",
          "legendFormat": "__auto",
          "range": true,
          "refId": "A"
        }
      ],
      "title": "Saturation",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [
            {
              "options": {
                "0": {
                  "color": "dark-green",
                  "index": 0,
                  "text": "OK"
                }
              },
              "type": "value"
            },
            {
              "options": {
                "from": 1,
                "result": {
                  "color": "dark-red",
                  "index": 1,
                  "text": "NOK"
                },
                "to": 10000
              },
              "type": "range"
            }
          ],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 4,
        "x": 9,
        "y": 5
      },
      "id": 45,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 12,
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_explicitly_dropped_since_startup)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packets Startup Update Drop Packet",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-YlRd"
          },
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 2,
        "x": 17,
        "y": 5
      },
      "id": 52,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "horizontal",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "text": {
          "titleSize": 30,
          "valueSize": 30
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_last_hour)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "1 Hour Packet",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "dark-purple",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 2,
        "x": 0,
        "y": 6
      },
      "id": 38,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_http_api_port)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "API Port",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "dark-blue",
                "value": 0
              },
              {
                "color": "semi-dark-red",
                "value": 50
              },
              {
                "color": "#EAB839",
                "value": 60
              },
              {
                "color": "super-light-yellow",
                "value": 70
              },
              {
                "color": "super-light-orange",
                "value": 80
              },
              {
                "color": "light-green",
                "value": 90
              },
              {
                "color": "green",
                "value": 93
              },
              {
                "color": "semi-dark-green",
                "value": 96
              },
              {
                "color": "dark-green",
                "value": 100
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 2,
        "x": 2,
        "y": 6
      },
      "id": 39,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_mix_port)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Mixnode Port",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "percentage",
            "steps": [
              {
                "color": "text",
                "value": null
              },
              {
                "color": "dark-yellow",
                "value": 0
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 2,
        "x": 4,
        "y": 6
      },
      "id": 40,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_mixnode_verloc_port)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Verloc Port",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-GrYlRd"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 37,
            "gradientMode": "hue",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "smooth",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "never",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 0,
        "y": 9
      },
      "id": 22,
      "options": {
        "legend": {
          "calcs": [
            "count",
            "lastNotNull"
          ],
          "displayMode": "table",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.0.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "avg by(mixnode) (http_nymcollector_packets_received_since_startup)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Recieved Sinced Startup",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-YlRd"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "text",
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 37,
            "gradientMode": "hue",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "smooth",
            "lineWidth": 3,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 9,
        "y": 9
      },
      "id": 23,
      "options": {
        "legend": {
          "calcs": [
            "count",
            "lastNotNull"
          ],
          "displayMode": "table",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.0.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "avg by(mixnode) (http_nymcollector_packets_sent_since_startup)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Sent Sinced Startup",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "No Packet Recieved",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "#1a1a96",
                "value": null
              }
            ]
          },
          "unit": "none"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 6,
        "x": 18,
        "y": 9
      },
      "id": 19,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "avg by(mixnode) (http_nymcollector_packets_sent_since_startup)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Sent Startup",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 6,
        "x": 18,
        "y": 13
      },
      "id": 47,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "avg by(mixnode) (http_nymcollector_packets_received_since_startup)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Recieved Startup",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-BlYlRd"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "series",
            "axisGridShow": false,
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "hue",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "always",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "normal"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "dark-orange",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 0,
        "y": 17
      },
      "id": 26,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "table",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.0.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_received_since_last_update)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Recieved Since Last Update",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "continuous-BlPu"
          },
          "custom": {
            "axisBorderShow": false,
            "axisCenteredZero": false,
            "axisColorMode": "series",
            "axisGridShow": false,
            "axisLabel": "",
            "axisPlacement": "auto",
            "barAlignment": 0,
            "drawStyle": "line",
            "fillOpacity": 0,
            "gradientMode": "scheme",
            "hideFrom": {
              "legend": false,
              "tooltip": false,
              "viz": false
            },
            "insertNulls": false,
            "lineInterpolation": "linear",
            "lineStyle": {
              "fill": "solid"
            },
            "lineWidth": 1,
            "pointSize": 5,
            "scaleDistribution": {
              "type": "linear"
            },
            "showPoints": "auto",
            "spanNulls": true,
            "stacking": {
              "group": "A",
              "mode": "none"
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "dark-orange",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 8,
        "w": 9,
        "x": 9,
        "y": 17
      },
      "id": 27,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom",
          "showLegend": true
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "pluginVersion": "9.0.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_sent_since_last_update)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Sent Since Last Update",
      "type": "timeseries"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "noValue": "No Packet Recieved",
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "#1a1a96",
                "value": null
              }
            ]
          },
          "unit": "none"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 6,
        "x": 18,
        "y": 17
      },
      "id": 46,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_sent_since_last_update)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Sent Since Last Update",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "fieldConfig": {
        "defaults": {
          "color": {
            "mode": "thresholds"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "text",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 4,
        "w": 6,
        "x": 18,
        "y": 21
      },
      "id": 17,
      "options": {
        "colorMode": "background",
        "graphMode": "none",
        "justifyMode": "center",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "lastNotNull"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto",
        "wideLayout": true
      },
      "pluginVersion": "10.2.2",
      "targets": [
        {
          "datasource": {
            "type": "prometheus",
            "uid": "${DS_PROMETHEUS}"
          },
          "disableTextWrap": false,
          "editorMode": "builder",
          "expr": "sum by(mixnode) (http_nymcollector_packets_received_since_last_update)",
          "fullMetaSearch": false,
          "includeNullMetadata": true,
          "legendFormat": "__auto",
          "range": true,
          "refId": "A",
          "useBackend": false
        }
      ],
      "title": "Packet Last Update Recieved Since Last Update",
      "type": "stat"
    }
  ],
  "refresh": "",
  "schemaVersion": 38,
  "tags": [
    "nym",
    "mainnet",
    "tendermint"
  ],
  "templating": {
    "list": [
      {
        "auto": false,
        "auto_count": 30,
        "auto_min": "10s",
        "current": {
          "selected": false,
          "text": "1m",
          "value": "1m"
        },
        "hide": 0,
        "label": "Interval",
        "name": "interval",
        "options": [
          {
            "selected": true,
            "text": "1m",
            "value": "1m"
          },
          {
            "selected": false,
            "text": "10m",
            "value": "10m"
          },
          {
            "selected": false,
            "text": "30m",
            "value": "30m"
          },
          {
            "selected": false,
            "text": "1h",
            "value": "1h"
          },
          {
            "selected": false,
            "text": "6h",
            "value": "6h"
          },
          {
            "selected": false,
            "text": "12h",
            "value": "12h"
          },
          {
            "selected": false,
            "text": "1d",
            "value": "1d"
          },
          {
            "selected": false,
            "text": "7d",
            "value": "7d"
          },
          {
            "selected": false,
            "text": "14d",
            "value": "14d"
          },
          {
            "selected": false,
            "text": "30d",
            "value": "30d"
          }
        ],
        "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d",
        "queryValue": "",
        "refresh": 2,
        "skipUrlSync": false,
        "type": "interval"
      }
    ]
  },
  "time": {
    "from": "now-24h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "NYM Mixnode",
  "uid": "D74KO7zpO",
  "version": 4,
  "weekStart": ""
}

Click next. Then click here and select your Prometheus instance as a source of data. Click import.

  1. If everything was done correctly, you should be able to see the following dashboard

Grafana is super customizable, you can change the refresh rate and date interval as well.

There is also a way to have all you NYM nodes in one Grafana.

This is all for today! Make sure you send any comments to my discord - konskaya.zalupa 🐴