Running Zabbix on a MikroTik ROSE Data Server (RDS): Monitoring 500+ Network Devices

Why Zabbix?

At Link Technologies, Inc., we use Zabbix extensively to monitor our internal and production infrastructure. This includes servers, routers, BGP sessions, OSPF neighbors, network interfaces, production services, application queues, email processing, and TowerCoverage.com production systems.

Zabbix is extremely powerful and, for the most part, the software itself is free. Basic monitoring is relatively straightforward. Once you start getting into application monitoring, SQL queues, custom triggers, specialized SNMP values, and unusual network equipment, however, the learning curve becomes considerably steeper.

We have even spent engineering time directly with Zabbix engineers working through the best methods for monitoring some specialized systems. Zabbix can monitor an enormous number of things. Sometimes figuring out exactly how to monitor them is the difficult part.

The MikroTik ROSE Data Server

The MikroTik RDS2216-2XG-4S+4XS-2XQ, commonly referred to as the ROSE Data Server or RDS, is an interesting piece of hardware because it combines storage, networking, RouterOS, and container/application capabilities into a single platform.

• 20 U.2 NVMe storage slots
• 16-core 2 GHz ARM64 CPU
• 32 GB DDR4 RAM
• 2 × 100G QSFP28 ports
• 4 × 25G SFP28 ports
• 4 × 10G SFP+ ports
• 2 × 10G Ethernet ports
• RouterOS v7 ROSE Edition
• Container/application support
• Dual hot-swappable power supplies

For this particular project, we installed six 1 TB U.2 NVMe drives and configured them as a RAID5 array.

Step 1: Upgrade RouterOS and RouterBOARD Firmware

The first thing I normally do with any new MikroTik device is upgrade it to the latest stable RouterOS release unless there is a specific reason not to.

I initially connected an Ethernet cable and used MAC WinBox to access the RDS. After setting the password, I added a DHCP client to the Ethernet interface so the RDS could obtain an IP address.

Once it had an IP address, I disconnected the MAC WinBox session and reconnected using IP WinBox. MAC WinBox is extremely useful for initial configuration and recovery, but whenever possible I prefer using an IP connection.

From WinBox, go to:

System → Packages → Check for Updates

Select the current stable release and choose Download & Install. The RDS will download the RouterOS update and reboot.

After RouterOS has been upgraded, also check System → RouterBOARD. If the Upgrade Firmware version is newer than Current Firmware, click Upgrade and reboot the RDS again.

Step 2: Verify the Required Packages

If you're using an RDS, the rose-storage package should already be installed. Before continuing, make sure the device has the packages required for the functions we're going to use.

• RouterOS system package
• ROSE storage package
• Container/application support

If you are doing something similar on another compatible MikroTik device, you may need to download the appropriate additional RouterOS packages, upload them to the device, and reboot.

Step 3: Enable Container Device Mode

One important RouterOS v7 security feature is Device Mode. MikroTik restricts potentially sensitive functionality, including containers, depending on the device's operating mode.

Check the current configuration:

/system/device-mode print

In our case, the important part of the output was:

mode: advanced
container: no

Obviously, we cannot run Zabbix as an application/container if container functionality is disabled.

Enable it with:

/system/device-mode/update container=yes

RouterOS will tell you that physical action is required to activate the change. Depending on the hardware and RouterOS version, this means physically power-cycling the device or pressing the appropriate reset/mode button within the displayed time window.

This is intentional. MikroTik requires physical access for certain Device Mode changes to prevent a remote attacker from simply enabling powerful features after compromising a router.

Step 4: Build the Six-Drive RAID5 Array

Our RDS contains six 1 TB U.2 NVMe drives. RouterOS sees these as NVMe devices.

We're using RAID5 because this deployment does not require a tremendous amount of storage, and RAID5 gives us approximately five drives' worth of usable capacity while allowing one drive to fail.

If your application requires surviving two simultaneous drive failures, RAID6 should be considered instead.

Create the RAID5 array:

/disk add type=raid slot=raid5 raid-type=5 raid-device-count=6 raid-chunk-size=1M

Next, assign each NVMe device to the array:

/disk
set nvme1 raid-master=raid5_nvme raid-role=0
set nvme2 raid-master=raid5_nvme raid-role=1
set nvme3 raid-master=raid5_nvme raid-role=2
set nvme4 raid-master=raid5_nvme raid-role=3
set nvme5 raid-master=raid5_nvme raid-role=4
set nvme6 raid-master=raid5_nvme raid-role=5

Of course, substitute the appropriate NVMe device numbers for your particular RDS.

Step 5: Verify the RAID Array

Before formatting anything, verify that the RAID array assembled correctly.

/disk print detail where slot=raid5

Verify that the RAID is reporting a healthy state before continuing.

With six approximately 1 TB drives, our resulting RAID5 device provides approximately 5 TB of usable capacity before filesystem overhead.

Step 6: Format the RAID Array

Once we're satisfied that the array is healthy, we can format it for use by RouterOS and our applications.

/disk format raid5_nvme file-system=btrfs label=raid5

Depending on your requirements, BTRFS or EXT4 may be appropriate.

After formatting completes, check the volume again:

/disk print detail where slot=raid5

You should now see the RAID5 volume mounted and available. If you go to Files, the RAID5 storage should appear as a folder/volume.

Step 7: Configure MikroTik Apps to Use the RAID

Now that the RAID5 volume exists, we need to tell the MikroTik application system where it should store application downloads and application data.

Go to the application settings and configure the storage paths to use the RAID5 volume.

disk=raid5
media-path=raid5/apps
download-path=raid5/apps

This gives the RDS a persistent storage location for the applications and their associated data.

Step 8: Install Zabbix

Now comes the fun part. Once the storage and application environment are ready, enable the Zabbix application:

/app/set zabbix disabled=no

RouterOS will install and start Zabbix.

Once it is running, determine the IP address assigned to the application and connect to the Zabbix web interface to finish the normal Zabbix configuration.

Tuning Zabbix for 500+ Devices

The default configuration worked, but once we started monitoring more than 500 devices, some tuning was required.

Cache Size:

CacheSize=1G

Pollers:

StartPollers=50

ICMP Pingers:

StartPingers=20

These values should not simply be copied into every installation. Zabbix does an excellent job of monitoring itself. If your pollers, pingers, cache, or other internal processes start approaching 100% utilization, that is a good indication that the installation may need tuning.

Real-World Performance

This was the part of the project I was most interested in: how well would an RDS actually perform when running Zabbix against a production network?

After approximately a month of operation, the RDS is monitoring a city-wide network containing more than 500 network devices.

Think about that for a moment.

We're monitoring hundreds of routers, switches, and infrastructure devices, processing hundreds of monitoring values every second, maintaining the monitoring database, and running the Zabbix application while the RDS CPU is barely working.

More Than Just a Storage Server

This project illustrates something that I think people sometimes overlook about modern MikroTik hardware. The RDS isn't simply a box where you install a bunch of NVMe drives.

RouterOS has evolved considerably. Combining high-speed networking, NVMe storage, RouterOS, and containers/applications in a single appliance creates some very interesting possibilities.

Depending on your requirements, an RDS could potentially host:

• Zabbix
• Network monitoring collectors
• Network utilities
• DNS services
• Automation tools
• Internal applications
• Custom Linux containers
• Storage services

Production architecture still matters. Backups, redundancy, security, resource allocation, failure domains, and monitoring the monitoring system all need to be considered. But for this particular deployment, the RDS has performed extremely well.

Need a MikroTik ROSE Data Server?

Link Technologies, Inc. sells the MikroTik RDS2216-2XG-4S+4XS-2XQ ROSE Data Server along with MikroTik routers, switches, wireless equipment, optics, accessories, and other networking hardware.

Purchase the MikroTik ROSE Data Server:
https://shop.linktechs.net/rose-data-server-rds

Shop MikroTik Hardware:
https://shop.linktechs.net/mikrotik-2

How Link Technologies Inc. Can Help

Link Technologies, Inc. provides ISP-grade and enterprise-grade engineering and consulting services. Whether you need help deploying an RDS, building a Zabbix monitoring system, designing an ISP network, or troubleshooting an existing MikroTik environment, our engineers can help.

• MikroTik RouterOS engineering and troubleshooting
• ROSE Data Server deployment and configuration
• Zabbix installation, configuration, and performance tuning
• Custom Zabbix templates and monitoring
• ISP and WISP network design
• BGP, OSPF, VRF, ECMP, MPLS, VPLS, VXLAN, and routing design
• Network monitoring and observability
• IPv4 and IPv6 engineering
• DDoS preparedness and mitigation planning
• Network security and firewalling
• Virtualization, storage, and server infrastructure
• VoIP and Grandstream system configuration

Learn more about our engineering and consulting services here:
https://shop.linktechs.net/what-we-do

About the Author

This article was written by Dennis Burgess. Learn more about Dennis, his background, MikroTik experience, training, and network engineering work at:
https://shop.linktechs.net/about-dennis-burgess