🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:15:17 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 103 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:30:17 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 135 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:45:17 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 24 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:00:19 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:15:33 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 109 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:30:21 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 46 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:45:29 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:00:18 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:15:21 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 38 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:30:36 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 21 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:45:32 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 22 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:00:30 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 30 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:15:31 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 14 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:30:24 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 55 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:45:37 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:00:21 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 21 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:15:21 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 19 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:30:16 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 68 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:45:16 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 77 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:00:18 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 18 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:15:16 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 14 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:30:16 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 82 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:45:17 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 135 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:00:14 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 14 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:15:15 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 127 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:30:14 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 100 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:45:14 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 35 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:00:14 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 13 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:15:14 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 11 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:30:17 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:45:16 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 27 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:00:16 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:15:19 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 137 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:30:15 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 134 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:45:32 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 51 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:00:19 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 19 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:15:26 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:30:28 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 67 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:45:24 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:00:31 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 29 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:15:30 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 66 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:30:30 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 37 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:45:23 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 21 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:00:26 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 26 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:15:37 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 20 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:30:35 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 13 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:45:32 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 49 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:00:31 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 11 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:18:00 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 29 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:30:33 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 11 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:45:33 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 16 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:00:39 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:15:39 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:30:31 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 62 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:45:37 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 13 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:00:21 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 21 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:15:21 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:30:21 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:45:21 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 20 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 10:00:25 🚨 [db1] ERROR: SMART status unknown on /dev/sda. Output: [db2] WARNING: Replication lag is 17 seconds. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 13:15:16 🚨 [db2] WARNING: Replication lag is 62 seconds. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 14:15:18 🚨 [db2] WARNING: Replication lag is 63 seconds. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 15:15:19 🚨 [db2] WARNING: Replication lag is 69 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 15:30:19 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db2] WARNING: Replication lag is 105 seconds. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 16:15:18 🚨 [db2] WARNING: Replication lag is 135 seconds. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 16:45:19 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. OK: All checks passed. Traceback (most recent call last): File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 257, in main() File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 218, in main res = check_remote_disk(node["host"], node["ssh_user"], disk, node["name"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 85, in check_remote_disk out = ssh_command(host, user, cmd) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 76, in ssh_command ssh.connect(hostname=host, username=user, timeout=10) File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/client.py", line 386, in connect sock.connect(addr) TimeoutError: timed out OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 19:30:19 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db1] CRITICAL: Service postgresql@16-main.service not running! [db1] ERROR: Could not check replication: connection to server at "cluster.db1.genesishostingtechnologies.com" (38.102.127.174), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? [db2] WARNING: Replication lag is 90 seconds. Failed to send Mastodon DM: This page is not correct - Mastodon Glitch Edition
Mastodon Glitch Edition

We're sorry, but something went wrong on our end.

OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 20:30:16 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 20:45:16 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:00:17 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:15:17 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:30:15 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [db2] WARNING: Replication lag is 66 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:45:19 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [db2] WARNING: Replication lag is 106 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:00:15 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:15:32 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [db2] WARNING: Replication lag is 95 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:30:30 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [db2] WARNING: Replication lag is 79 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:45:38 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:00:28 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:15:47 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:30:29 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:45:34 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:00:23 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:15:15 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:30:16 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:45:17 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog [db2] WARNING: Replication lag is 77 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:00:15 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:15:15 🚨 [shredder] WARNING: Pattern 'FATAL' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:30:16 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db2] WARNING: Replication lag is 72 seconds. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 02:30:18 🚨 [db2] WARNING: Replication lag is 89 seconds. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:00:16 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:15:18 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:30:17 🚨 [db2] WARNING: Replication lag is 135 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:45:17 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 04:00:16 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 04:45:21 🚨 [db2] WARNING: Replication lag is 64 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 05:00:27 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 08:00:20 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 08:15:20 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 09:45:24 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 10:00:18 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 10:30:16 🚨 [db2] WARNING: Replication lag is 75 seconds. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 12:00:31 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 12:45:24 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog [db2] WARNING: Replication lag is 94 seconds. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 13:00:33 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. OK: All checks passed. OK: All checks passed. OK: All checks passed. 🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 14:15:19 🚨 [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog OK: All checks passed. File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 181 f.write(" ^ SyntaxError: unterminated string literal (detected at line 181)