2025-04-28 14:50:20 -04:00
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:15:17 🚨
2025-04-26 09:16:46 -04:00
[db1] ERROR: SMART status unknown on /dev/sda. Output:
2025-04-28 14:50:20 -04:00
[db2] WARNING: Replication lag is 103 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:30:17 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 135 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 19:45:17 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 24 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:00:19 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:15:33 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 109 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:30:21 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 46 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 20:45:29 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:00:18 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:15:21 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 38 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:30:36 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 21 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 21:45:32 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 22 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:00:30 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 30 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:15:31 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 14 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:30:24 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 55 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 22:45:37 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:00:21 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 21 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:15:21 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 19 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:30:16 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 68 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-26 23:45:16 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 77 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:00:18 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 18 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:15:16 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 14 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:30:16 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 82 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 00:45:17 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 135 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:00:14 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 14 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:15:15 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 127 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:30:14 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 100 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 01:45:14 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 35 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:00:14 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 13 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:15:14 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 11 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:30:17 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 02:45:16 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 27 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:00:16 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:15:19 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 137 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:30:15 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 134 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 03:45:32 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 51 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:00:19 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 19 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:15:26 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:30:28 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 67 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 04:45:24 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:00:31 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 29 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:15:30 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 66 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:30:30 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 37 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 05:45:23 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 21 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:00:26 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 26 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:15:37 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 20 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:30:35 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 13 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 06:45:32 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 49 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:00:31 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 11 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:18:00 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 29 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:30:33 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 11 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 07:45:33 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 16 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:00:39 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:15:39 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:30:31 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 62 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 08:45:37 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 13 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:00:21 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 21 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:15:21 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:30:21 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 09:45:21 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 20 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 10:00:25 🚨
[db1] ERROR: SMART status unknown on /dev/sda. Output:
[db2] WARNING: Replication lag is 17 seconds.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 13:15:16 🚨
[db2] WARNING: Replication lag is 62 seconds.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 14:15:18 🚨
[db2] WARNING: Replication lag is 63 seconds.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 15:15:19 🚨
[db2] WARNING: Replication lag is 69 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 15:30:19 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db2] WARNING: Replication lag is 105 seconds.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 16:15:18 🚨
[db2] WARNING: Replication lag is 135 seconds.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 16:45:19 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
2025-04-26 09:16:46 -04:00
Traceback (most recent call last):
2025-04-28 14:50:20 -04:00
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 257, in <module>
2025-04-26 09:16:46 -04:00
main()
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 218, in main
res = check_remote_disk(node["host"], node["ssh_user"], disk, node["name"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 85, in check_remote_disk
out = ssh_command(host, user, cmd)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 76, in ssh_command
ssh.connect(hostname=host, username=user, timeout=10)
2025-04-28 14:50:20 -04:00
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/client.py", line 386, in connect
sock.connect(addr)
TimeoutError: timed out
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 19:30:19 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db1] CRITICAL: Service postgresql@16-main.service not running!
[db1] ERROR: Could not check replication: connection to server at "cluster.db1.genesishostingtechnologies.com" (38.102.127.174), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
[db2] WARNING: Replication lag is 90 seconds.
Failed to send Mastodon DM: <!DOCTYPE html>
<html lang='en'>
<head>
<meta content='text/html; charset=UTF-8' http-equiv='Content-Type'>
<meta charset='utf-8'>
<title>This page is not correct
- Mastodon Glitch Edition</title>
<meta content='width=device-width,initial-scale=1' name='viewport'>
<link rel="stylesheet" crossorigin="anonymous" href="/packs/css/skins/glitch/mastodon-light-ba448ef4.chunk.css" media="not all and (prefers-color-scheme: dark)" integrity="sha256-P4BQPI06vGtRMsuyy44qvSrh1psmbDTSDhcI2ZZsDcI=" /><link rel="stylesheet" crossorigin="anonymous" href="/packs/css/skins/glitch/default-b4389951.chunk.css" media="(prefers-color-scheme: dark)" integrity="sha256-Ve2Dw/Y6VxR48zbYZoB8JdFFHsH9QiYGIZs5DLuH/U4=" />
<script src="/packs/js/common-8028e752d0f2aa93a5d6.js" crossorigin="anonymous" integrity="sha256-Yu9sY5Ca6EUBu337KcDfy59RIzPAUoLndPw2vudg2pg="></script>
<script src="/packs/js/flavours/glitch/error-10438b0fe55aa68f54dd.chunk.js" crossorigin="anonymous" integrity="sha256-DL1chTim8F/pYKgN8OlM4xXp5561mQ8KKtSZsRT9bwA="></script>
</head>
<body class='error'>
<div class='dialog'>
<div class='dialog__illustration'>
<img alt='Mastodon Glitch Edition' src='/oops.png'>
</div>
<div class='dialog__message'>
<h1>We're sorry, but something went wrong on our end.
</h1>
</div>
</div>
</body>
</html>
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 20:30:16 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 20:45:16 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:00:17 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:15:17 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:30:15 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[db2] WARNING: Replication lag is 66 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 21:45:19 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[db2] WARNING: Replication lag is 106 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:00:15 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:15:32 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[db2] WARNING: Replication lag is 95 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:30:30 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[db2] WARNING: Replication lag is 79 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 22:45:38 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:00:28 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:15:47 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:30:29 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-27 23:45:34 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:00:23 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:15:15 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:30:16 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 00:45:17 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
[db2] WARNING: Replication lag is 77 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:00:15 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:15:15 🚨
[shredder] WARNING: Pattern 'FATAL' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 01:30:16 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db2] WARNING: Replication lag is 72 seconds.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 02:30:18 🚨
[db2] WARNING: Replication lag is 89 seconds.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:00:16 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:15:18 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:30:17 🚨
[db2] WARNING: Replication lag is 135 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 03:45:17 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 04:00:16 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 04:45:21 🚨
[db2] WARNING: Replication lag is 64 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 05:00:27 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 08:00:20 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 08:15:20 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 09:45:24 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 10:00:18 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 10:30:16 🚨
[db2] WARNING: Replication lag is 75 seconds.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 12:00:31 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 12:45:24 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
[db2] WARNING: Replication lag is 94 seconds.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 13:00:33 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
OK: All checks passed.
🚨 Genesis Radio Multi-Node Healthcheck 2025-04-28 14:15:19 🚨
[mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
OK: All checks passed.
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 181
f.write("<table border='1' cellpadding='5' style='border-collapse: collapse;'>
^
SyntaxError: unterminated string literal (detected at line 181)
2025-04-28 15:03:18 -04:00
Traceback (most recent call last):
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 187, in <module>
main()
File "/home/doc/genesis-tools/miscellaneous/dbcheck1.py", line 113, in main
res = check_remote_disk(node["host"], node["ssh_user"], disk, node["name"])
^^^^^^^^^^^^^^^^^
NameError: name 'check_remote_disk' is not defined. Did you mean: 'check_remote_logs'?
2025-04-28 15:15:22 -04:00
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 15:15:11 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 105 seconds.
2025-04-28 19:39:48 -04:00
✅ Genesis Radio Healthcheck 2025-04-28 15:30:13: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 15:45:13 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-28 16:00:12: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 16:15:11 ⚠️
⚡ 3 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-28 16:30:11: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 16:45:11 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 88 seconds.
✅ Genesis Radio Healthcheck 2025-04-28 17:00:10: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-28 17:15:13: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 17:30:14 ⚠️
⚡ 2 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 17:45:15 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-28 18:00:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-28 18:15:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-28 18:30:11: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-28 18:45:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-28 19:00:15: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-28 19:15:12 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 101 seconds.
✅ Genesis Radio Healthcheck 2025-04-28 19:30:12: All systems normal.
2025-04-28 19:48:01 -04:00
✅ Genesis Radio Healthcheck 2025-04-28 19:45:14: All systems normal.
2025-04-28 20:02:46 -04:00
✅ Genesis Radio Healthcheck 2025-04-28 20:00:14: All systems normal.
2025-04-28 20:20:29 -04:00
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
2025-04-29 04:47:32 -04:00
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
/home/doc/dbcheck/bin/python: can't open file '/home/doc/genesis-tools/miscellaneous/dbcheck1.py': [Errno 2] No such file or directory
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 04:00:12 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 04:15:11 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 76 seconds.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 04:30:12 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 95 seconds.
✅ Genesis Radio Healthcheck 2025-04-29 04:45:14: All systems normal.
2025-04-29 05:20:21 -04:00
✅ Genesis Radio Healthcheck 2025-04-29 05:00:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 05:15:10: All systems normal.
2025-04-29 09:54:38 -04:00
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 05:30:14 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 05:45:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 06:00:12: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 06:15:13 ⚠️
⚡ 2 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- 💥 [db2] WARNING: Replication lag is 168 seconds.
✅ Genesis Radio Healthcheck 2025-04-29 06:30:11: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 06:45:16: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 07:00:16 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 07:15:14: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 07:30:13 ⚠️
⚡ 2 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- 💥 [db2] WARNING: Replication lag is 65 seconds.
✅ Genesis Radio Healthcheck 2025-04-29 07:45:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 08:00:13: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 08:15:14 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 08:30:14: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 08:45:12: All systems normal.
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 09:00:10 ⚠️
⚡ 4 warnings found:
- ⚠️ [shredder] ERROR: Could not read log /var/log/nginx/error.log: Error reading SSH protocol banner
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [db1] ERROR: Could not read log /var/log/syslog: Error reading SSH protocol banner
- ⚠️ [db2] ERROR: Could not read log /var/log/nginx/error.log: Error reading SSH protocol banner
✅ Genesis Radio Healthcheck 2025-04-29 09:15:07: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 09:30:12 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 73 seconds.
✅ Genesis Radio Healthcheck 2025-04-29 09:45:14: All systems normal.
2025-04-30 08:47:49 -04:00
✅ Genesis Radio Healthcheck 2025-04-29 10:00:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 10:15:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 10:30:15: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 10:45:29: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 11:00:11: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 11:15:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 11:30:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 11:45:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 12:00:12: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 12:15:17 ⚠️
⚡ 2 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 12:30:17: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 12:45:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 13:00:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 13:15:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 13:30:14: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 13:45:14: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 14:00:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 14:15:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 14:30:11: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 14:45:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 15:00:16: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 15:16:51 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 15:32:34 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 15:45:24: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 16:00:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 16:15:21: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 16:30:27: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 16:45:20: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 17:00:20: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 17:15:18 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 17:30:18 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 17:45:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 18:00:17: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 18:15:27 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 18:30:24: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 18:45:22 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 19:00:17: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 19:15:19: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 19:30:11: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 19:45:11 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 68 seconds.
✅ Genesis Radio Healthcheck 2025-04-29 20:00:17: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 20:15:21: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 20:30:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 20:45:10: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 21:00:14 ⚠️
⚡ 56 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 21:15:13 ⚠️
⚡ 3 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 21:30:26 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 21:45:28: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 22:00:26 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 22:15:28: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-29 22:30:22 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-29 22:45:26: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 23:00:23: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 23:15:33: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 23:30:23: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-29 23:45:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 00:00:23: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 00:15:24 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 00:30:16: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 00:45:23: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 01:00:20: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 01:15:17 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 01:30:19: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 01:45:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 02:00:21: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 02:15:25: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 02:30:21 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 02:45:13 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 60 seconds.
✅ Genesis Radio Healthcheck 2025-04-30 03:00:15: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 03:15:13 ⚠️
⚡ 1 warnings found:
- 💥 [db2] WARNING: Replication lag is 61 seconds.
✅ Genesis Radio Healthcheck 2025-04-30 03:30:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 03:45:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 04:00:11: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 04:15:20: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 04:30:18: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 04:45:21: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 05:00:19: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 05:15:21 ⚠️
⚡ 24 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 05:30:19: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 05:45:17 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 06:00:17: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 06:15:28 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 06:30:13: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 06:45:19 ⚠️
⚡ 5 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 07:00:13 ⚠️
⚡ 1 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 07:15:10: All systems normal.
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 07:30:12 ⚠️
⚡ 2 warnings found:
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 07:45:13: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 08:00:11: All systems normal.
Exception (client): Error reading SSH protocol banner[Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 663, in _read_timeout
x = self.__socket.recv(128)
^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner[Errno 104] Connection reset by peer
Exception (client): Error reading SSH protocol banner
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2369, in _check_banner
buf = self.packetizer.readline(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 395, in readline
buf += self._read_timeout(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/packet.py", line 665, in _read_timeout
raise EOFError()
EOFError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2185, in run
self._check_banner()
File "/home/doc/dbcheck/lib/python3.12/site-packages/paramiko/transport.py", line 2373, in _check_banner
raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
⚠️ Genesis Radio Warning Healthcheck 2025-04-30 08:15:12 ⚠️
⚡ 10 warnings found:
- ⚠️ [shredder] ERROR: Could not read log /var/log/nginx/error.log: Error reading SSH protocol banner[Errno 104] Connection reset by peer
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
- ⚠️ [mastodon] WARNING: Pattern 'ERROR' in /var/log/syslog
✅ Genesis Radio Healthcheck 2025-04-30 08:30:12: All systems normal.
✅ Genesis Radio Healthcheck 2025-04-30 08:45:18: All systems normal.
