admin/raven
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

build 8.0.8 runaway log file fixes

Browse Source
This commit is contained in:
John Cardinal
2022-09-11 17:55:18 +00:00
parent 93cbbdf2bc
commit 253228abbe
6 changed files with 9 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
devdocs/todo.txt
View File

@@ -2,42 +2,14 @@
todo: URGENT - RUNAWAY LOGGING EATING UP SERVER SPACE
Problem:
AyaNova log and system journal eating up all disk space due to postgres being down for unknown reason and ayanova repeatedly trying to do stuff in this condition and then issuing errors and LOGGING
potential fixes: TODO HERE:
FIXED Detect postgres is unreachable and stop digging a hole, wait to resolve it before continuing on again
FIXED don't dual log to system journal logs and ayanova log
Ayanova is not only logging but also directing it's output to the terminal window which in linux system service is being logged also to the journal logs
AyaNova should only ouput to it's log file, not to terminal with the sole exception of the initial boot up period
This may be a configuration in the logging library set this way??
TODO: Add this to the standard server config script
cap system journal logs in linux to 250mb or some reasonable value
https://linuxhandbook.com/clear-systemd-journal-logs/
TODO: cap ayanova logs in configuration
something in here, but maybe it's config is different compare with code for logrotate in use now and see what the exact settings are
https://github.com/NLog/NLog/wiki/Configuration-file
https://github.com/NLog/NLog/wiki/File-target#archival-options
https://nlog-project.org/config/
NOTE: it may make sense to swtich from rotating every week to rotating on size instead, after all why force to rotate a nearly empty log?
Log issue final steps to complete:
TODO: update build number when done and repost
TODO: test in devops see what the what
DONE: If detect db is unavailable then log that and stop job processor etc basically freeze the server jobs until the db becomes available
server state db unavailble not just at boot but during ops
quick fixes:
https://support.hostway.com/hc/en-us/articles/360001972270-How-to-clean-log-files-in-Linux
https://linuxhandbook.com/clear-systemd-journal-logs/
Lessons learned:
Use less to view humongous log files, it's perfect for it using the commands properly
@@ -1426,6 +1398,6 @@ https://www.ayanova.com/download/next/ayanova-linux-x64-server.zip
https://www.ayanova.com/download/next/ayanova-windows-x64-lan-setup.exe
Current v8 docs home: https://www.ayanova.com/docs/next
BUILD 8.0.7 CHANGES OF NOTE
BUILD 8.0.8 CHANGES OF NOTE
Subscription / perpetual license code and stuff