Minor technical issues

[Q]

So, as we recurringly stumble upon technical issues, I feel we should log, or weblog about them.
I carry the hope, that through the continued writing about issues, their solution and our angle of observation being laid into the open, we will together uncover not only safe and sound ways to keep FutanariPalace running smooth and predictable.
But further I hope that in the long-term we will find a way or two to make FutanariPalace more fail-safe.
So as to hope that we can bring more people the joy of Futanari in the future.

In the light of the recent down-time from 5 days ago.. let's start this little Blog.


Posting rules: Futanari Palace staff herein denotes issues/incidents with bold letters. All other personnel shall refrain here from using bold letters.

[Q]

Downtime on 2025-05-16

Again the storage space ran full. I compressed the respective log files that became too large. Changed the logging to be a little less verbose. Increased our security buffer file, which we have for such emergencies.  After a restart of the services, the Forum ran again smoothly.

[Q]

Expired Certificate on 2025-05-26

Unfortunately our Let's Encrypt certificate renewal wasn't working as it should. I updated our internal Administration notes in regard to that.

[Keahi19]

Thank you for the work and speed.

[Q]

Downtime on 2025-09-29

Again the storage space ran full. I apologize for the inconvenience.

To prevent this from happening again, I will prioritize to add a simple "alerting" functionality to our server, to send an email to all admins, when the storage space is becoming small. I am scheduling that task for 12th October .

[Q]

Downtime 2026-01-06 from 19:09 to 19:58 UTC.

Storage space ran full. Again.

Alerting told me about it, that's a plus.

Also I should definitely disable the 'slow query log' of the Forum's MariaDB database server, as it's the cause for cluttering up the hard drive.

[Q]

Downtime on 2026-03-14

So uh.. I unfortunately opened up an old, compressed logfile of 1 GB size compressed, and typed Shift + g, to scroll to the end of the file. Unfortunately it was about 20 GB in size uncompressed. So the server behind FP kept on churning through the file, and as it hit it's RAM limit it started swapping to disk (using swap ram) while simultaneously decompressing the log file in chunks... My connection aborted and FP came to a stand still. Unexpectedly FP did not recover from this within 10 hours so our Owner swammy had to restart the hardware behind the server by hand. Which resolved the issue.

Thanks swammy.
I shall not try to scroll through a compressed logfile again, with no limitations set through c-group settings (with limits for cpu cycles or ram usage). Instead I should download such files in-before.

I apologize.