why dont we use the original log2ram ?

[alexandrecuer]

@TrystanLea : the original log2ram seems to be regurlarly maintained, it is writing the old files to /var/hdd.log so I was wondering why we use a different version ?

The original log2ram now uses rsync by default

https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L47 https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L75 https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L10

https://github.com/azlux/log2ram/commit/1af7db2b28899a5584c01bc2d0fa17f416aac965

[TrystanLea]

Im not sure to be honest, this was something pb66 introduced a while back, it looks like it might be good to revisit this again and evaluate the differences.

[alexandrecuer]

You have a few logrotate custom files but emonhub already has its own rotation implemention, using logging.handlers.RotatingFileHandler.

You need specifically logrotate for emoncms, as it maybe does not have any rotation inside ? or am I mistaking ?

[borpin]

You have a few logrotate custom files

Yes because not all systems rotate their files by default!

Don't confuse logrotate with log2ram.

log2ram works fine as long as logrotate is configured correctly.

You need specifically logrotate for emoncms, as it maybe does not have any rotation inside

Correct. we rely on logrotate to rotate the emoncms logs.

In answer to the first question, I think I asked this a while back. Key difference is that it doesn't use CRON but rather a service.timer.

[alexandrecuer]

I dont confuse log2ram and logrotate but the log2ram version used by the EmonScripts does itself the confusion : the logrotate configuration of 00_olddir is done whithin log2ram install script. This is not very nice ;-(

[alexandrecuer]

it is better to install log2ram before apache2 and other packages.... In the scripts, the install of log2ram is done at the end of the whole process

[borpin]

the logrotate configuration of 00_olddir is done whithin log2ram install script.

Yes as olddir is only relevant (for emoncms) when log2ram is installed.

it is better to install log2ram before apache2 and other packages

Because? I cannot see it makes any difference.

[edit] I run emoncms on an LXC. I don't use log2ram but I do want all the logs to rotate correctly in place.

[alexandrecuer]

I always have problems when installing log2ram at the end of the process. Since I install it first, i face no more problem but I am now using the official log2ram apt package. For me, it makes sense that it is a better/more natural practise to install the log file system manager before the different softwares begin writing on their log files... in the doc, it is written REBOOT before installing anything else (for example apache2) even if it is not implicit, it implies that log2ram is installed first

[borpin]

I always have problems when installing log2ram at the end of the process.

This might be because the persistent journald file exists. Trystan had the same problem.

I am now using the official log2ram apt package.

Interesting, did not know that was a thing. Where from? It isn't in the RaspberryOS repo list.

For me, it makes sense that it is a better/more natural practise to install the log file system manager before the different softwares begin writing on their log files.

Fair enough - I suspect that has never been considered.

You have a few logrotate custom files but emonhub already has its own rotation implemention, using logging.handlers.RotatingFileHandler.

Yes but see https://github.com/openenergymonitor/emonhub/issues/194. Personally, I think using logrotate especially when log2ram is in use, is better than using the inbuilt log rotation as the inbuilt rotation does not respect olddir directive.

No emoncms just sends the logs to file, no rotation.

Note that logrotate does not rotate it's own logs by default!

[alexandrecuer]

The original log2ram repo is there

https://github.com/azlux/log2ram

It explains how to install through apt :

https://github.com/azlux/log2ram#via-apt-recommended

It is quite instructive and there is a script to build the deb file :

https://github.com/azlux/log2ram/blob/master/build-packages.sh

[alexandrecuer]

I have started to use a simple script for my rotations, it is enough and i dont have the headache to maintain a custom conf for logrotate....

here is it :

#!/bin/bash
# archivage des logs inertes
# $1 dossier des log eg /var/log
# $2 software eg emoncms or emonhub
# on veut envoyer tous les fichiers .1 dans /var/log.old/software
# A automatiser avec cron :
# crontab -e
# pour exécuter cette tache tous les jours à 8h30
# 30 08 * * * /path/to/script/./archive_log.sh /var/log bios

# creating /var/log.old if it does not exist
if [ ! -d $1.old ]; then
    sudo mkdir $1.old
fi

# creating /var/log.old/software if it does not exist
if [ ! -d $1.old/$2 ]; then
    sudo mkdir $1.old/$2
fi

# reading the /var/log/software folder
for file in $1/$2/*
do
    if [[ $file == *".1" ]]; then
        fullname=$(echo $file | rev | cut -d"." -f2-  | rev)
        nb=$(echo $file | rev | cut -d"." -f1  | rev)
        name=$(echo $fullname | rev | cut -d"/" -f1  | rev)
        # relabelling .1, .2, .3... files in /var/log.old/software
        # to .2, .3, .4
        # limiting to 10 files max
        # using tmp files so not to get stuck
        for arch in $1.old/$2/$name.*
        do
            if [ -f $arch ]; then
                fullarch=$(echo $arch | rev | cut -d"." -f2-  | rev)
                nbarch=$(echo $arch | rev | cut -d"." -f1  | rev)
                if [[ $((nbarch)) == 10 ]]; then
                    sudo rm -Rf $arch
                fi
                if [ -f $arch ]; then
                    nbarch_increased=$((nbarch+1))
                    sudo mv $arch $fullarch.$nbarch_increased.tmp
                fi
            fi
        done
        for tmp_file in $1.old/$2/$name.*.tmp
        do
            if [ -f $tmp_file ]; then
                real_name=$(echo $tmp_file | rev | cut -d"." -f2-  | rev)
                sudo mv $tmp_file $real_name
            fi
        done
        # moving the .1 file to the /var/log.old/software folder
        sudo mv $file $1.old/$2/$name.$nb
    fi
done

[borpin]

I have started to use a simple script for my rotations

Which to me is crazy when Debian has a perfectly capable system that simply needs a config file to work. YMMV.

It explains how to install through apt :

Thanks. I was too lazy to look.

It'll be worth looking at at some point. The issue though is (as ever) migrating current users or making the update script account for the type of install in place.

I think this is an ain't broke; don't fix

[alexandrecuer]

Which to me is crazy when Debian has a perfectly capable system that simply needs a config file to work

What I find also crazy is to use a rotation manager (logrotate) on a sofware that already integrates its own rotation system (emonhub) :-).. What you need is not another rotation manager but just an archiving tool.

[borpin]

What you need is not another rotation manager but just an archiving tool.

Because it doesn't work with log2ram (creates rotated files that are then not rotated and saved by log2ram), has a limited life of rotated files and does not use olddir. Other than that it is fine :)