Yunohost backup via fail since 3.7.1

Support
,
1

Hello,
I updated to 3.7.1 two days ago and since then I have following issue:
I have a backup script scheduled by cron which run at night and do a yunohost backup create
This script is run as root user.
Since the upgrade, the script always fail with following message :

[2020-04-11 03:00:20] /usr/share/yunohost/hooks/backup/05-conf_ldap: ligne 14: slapcat : commande introuvable
[2020-04-11 03:00:20] Échec de l’exécution du script : /usr/share/yunohost/hooks/backup/05-conf_ldap
[2020-04-11 03:00:25] Traceback (most recent call last):
[2020-04-11 03:00:25]   File "/usr/bin/yunohost", line 214, in <module>
[2020-04-11 03:00:25]     timeout=opts.timeout,
[2020-04-11 03:00:25]   File "/usr/lib/python2.7/dist-packages/moulinette/__init__.py", line 135, in cli
[2020-04-11 03:00:25]     moulinette.run(args, output_as=output_as, password=password, timeout=timeout)
[2020-04-11 03:00:25]   File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/cli.py", line 425, in run
[2020-04-11 03:00:25]     ret = self.actionsmap.process(args, timeout=timeout)
[2020-04-11 03:00:25]   File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 527, in process
[2020-04-11 03:00:25]     return func(**arguments)
[2020-04-11 03:00:25]   File "/usr/lib/moulinette/yunohost/backup.py", line 2174, in backup_create
[2020-04-11 03:00:25]     backup_manager.collect_files()
[2020-04-11 03:00:25]   File "/usr/lib/moulinette/yunohost/backup.py", line 511, in collect_files
[2020-04-11 03:00:25]     self._collect_system_files()
[2020-04-11 03:00:25]   File "/usr/lib/moulinette/yunohost/backup.py", line 609, in _collect_system_files
[2020-04-11 03:00:25]     for hook, infos in ret.items()
[2020-04-11 03:00:25]   File "/usr/lib/moulinette/yunohost/backup.py", line 610, in <dictcomp>
[2020-04-11 03:00:25]     if any(result["state"] == "failed" for result in infos.values())}
[2020-04-11 03:00:25] AttributeError: 'builtin_function_or_method' object has no attribute 'items'

If I run the script in a shell (still as root user) then the backup is successfully created.
Any idea?

2

Hello,

I have the same error with a Borg script. Like you, I ran it yesterday as root and it worked. So I can confirm that the problem appeared with the last update.

3

Hm … that’s weird, that looks like a similar error I have while doing completely different things in a cron job as well … It looks like an unconfigured PATH variable … But I don’t understand why we’re only encountering this right now.

Can you confirm that it was working before ? (I guess yes…) Do you know what was the last version where it was confirmed to be working (was it 3.7.0.x ? Or did you upgrade from 3.6.x ?)

4

Hi Aleks,
Yes, i can confirm it was working before (the script is running for the last 2 years).
I did upgrade from 3.6.x to 3.7.0 last week and everything was working fine, only when I updated to 3.7.1 did the issue arise.
Hope this help

5

Hi, same problem here, since 3.7.1 too.

6

Yes, i confirm that the problem appeared after the last update:

apticron report [Thu, 09 Apr 2020 21:44:25 +0200]
========================================================================

apticron has detected that some packages need upgrading on:

mydomain.ynh.fr 
[ 192.168.0.20 ]

 The following packages are currently pending an upgrade:

moulinette 3.7.1
ssowat 3.7.1
yunohost 3.7.1
yunohost-admin 3.7.1.1

I upgraded from 3.7.0

7

Can any of you reproducing the issue paste the exact content of the cron script being ran including any #!/bin/bash at the top or lack of, and the location of the script (I’m guessing /etc/cron.daily ?)

8

Alright nevermind … For some reason some people decided that some cron jobs in linux are ran with PATH=/usr/bin:/bin, because why would you use the regular value that’s used everywhere else ?

Will make a fix…

9

Fix on the way here https://github.com/YunoHost/yunohost/commit/b0cd37aecad25bacd74c765101473d8ca8150d7d

We’ll make a 3.7.1.1 soonish

3 Likes
10

Should be fixed in 3.7.1.1

Thanks for the feedback!

2 Likes
11

Problem solved with 3.7.1.1, thanks @Aleks :hugs:

12

Perfect! Problem solved, thank you :slightly_smiling_face:

13

Same here, it worked after the update! Thanks!

14

I have a problem in my backups since the 16th of april, I don’t know if it’s related, but I a problem with similar python files (moulinette).

I made the update (to 3.7.1.1) but it’s still the same:

Info: Collecting files to be backed up for the app 'roundcube'…
Info: [+++.................] > Loading installation settings...
Info: [###+++..............] > Backing up the main app directory...
Info: [######++++..........] > Backing up nginx web server configuration...
Info: [##########+++.......] > Backing up php-fpm configuration...
Info: [#############+++....] > Backing up the MySQL database...
Info: Creating a backup archive from the collected files…
Traceback (most recent call last):
  File "/usr/bin/yunohost", line 218, in <module>
    timeout=opts.timeout,
  File "/usr/lib/python2.7/dist-packages/moulinette/__init__.py", line 135, in cli
    moulinette.run(args, output_as=output_as, password=password, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/cli.py", line 425, in run
    ret = self.actionsmap.process(args, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 527, in process
    return func(**arguments)
  File "/usr/lib/moulinette/yunohost/backup.py", line 2178, in backup_create
    backup_manager.backup()
  File "/usr/lib/moulinette/yunohost/backup.py", line 754, in backup
    method.mount_and_backup(self)
  File "/usr/lib/moulinette/yunohost/backup.py", line 1591, in mount_and_backup
    self.backup()
  File "/usr/lib/moulinette/yunohost/backup.py", line 1892, in backup
    tar.add(path['source'], arcname=path['dest'])
  File "/usr/lib/python2.7/tarfile.py", line 2032, in add
    recursive, exclude, filter)
  File "/usr/lib/python2.7/tarfile.py", line 2032, in add
    recursive, exclude, filter)
  File "/usr/lib/python2.7/tarfile.py", line 2032, in add
    recursive, exclude, filter)
  File "/usr/lib/python2.7/tarfile.py", line 2032, in add
    recursive, exclude, filter)
  File "/usr/lib/python2.7/tarfile.py", line 2028, in add
    self.addfile(tarinfo)
  File "/usr/lib/python2.7/tarfile.py", line 2048, in addfile
    buf = tarinfo.tobuf(self.format, self.encoding, self.errors)
  File "/usr/lib/python2.7/tarfile.py", line 1008, in tobuf
    return self.create_gnu_header(info)
  File "/usr/lib/python2.7/tarfile.py", line 1039, in create_gnu_header
    return buf + self._create_header(info, GNU_FORMAT)
  File "/usr/lib/python2.7/tarfile.py", line 1124, in _create_header
    itn(info.get("mtime", 0), 12, format),
  File "/usr/lib/python2.7/tarfile.py", line 221, in itn
    s = chr(n & 0377) + s
TypeError: unsupported operand type(s) for &: 'float' and 'int'
15

Meh, that one looks more like a bug inside the tar lib itself … Or we’re trying to backup something funky …

Do you reproduce the issue everytime ?

16

I don’t think we added anything special into our files.

I’ve just tried to launch those commands:

sudo yunohost backup create --system data_mail
and
sudo yunohost backup create --apps

both succeed.

Success! Backup created
name: 20200421-132713
results: 
  apps: 
    diagramsnet: Success
    etherpad_mypads: Success
    mattermost: Success
    mumbleserver: Success
    netdata: Success
    nextcloud: Success
    onlyoffice: Success
    roundcube: Success
    sogo: Success
  system: 
size: 1346234136

it only happens if I backup the system “data_home”:

sudo yunohost backup create --system data_home

It happens every time. Here is my /home/yunohost.backup/archives :

it worked until the 15th april (no info.json file after that):

-rw-r--r-- 1 root  root 687M Apr 13 17:32 20200413-153002.tar.gz
-rw-r--r-- 1 root  root 2.9K Apr 14 17:32 20200414-153001.info.json
-rw-r--r-- 1 root  root 1.2G Apr 14 17:32 20200414-153001.tar.gz
-rw-r--r-- 1 root  root 2.9K Apr 15 17:32 20200415-153002.info.json
-rw-r--r-- 1 root  root 1.2G Apr 15 17:32 20200415-153002.tar.gz
-rw-r--r-- 1 root  root 361M Apr 16 17:30 20200416-153002.tar.gz
-rw-r--r-- 1 root  root 361M Apr 17 17:30 20200417-153001.tar.gz
-rw-r--r-- 1 root  root 361M Apr 18 17:30 20200418-153002.tar.gz
-rw-r--r-- 1 root  root 361M Apr 19 17:30 20200419-153001.tar.gz
-rw-r--r-- 1 root  root 361M Apr 20 17:30 20200420-153001.tar.gz
-rw-r--r-- 1 root  root 361M Apr 21 13:44 20200421-114356.tar.gz
-rw-r--r-- 1 root  root 361M Apr 21 13:56 20200421-115608.tar.gz
-rw-r--r-- 1 root  root 361M Apr 21 15:14 20200421-131336.tar.gz
-rw-r--r-- 1 root  root  18M Apr 21 15:24 20200421-132352.tar.gz
-rw-r--r-- 1 root  root 361M Apr 21 15:24 20200421-132407.tar.gz

moreover, if I try to enter the admin > backup module, I get this error:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/api.py", line 439, in process
    ret = self.actionsmap.process(arguments, timeout=30, route=_route)
  File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 527, in process
    return func(**arguments)
  File "/usr/lib/moulinette/yunohost/backup.py", line 2289, in backup_list
    d[a] = backup_info(a, human_readable=human_readable)
  File "/usr/lib/moulinette/yunohost/backup.py", line 2329, in backup_info
    if "info.json" in tar.getnames():
  File "/usr/lib/python2.7/tarfile.py", line 1846, in getnames
    return [tarinfo.name for tarinfo in self.getmembers()]
  File "/usr/lib/python2.7/tarfile.py", line 1838, in getmembers
    self._load()        # all members, we first have to
  File "/usr/lib/python2.7/tarfile.py", line 2419, in _load
    tarinfo = self.next()
  File "/usr/lib/python2.7/tarfile.py", line 2350, in next
    self.fileobj.seek(self.offset - 1)
  File "/usr/lib/python2.7/gzip.py", line 443, in seek
    self.read(1024)
  File "/usr/lib/python2.7/gzip.py", line 268, in read
    self._read(readsize)
  File "/usr/lib/python2.7/gzip.py", line 319, in _read
    uncompress = self.decompress.decompress(buf)
error: Error -3 while decompressing: invalid block type
17

I’ve discovered something. I’ve checked out a git repository in one of the home folder (258 sub-directories, 1158 files), and it’s after that the backups stopped working. I’ve moved the git repository somewhere else, and now the backups are working again!

There is nothing special in this repository, some pdf, some yaml, md, java and images files…

I might open a new issue about this to investigate further (I’ve done it there: https://github.com/YunoHost/issues/issues/1570). I don’t mind having this repository elsewhere (I don’t need it into /home), but I suppose it’s better yunohost doesn’t fail to backup in such a case… (I can compress the whole directory with “tar cfvz”, so I suppose it’s a problem with the python implementation)

18

as explained at the end of the issue mentioned above, the problem was caused by a static site generator (hugo), created a wrong timestamp for a generated folder, which caused tarfile.py to crash.

19

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.