xianqu
December 24, 2023, 10:28am
1
Hardware: VPS bought onlineYunoHost version: 11.2.8.2I have access to my server : through the webadminAre you in a special context or did you perform some particular tweaking on your YunoHost instance ? : no
L’installation de peertube n’a pas répondu, j’ai essayé plusieurs fois et attendu des heures et je n’arrive toujours pas à l’installer !
wbk
December 24, 2023, 10:45am
2
Hi xiangqu,
Bienvenue au forum!
Je ne parle pas Francais, je continues en Anglais
I seem to recall Peertube needs quite a bit of RAM when installing (but it was a long time ago I did). How much RAM does your VPS have?
Is there any info at the bottom of the screen ,
You can pull it upwards, to give information about the current step.
If that does not work, could you close the browser tab and login again, and go to tools → logs
and click on the log for Peertube installation. Maybe it takes a while, it would look like this:
after that you can press the green button to upload the error and share the link with us.
The VPS must have the option to log in via SSH. Could you try figuring out how to do that? If the installation via the webadmin hangs, it is difficult to troubleshoot. Installing via SSH could give more info.
1 Like
Que disent les journaux d’installation ?
xianqu
December 24, 2023, 11:43am
4
xianqu
December 24, 2023, 11:43am
5
2023-12-24 13:36:46,791: DEBUG - error An unexpected error occurred: “https://registry.yarnpkg.com/date-fns/-/date-fns-2.30.0.tgz: ESOCKETTIMEDOUT”.yarn install | Yarn for documentation about this command.
I had the same issue in the previous days.It’s an upstream issue, the dependency (Yarn) server being badly reachable I guess .
You may retry now or later, I don’t know how it will be fixed.
2 Likes
opened 10:21AM - 17 Jul 20 UTC
closed 01:29PM - 02 Jan 21 UTC
fixed-in-modern
### Bug description
Windows build of our electron app is consistently failing. … `yarn install --frozen-lockfile` failed to download `https://registry.yarnpkg.com/date-fns/-/date-fns-2.12.0.tgz` and `https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.9.1.tgz`. Failure to download `@material-ui/icons` was reported with `ESOCKETTIMEDOUT`. However, I expected the buildhost to have a well-provisioned network as it was a Github Actions runner. Linux build was working fine.
I assumed that high-latency disk IO may be a reason and managed to get a test-case that reproduces the issue reliably: `ESOCKETTIMEDOUT` is reliably triggered on Linux when small and realistic delay (8ms) is injected to disk IO system calls.
`ESOCKETTIMEDOUT` being reported because of slow disk IO is very confusing behavior, as it sounds like temporary network error while the root cause is different. It does not match my understanding of _"Developing Javascript projects shouldn't leave the door open to surprises"_ motto, so I'm reporting this test-case as a separate issue despite possible duplicates in the issue tracker. :slightly_smiling_face:
**Command**
```sh
ubuntu $ docker run --privileged --rm -ti --net=host node:12-buster /bin/bash
docker # apt update && apt install -y strace
docker # mkdir ~/prj && cd ~/prj
docker # strace -f -o /dev/null -e trace=stat -e inject=stat:delay_exit=8000 yarn add @material-ui/icons@^4.5.1
```
**What is the current behavior?**
```
yarn add v1.22.4
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
error An unexpected error occurred: "https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.9.1.tgz: ESOCKETTIMEDOUT".
info If you think this is a bug, please open a bug report with the information provided in "/root/prj/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
```
**What is the expected behavior?**
If I run exactly same command with delay_exit=1 (0.001ms) instead of delay_exit=8000 (8ms), I get the expected behavior:
```
yarn add v1.22.4
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > @material-ui/icons@4.9.1" has unmet peer dependency "@material-ui/core@^4.0.0".
warning " > @material-ui/icons@4.9.1" has unmet peer dependency "react@^16.8.0".
warning " > @material-ui/icons@4.9.1" has unmet peer dependency "react-dom@^16.8.0".
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 3 new dependencies.
info Direct dependencies
└─ @material-ui/icons@4.9.1
info All dependencies
├─ @babel/runtime@7.10.5
├─ @material-ui/icons@4.9.1
└─ regenerator-runtime@0.13.5
Done in 53.01s.
```
**Steps to Reproduce**
1. See _"Command"_ paragraph. Also, the test-case may need some small changes depending on environment.
First, `strace` adds some overhead on it's own and it may affect reproducibility. E.g. `yarn add @material-ui/icons@^4.5.1` is `Done in 5.76s.` in the very same environment without `strace` wrapper. That's why I compare strace-with-delay to strace-without-delay and don't compare it to "clean" run.
Second, I've taken `stat()` call from the following:
- I've launched `strace -f -o ~/yarn-trace yarn add @material-ui/icons@^4.5.1`
- I've looked at the output of `grep -F AccessAlarmsRounded.d.ts ~/yarn-trace`. It had 5 `openat()` calls, 4 `lstat()` calls, 1 `stat()` call, 1 `chmod()` call. So I've taken `stat(/usr/local/share/.cache/yarn/v6/.../AccessAlarmsRounded.d.ts)` as a place to inject delay to.
Third, I've taken 8ms delay assuming that there is single `stat()` system call per unpacked file and I was emulating HDD-based system having 125 IOPS performance. It's all a ballpark estimate: 1ms delay works on my system, 2ms fails with `ESOCKETTIMEDOUT` once but manages to install a package after retry, 4ms and 8ms fail reliably.
Fourth, as soon as TCP buffering is involved (see comment on TCP ZeroWindow later), available network bandwidth and size of socket buffer may be also a factor playing a role in bug reproducibility. I've reproduced the bug with these exact variable values with Ubuntu 16.04 laptop connected by 100 Mbit/s link in Russian St. Petersburg and on Linode VM in Newark (see below).
Fifth, your `node` build may interact with OS kernel a bit differently, e.g. it may use `open()` instead of `openat()`. So, if the test-case fails for you, try to increase the injected latency for the disk-related system call or change a disk-related system call. I reproduced the issue on Ubuntu 18.04 VM in [Linode](https://www.linode.com/) Newark availability zone, but I had to use `openat` as a latency-injection point instead of `stat`. 4 `statx()` and 3 `openat()` syscalls were made for the aforementioned filename at that VM.
**Comments and assumptions**
SQLite has ["faster than FS" benchmarks](https://www.sqlite.org/fasterthanfs.html) showing that Windows had pretty bad performance (compared to Linux) while operating with lots of small files. Both `date-fns` and `@material-ui/icons` have thousands of files as well as packages mentioned in "Possible duplicates" section. That explains that Windows users are suffering way more from `ESOCKETTIMEDOUT` happening while installing packages with thousands of files.
@FredyC came to the same idea that high-latency HDD being used instead of low-latency SSD triggers the `ESOCKETTIMEDOUT` in https://github.com/yarnpkg/yarn/issues/6221#issuecomment-412844630
@Hinaser made an excellent comment describing packet capture https://github.com/yarnpkg/yarn/issues/5259#issuecomment-389096161 `yarn` probably stops reading from a socket (so client OS sends `TCP ZeroWindow`) and eventually closes the socket from the client side.
I assume that `node` or `yarn` is busy unpacking well-compressed tarball full of small files and does not restart reading from socket for long enough time, so `ESOCKETTIMEDOUT` is triggered. I assume that the code also does not disable socket timeout while putting stream in paused state.
I assume, the possible fix is to download `.tgz` to a temporary file with some timeouts for network interactions and to unpack it without any timeouts as disk can't write faster anyway. Unfortunately, I'm not familiar with yarn codebase to provide a good PR.
**Environment**
- Node Version: `12.18.2`
- Yarn v1 Version: `1.22.4`
- OS and version: Docker container `node:12-buster` running on top of Ubuntu 16.04 or 18.04
`yarn-error.log` is the following:
```
Arguments:
/usr/local/bin/node /opt/yarn-v1.22.4/bin/yarn.js add @material-ui/icons@^4.5.1
PATH:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Yarn version:
1.22.4
Node version:
12.18.2
Platform:
linux x64
Trace:
Error: https://registry.yarnpkg.com/@material-ui/icons/-/icons-4.9.1.tgz: ESOCKETTIMEDOUT
at ClientRequest.<anonymous> (/opt/yarn-v1.22.4/lib/cli.js:141375:19)
at Object.onceWrapper (events.js:421:28)
at ClientRequest.emit (events.js:315:20)
at TLSSocket.emitRequestTimeout (_http_client.js:709:9)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:327:22)
at TLSSocket.Socket._onTimeout (net.js:481:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
npm manifest:
No manifest
yarn manifest:
No manifest
Lockfile:
No lockfile
```
**Possible duplicates**:
- https://github.com/yarnpkg/yarn/issues/4890#issuecomment-414631998 — 27 Mib `grid-styled@4.1.0` with 29090 files on Windows
- https://github.com/yarnpkg/yarn/issues/5259#issuecomment-389096161 — 3 MiB `nyc@11.7.3` with 4742 files on Windows 10
- https://github.com/yarnpkg/yarn/issues/5540 — 32 MiB `material-design-icons@3.0.1` on macOS 10.13; 14 Mib `rxjs-6.5.3` on Windows
- https://github.com/yarnpkg/yarn/issues/5546 — 31 MiB `material-design-icons@3.0.1` on Windows
- https://github.com/yarnpkg/yarn/issues/5950 — large `material-design-icons` on Windows
- https://github.com/yarnpkg/yarn/issues/6115 — 0.2 MiB `rxjs-compat@6.2.2` with 3115 files on Windows 10
- https://github.com/yarnpkg/yarn/issues/6221 — 31 Mib `material-design-icons-3.0.1` with 89814 files on Windows 10
- https://github.com/yarnpkg/yarn/issues/6392 — 15 MiB `lottie-react-native-2.3.2` with 6275 files on Windows 10
- https://github.com/yarnpkg/yarn/issues/7171 — 1.5 MiB `rxjs-5.5.12` with 3661 files on Windows
- https://github.com/yarnpkg/yarn/issues/7455 — 1.2 MiB `@material-ui/icons@4.0.1` with 15667 files on Windows
- https://github.com/yarnpkg/yarn/issues/7581 — 4.6 MiB `npm-6.11.3` with 4086 files on Windows
- https://github.com/yarnpkg/yarn/issues/7738 — 0.6 Mib `@carbon/icons-react@0.0.1-beta.5` with 6165 files on Windows
- https://github.com/yarnpkg/yarn/issues/7873 — 0.5 Mib `core-js@2.6.11` with 1489 files on Windows
yarnpkg:master ← anhvut:fix-socket-timeout-slow-disk
opened 11:05AM - 30 Sep 20 UTC
Avoid message `There appears to be trouble with your network connection. Retryin… g...` and ends up with `ESOCKETTIMEDOUT` during install a large package. Fixes #8242 and many potential duplicates.
**Summary**
During install from a remote registry, the `tgz` package download is streamed to uncompress files at same time. When uncompress time is higher than the socket timeout, the network connection is lost to download next parts of tgz file.
This issue is likely to occur:
- on Windows (slow) NTFS filesystem
- when an antivirus is scanning in real-time
- when a large package is to be installed
- and/or the package contains lots of small files, causing a significant overhead
This issue can be worked around by increasing the `network-timeout` option. However, the filesystem overhead is not really linked to network socket. Therefore a more appropriate solution is preferred.
In order to fix the timeout error, the install process is split in two parts:
- the `tgz` package is downloaded first to yarn cache.
- then the downloaded file is streamed in read access to uncompress, compute checksum integrity.
This is just a technical fix, impact is very isolated, therefore there is no need to update `CHANGELOG.md`
**Test plan**
No regression on automated tests.
Yarn install failed on my machine with package @material-ui/icons 4.9.1 (16645 files) and default timeout 30s.
After the fix, yarn successfully install this package with network-timeout 5s only.
That’s an upstream issue… with slow disks.
system
January 26, 2024, 2:34pm
9
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
