I: pbuilder: network access will be disabled during build I: Current time: Tue Oct 22 06:15:27 +14 2024 I: pbuilder-time-stamp: 1729527327 I: Building the build Environment I: extracting base tarball [/var/cache/pbuilder/trixie-reproducible-base.tgz] I: copying local configuration W: --override-config is not set; not updating apt.conf Read the manpage for details. I: mounting /proc filesystem I: mounting /sys filesystem I: creating /{dev,run}/shm I: mounting /dev/pts filesystem I: redirecting /dev/ptmx to /dev/pts/ptmx I: policy-rc.d already exists I: Copying source file I: copying [dask.distributed_2024.5.2+ds.1-7.dsc] I: copying [./dask.distributed_2024.5.2+ds.1.orig.tar.xz] I: copying [./dask.distributed_2024.5.2+ds.1-7.debian.tar.xz] I: Extracting source gpgv: Signature made Thu Aug 29 18:50:41 2024 gpgv: using RSA key 7E1AC3FA27A649284C323EF6695C429316C07BFE gpgv: issuer "jdg@debian.org" gpgv: Can't check signature: No public key dpkg-source: warning: cannot verify inline signature for ./dask.distributed_2024.5.2+ds.1-7.dsc: no acceptable signature found dpkg-source: info: extracting dask.distributed in dask.distributed-2024.5.2+ds.1 dpkg-source: info: unpacking dask.distributed_2024.5.2+ds.1.orig.tar.xz dpkg-source: info: unpacking dask.distributed_2024.5.2+ds.1-7.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying install-tests.patch dpkg-source: info: applying docs-use-local-images.patch dpkg-source: info: applying use-local-intersphinx-inventory.patch dpkg-source: info: applying use-youtube-nocookie.patch dpkg-source: info: applying use-local-favicon.patch dpkg-source: info: applying fall-back-to-ipv6-localhost.patch dpkg-source: info: applying skip-bokeh-tests.patch dpkg-source: info: applying skip-directory-test-when-root.patch dpkg-source: info: applying skip-pyarrow-tests.patch dpkg-source: info: applying adjust-timeouts.patch dpkg-source: info: applying mark-tests-flaky.patch dpkg-source: info: applying skip-32bit-incompatible-test.patch dpkg-source: info: applying skip-coverage-test.patch dpkg-source: info: applying fix-sphinx-config.patch dpkg-source: info: applying skip-memray.patch dpkg-source: info: applying fix-privacy-breaches.patch dpkg-source: info: applying 32-bits.patch dpkg-source: info: applying fix-version.patch I: Not using root during the build. I: Installing the build-deps I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/D01_modify_environment starting debug: Running on ionos1-amd64. I: Changing host+domainname to test build reproducibility I: Adding a custom variable just for the fun of it... I: Changing /bin/sh to bash '/bin/sh' -> '/bin/bash' lrwxrwxrwx 1 root root 9 Oct 21 16:15 /bin/sh -> /bin/bash I: Setting pbuilder2's login shell to /bin/bash I: Setting pbuilder2's GECOS to second user,second room,second work-phone,second home-phone,second other I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/D01_modify_environment finished I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/D02_print_environment starting I: set BASH=/bin/sh BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:globskipdots:hostcomplete:interactive_comments:patsub_replacement:progcomp:promptvars:sourcepath BASH_ALIASES=() BASH_ARGC=() BASH_ARGV=() BASH_CMDS=() BASH_LINENO=([0]="12" [1]="0") BASH_LOADABLES_PATH=/usr/local/lib/bash:/usr/lib/bash:/opt/local/lib/bash:/usr/pkg/lib/bash:/opt/pkg/lib/bash:. BASH_SOURCE=([0]="/tmp/hooks/D02_print_environment" [1]="/tmp/hooks/D02_print_environment") BASH_VERSINFO=([0]="5" [1]="2" [2]="32" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu") BASH_VERSION='5.2.32(1)-release' BUILDDIR=/build/reproducible-path BUILDUSERGECOS='second user,second room,second work-phone,second home-phone,second other' BUILDUSERNAME=pbuilder2 BUILD_ARCH=amd64 DEBIAN_FRONTEND=noninteractive DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all parallel=20 ' DIRSTACK=() DISTRIBUTION=trixie EUID=0 FUNCNAME=([0]="Echo" [1]="main") GROUPS=() HOME=/root HOSTNAME=i-capture-the-hostname HOSTTYPE=x86_64 HOST_ARCH=amd64 IFS=' ' INVOCATION_ID=350a2f94d4cf4409830d8be6f1095b76 LANG=C LANGUAGE=et_EE:et LC_ALL=C MACHTYPE=x86_64-pc-linux-gnu MAIL=/var/mail/root OPTERR=1 OPTIND=1 OSTYPE=linux-gnu PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path PBCURRENTCOMMANDLINEOPERATION=build PBUILDER_OPERATION=build PBUILDER_PKGDATADIR=/usr/share/pbuilder PBUILDER_PKGLIBDIR=/usr/lib/pbuilder PBUILDER_SYSCONFDIR=/etc PIPESTATUS=([0]="0") POSIXLY_CORRECT=y PPID=1538210 PS4='+ ' PWD=/ SHELL=/bin/bash SHELLOPTS=braceexpand:errexit:hashall:interactive-comments:posix SHLVL=3 SUDO_COMMAND='/usr/bin/timeout -k 24.1h 24h /usr/bin/ionice -c 3 /usr/bin/nice -n 11 /usr/bin/unshare --uts -- /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/r-b-build.S3dTZzvM/pbuilderrc_3Cpl --distribution trixie --hookdir /etc/pbuilder/rebuild-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/trixie-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/r-b-build.S3dTZzvM/b2 --logfile b2/build.log dask.distributed_2024.5.2+ds.1-7.dsc' SUDO_GID=110 SUDO_UID=105 SUDO_USER=jenkins TERM=unknown TZ=/usr/share/zoneinfo/Etc/GMT-14 UID=0 USER=root _='I: set' http_proxy=http://46.16.76.132:3128 I: uname -a Linux i-capture-the-hostname 6.1.0-26-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.112-1 (2024-09-30) x86_64 GNU/Linux I: ls -l /bin lrwxrwxrwx 1 root root 7 Aug 4 21:30 /bin -> usr/bin I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/D02_print_environment finished -> Attempting to satisfy build-dependencies -> Creating pbuilder-satisfydepends-dummy package Package: pbuilder-satisfydepends-dummy Version: 0.invalid.0 Architecture: amd64 Maintainer: Debian Pbuilder Team Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder This package was created automatically by pbuilder to satisfy the build-dependencies of the package being currently built. Depends: debhelper-compat (= 13), dh-sequence-python3, dh-sequence-sphinxdoc, libjs-sortable-tablesort, pybuild-plugin-pyproject, python-dask-doc, python-fsspec-doc, python-numpy-doc, python-pandas-doc, python3-all, python3-click (>= 6.6), python3-cloudpickle (>= 1.5.0), python3-cryptography, python3-dask (<< 2024.5.2.1~), python3-dask (>= 2024.5.2), python3-dask-sphinx-theme, python3-doc, python3-jinja2, python3-jupyter-server (>= 1.23.3-2), python3-locket, python3-msgpack, python3-numpy, python3-numpydoc, python3-packaging, python3-pandas, python3-psutil, python3-pytest, python3-pytest-asyncio, python3-pytest-rerunfailures, python3-pytest-timeout, python3-setuptools, python3-sortedcollections, python3-sphinx, python3-sphinx-click, python3-sphinx-design, python3-tblib, python3-toolz (>= 0.8.2), python3-tornado (>= 5), python3-urllib3, python3-versioneer, python3-yaml, python3-zict (>= 3.0.0), tzdata-legacy, uglifyjs dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'. Selecting previously unselected package pbuilder-satisfydepends-dummy. (Reading database ... 19799 files and directories currently installed.) Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ... Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ... dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested: pbuilder-satisfydepends-dummy depends on debhelper-compat (= 13); however: Package debhelper-compat is not installed. pbuilder-satisfydepends-dummy depends on dh-sequence-python3; however: Package dh-sequence-python3 is not installed. pbuilder-satisfydepends-dummy depends on dh-sequence-sphinxdoc; however: Package dh-sequence-sphinxdoc is not installed. pbuilder-satisfydepends-dummy depends on libjs-sortable-tablesort; however: Package libjs-sortable-tablesort is not installed. pbuilder-satisfydepends-dummy depends on pybuild-plugin-pyproject; however: Package pybuild-plugin-pyproject is not installed. pbuilder-satisfydepends-dummy depends on python-dask-doc; however: Package python-dask-doc is not installed. pbuilder-satisfydepends-dummy depends on python-fsspec-doc; however: Package python-fsspec-doc is not installed. pbuilder-satisfydepends-dummy depends on python-numpy-doc; however: Package python-numpy-doc is not installed. pbuilder-satisfydepends-dummy depends on python-pandas-doc; however: Package python-pandas-doc is not installed. pbuilder-satisfydepends-dummy depends on python3-all; however: Package python3-all is not installed. pbuilder-satisfydepends-dummy depends on python3-click (>= 6.6); however: Package python3-click is not installed. pbuilder-satisfydepends-dummy depends on python3-cloudpickle (>= 1.5.0); however: Package python3-cloudpickle is not installed. pbuilder-satisfydepends-dummy depends on python3-cryptography; however: Package python3-cryptography is not installed. pbuilder-satisfydepends-dummy depends on python3-dask (<< 2024.5.2.1~); however: Package python3-dask is not installed. pbuilder-satisfydepends-dummy depends on python3-dask (>= 2024.5.2); however: Package python3-dask is not installed. pbuilder-satisfydepends-dummy depends on python3-dask-sphinx-theme; however: Package python3-dask-sphinx-theme is not installed. pbuilder-satisfydepends-dummy depends on python3-doc; however: Package python3-doc is not installed. pbuilder-satisfydepends-dummy depends on python3-jinja2; however: Package python3-jinja2 is not installed. pbuilder-satisfydepends-dummy depends on python3-jupyter-server (>= 1.23.3-2); however: Package python3-jupyter-server is not installed. pbuilder-satisfydepends-dummy depends on python3-locket; however: Package python3-locket is not installed. pbuilder-satisfydepends-dummy depends on python3-msgpack; however: Package python3-msgpack is not installed. pbuilder-satisfydepends-dummy depends on python3-numpy; however: Package python3-numpy is not installed. pbuilder-satisfydepends-dummy depends on python3-numpydoc; however: Package python3-numpydoc is not installed. pbuilder-satisfydepends-dummy depends on python3-packaging; however: Package python3-packaging is not installed. pbuilder-satisfydepends-dummy depends on python3-pandas; however: Package python3-pandas is not installed. pbuilder-satisfydepends-dummy depends on python3-psutil; however: Package python3-psutil is not installed. pbuilder-satisfydepends-dummy depends on python3-pytest; however: Package python3-pytest is not installed. pbuilder-satisfydepends-dummy depends on python3-pytest-asyncio; however: Package python3-pytest-asyncio is not installed. pbuilder-satisfydepends-dummy depends on python3-pytest-rerunfailures; however: Package python3-pytest-rerunfailures is not installed. pbuilder-satisfydepends-dummy depends on python3-pytest-timeout; however: Package python3-pytest-timeout is not installed. pbuilder-satisfydepends-dummy depends on python3-setuptools; however: Package python3-setuptools is not installed. pbuilder-satisfydepends-dummy depends on python3-sortedcollections; however: Package python3-sortedcollections is not installed. pbuilder-satisfydepends-dummy depends on python3-sphinx; however: Package python3-sphinx is not installed. pbuilder-satisfydepends-dummy depends on python3-sphinx-click; however: Package python3-sphinx-click is not installed. pbuilder-satisfydepends-dummy depends on python3-sphinx-design; however: Package python3-sphinx-design is not installed. pbuilder-satisfydepends-dummy depends on python3-tblib; however: Package python3-tblib is not installed. pbuilder-satisfydepends-dummy depends on python3-toolz (>= 0.8.2); however: Package python3-toolz is not installed. pbuilder-satisfydepends-dummy depends on python3-tornado (>= 5); however: Package python3-tornado is not installed. pbuilder-satisfydepends-dummy depends on python3-urllib3; however: Package python3-urllib3 is not installed. pbuilder-satisfydepends-dummy depends on python3-versioneer; however: Package python3-versioneer is not installed. pbuilder-satisfydepends-dummy depends on python3-yaml; however: Package python3-yaml is not installed. pbuilder-satisfydepends-dummy depends on python3-zict (>= 3.0.0); however: Package python3-zict is not installed. pbuilder-satisfydepends-dummy depends on tzdata-legacy; however: Package tzdata-legacy is not installed. pbuilder-satisfydepends-dummy depends on uglifyjs; however: Package uglifyjs is not installed. Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ... Reading package lists... Building dependency tree... Reading state information... Initializing package states... Writing extended state information... Building tag database... pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0) The following NEW packages will be installed: autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} ca-certificates{a} debhelper{a} dh-autoreconf{a} dh-python{a} dh-strip-nondeterminism{a} docutils-common{a} dwz{a} file{a} fonts-font-awesome{a} fonts-glyphicons-halflings{a} fonts-lato{a} fonts-mathjax{a} gettext{a} gettext-base{a} groff-base{a} intltool-debian{a} libarchive-zip-perl{a} libargon2-1{a} libblas3{a} libbrotli1{a} libcares2{a} libcom-err2{a} libdebhelper-perl{a} libelf1t64{a} libexpat1{a} libfile-stripnondeterminism-perl{a} libgfortran5{a} libgssapi-krb5-2{a} libicu72{a} libjs-bootstrap{a} libjs-jquery{a} libjs-mathjax{a} libjs-sortable-tablesort{a} libjs-sphinxdoc{a} libjs-sprintf-js{a} libjs-underscore{a} libjson-perl{a} libk5crypto3{a} libkeyutils1{a} libkrb5-3{a} libkrb5support0{a} liblapack3{a} libmagic-mgc{a} libmagic1t64{a} libnghttp2-14{a} libnode115{a} libnorm1t64{a} libnsl2{a} libpgm-5.3-0t64{a} libpipeline1{a} libpython3-stdlib{a} libpython3.12-minimal{a} libpython3.12-stdlib{a} libreadline8t64{a} libsodium23{a} libtirpc-common{a} libtirpc3t64{a} libtool{a} libuchardet0{a} libuv1t64{a} libxml2{a} libyaml-0-2{a} libzmq5{a} m4{a} man-db{a} media-types{a} netbase{a} node-acorn{a} node-argparse{a} node-balanced-match{a} node-brace-expansion{a} node-cjs-module-lexer{a} node-esprima{a} node-js-yaml{a} node-minimatch{a} node-sprintf-js{a} node-uglify-js{a} node-undici{a} node-xtend{a} nodejs{a} openssl{a} po-debconf{a} pybuild-plugin-pyproject{a} python-babel-localedata{a} python-dask-doc{a} python-fsspec-doc{a} python-numpy-doc{a} python-pandas-doc{a} python-tinycss2-common{a} python3{a} python3-alabaster{a} python3-all{a} python3-anyio{a} python3-argon2{a} python3-arrow{a} python3-async-generator{a} python3-attr{a} python3-autocommand{a} python3-babel{a} python3-bcrypt{a} python3-bleach{a} python3-bs4{a} python3-build{a} python3-certifi{a} python3-cffi-backend{a} python3-chardet{a} python3-charset-normalizer{a} python3-click{a} python3-cloudpickle{a} python3-colorama{a} python3-cryptography{a} python3-dask{a} python3-dask-sphinx-theme{a} python3-dateutil{a} python3-decorator{a} python3-defusedxml{a} python3-doc{a} python3-docutils{a} python3-fastjsonschema{a} python3-fqdn{a} python3-fsspec{a} python3-html5lib{a} python3-idna{a} python3-imagesize{a} python3-inflect{a} python3-iniconfig{a} python3-installer{a} python3-isoduration{a} python3-jaraco.context{a} python3-jaraco.functools{a} python3-jinja2{a} python3-json-pointer{a} python3-jsonschema{a} python3-jsonschema-specifications{a} python3-jupyter-client{a} python3-jupyter-core{a} python3-jupyter-events{a} python3-jupyter-server{a} python3-jupyter-server-terminals{a} python3-jupyterlab-pygments{a} python3-locket{a} python3-markdown-it{a} python3-markupsafe{a} python3-mdurl{a} python3-minimal{a} python3-mistune{a} python3-more-itertools{a} python3-msgpack{a} python3-nbclient{a} python3-nbconvert{a} python3-nbformat{a} python3-numpy{a} python3-numpydoc{a} python3-overrides{a} python3-packaging{a} python3-pandas{a} python3-pandas-lib{a} python3-pandocfilters{a} python3-partd{a} python3-pkg-resources{a} python3-platformdirs{a} python3-pluggy{a} python3-prometheus-client{a} python3-psutil{a} python3-ptyprocess{a} python3-py{a} python3-pydata-sphinx-theme{a} python3-pygments{a} python3-pyproject-hooks{a} python3-pytest{a} python3-pytest-asyncio{a} python3-pytest-rerunfailures{a} python3-pytest-timeout{a} python3-pythonjsonlogger{a} python3-referencing{a} python3-requests{a} python3-rfc3339-validator{a} python3-rfc3986-validator{a} python3-rich{a} python3-roman{a} python3-rpds-py{a} python3-send2trash{a} python3-setuptools{a} python3-sniffio{a} python3-snowballstemmer{a} python3-sortedcollections{a} python3-sortedcontainers{a} python3-soupsieve{a} python3-sphinx{a} python3-sphinx-book-theme{a} python3-sphinx-click{a} python3-sphinx-design{a} python3-tabulate{a} python3-tblib{a} python3-terminado{a} python3-tinycss2{a} python3-toml{a} python3-toolz{a} python3-tornado{a} python3-traitlets{a} python3-typeguard{a} python3-typeshed{a} python3-typing-extensions{a} python3-tz{a} python3-uritemplate{a} python3-urllib3{a} python3-versioneer{a} python3-webcolors{a} python3-webencodings{a} python3-websocket{a} python3-wheel{a} python3-yaml{a} python3-zict{a} python3-zipp{a} python3-zmq{a} python3.12{a} python3.12-doc{a} python3.12-minimal{a} readline-common{a} sensible-utils{a} sgml-base{a} sphinx-common{a} sphinx-rtd-theme-common{a} tzdata{a} tzdata-legacy{a} uglifyjs{a} xml-core{a} The following packages are RECOMMENDED but will NOT be installed: curl git javascript-common krb5-locales libarchive-cpio-perl libjson-xs-perl libltdl-dev libmail-sendmail-perl libpaper-utils lynx nodejs-doc pandoc python3-blosc python3-bottleneck python3-distributed python3-dropbox python3-fusepy python3-gi python3-libarchive-c python3-lxml python3-matplotlib python3-numexpr python3-odf python3-openpyxl python3-paramiko python3-pil python3-pygit2 python3-rfc3987 python3-scipy python3-tables wget 0 packages upgraded, 232 newly installed, 0 to remove and 0 not upgraded. Need to get 136 MB of archives. After unpacking 855 MB will be used. Writing extended state information... Get: 1 http://deb.debian.org/debian trixie/main amd64 fonts-lato all 2.015-1 [2780 kB] Get: 2 http://deb.debian.org/debian trixie/main amd64 libpython3.12-minimal amd64 3.12.6-1 [814 kB] Get: 3 http://deb.debian.org/debian trixie/main amd64 libexpat1 amd64 2.6.3-1 [105 kB] Get: 4 http://deb.debian.org/debian trixie/main amd64 python3.12-minimal amd64 3.12.6-1 [2168 kB] Get: 5 http://deb.debian.org/debian trixie/main amd64 python3-minimal amd64 3.12.6-1 [26.7 kB] Get: 6 http://deb.debian.org/debian trixie/main amd64 media-types all 10.1.0 [26.9 kB] Get: 7 http://deb.debian.org/debian trixie/main amd64 netbase all 6.4 [12.8 kB] Get: 8 http://deb.debian.org/debian trixie/main amd64 tzdata all 2024a-4 [255 kB] Get: 9 http://deb.debian.org/debian trixie/main amd64 libkrb5support0 amd64 1.21.3-3 [32.5 kB] Get: 10 http://deb.debian.org/debian trixie/main amd64 libcom-err2 amd64 1.47.1-1 [22.9 kB] Get: 11 http://deb.debian.org/debian trixie/main amd64 libk5crypto3 amd64 1.21.3-3 [79.9 kB] Get: 12 http://deb.debian.org/debian trixie/main amd64 libkeyutils1 amd64 1.6.3-3 [8952 B] Get: 13 http://deb.debian.org/debian trixie/main amd64 libkrb5-3 amd64 1.21.3-3 [324 kB] Get: 14 http://deb.debian.org/debian trixie/main amd64 libgssapi-krb5-2 amd64 1.21.3-3 [136 kB] Get: 15 http://deb.debian.org/debian trixie/main amd64 libtirpc-common all 1.3.4+ds-1.3 [10.9 kB] Get: 16 http://deb.debian.org/debian trixie/main amd64 libtirpc3t64 amd64 1.3.4+ds-1.3 [82.7 kB] Get: 17 http://deb.debian.org/debian trixie/main amd64 libnsl2 amd64 1.3.0-3+b2 [40.3 kB] Get: 18 http://deb.debian.org/debian trixie/main amd64 readline-common all 8.2-5 [69.3 kB] Get: 19 http://deb.debian.org/debian trixie/main amd64 libreadline8t64 amd64 8.2-5 [169 kB] Get: 20 http://deb.debian.org/debian trixie/main amd64 libpython3.12-stdlib amd64 3.12.6-1 [1963 kB] Get: 21 http://deb.debian.org/debian trixie/main amd64 python3.12 amd64 3.12.6-1 [669 kB] Get: 22 http://deb.debian.org/debian trixie/main amd64 libpython3-stdlib amd64 3.12.6-1 [9692 B] Get: 23 http://deb.debian.org/debian trixie/main amd64 python3 amd64 3.12.6-1 [27.8 kB] Get: 24 http://deb.debian.org/debian trixie/main amd64 sgml-base all 1.31 [15.4 kB] Get: 25 http://deb.debian.org/debian trixie/main amd64 sensible-utils all 0.0.24 [24.8 kB] Get: 26 http://deb.debian.org/debian trixie/main amd64 openssl amd64 3.3.2-1 [1381 kB] Get: 27 http://deb.debian.org/debian trixie/main amd64 ca-certificates all 20240203 [158 kB] Get: 28 http://deb.debian.org/debian trixie/main amd64 libmagic-mgc amd64 1:5.45-3 [314 kB] Get: 29 http://deb.debian.org/debian trixie/main amd64 libmagic1t64 amd64 1:5.45-3 [105 kB] Get: 30 http://deb.debian.org/debian trixie/main amd64 file amd64 1:5.45-3 [42.9 kB] Get: 31 http://deb.debian.org/debian trixie/main amd64 gettext-base amd64 0.22.5-2 [200 kB] Get: 32 http://deb.debian.org/debian trixie/main amd64 libuchardet0 amd64 0.0.8-1+b1 [68.8 kB] Get: 33 http://deb.debian.org/debian trixie/main amd64 groff-base amd64 1.23.0-5 [1181 kB] Get: 34 http://deb.debian.org/debian trixie/main amd64 bsdextrautils amd64 2.40.2-9 [97.2 kB] Get: 35 http://deb.debian.org/debian trixie/main amd64 libpipeline1 amd64 1.5.8-1 [42.0 kB] Get: 36 http://deb.debian.org/debian trixie/main amd64 man-db amd64 2.13.0-1 [1420 kB] Get: 37 http://deb.debian.org/debian trixie/main amd64 m4 amd64 1.4.19-4 [287 kB] Get: 38 http://deb.debian.org/debian trixie/main amd64 autoconf all 2.72-3 [493 kB] Get: 39 http://deb.debian.org/debian trixie/main amd64 autotools-dev all 20220109.1 [51.6 kB] Get: 40 http://deb.debian.org/debian trixie/main amd64 automake all 1:1.16.5-1.3 [823 kB] Get: 41 http://deb.debian.org/debian trixie/main amd64 autopoint all 0.22.5-2 [723 kB] Get: 42 http://deb.debian.org/debian trixie/main amd64 libdebhelper-perl all 13.20 [89.7 kB] Get: 43 http://deb.debian.org/debian trixie/main amd64 libtool all 2.4.7-7 [517 kB] Get: 44 http://deb.debian.org/debian trixie/main amd64 dh-autoreconf all 20 [17.1 kB] Get: 45 http://deb.debian.org/debian trixie/main amd64 libarchive-zip-perl all 1.68-1 [104 kB] Get: 46 http://deb.debian.org/debian trixie/main amd64 libfile-stripnondeterminism-perl all 1.14.0-1 [19.5 kB] Get: 47 http://deb.debian.org/debian trixie/main amd64 dh-strip-nondeterminism all 1.14.0-1 [8448 B] Get: 48 http://deb.debian.org/debian trixie/main amd64 libelf1t64 amd64 0.191-2 [188 kB] Get: 49 http://deb.debian.org/debian trixie/main amd64 dwz amd64 0.15-1+b1 [110 kB] Get: 50 http://deb.debian.org/debian trixie/main amd64 libicu72 amd64 72.1-5 [9396 kB] Get: 51 http://deb.debian.org/debian trixie/main amd64 libxml2 amd64 2.12.7+dfsg+really2.9.14-0.1 [699 kB] Get: 52 http://deb.debian.org/debian trixie/main amd64 gettext amd64 0.22.5-2 [1601 kB] Get: 53 http://deb.debian.org/debian trixie/main amd64 intltool-debian all 0.35.0+20060710.6 [22.9 kB] Get: 54 http://deb.debian.org/debian trixie/main amd64 po-debconf all 1.0.21+nmu1 [248 kB] Get: 55 http://deb.debian.org/debian trixie/main amd64 debhelper all 13.20 [915 kB] Get: 56 http://deb.debian.org/debian trixie/main amd64 python3-autocommand all 2.2.2-3 [13.6 kB] Get: 57 http://deb.debian.org/debian trixie/main amd64 python3-more-itertools all 10.5.0-1 [63.8 kB] Get: 58 http://deb.debian.org/debian trixie/main amd64 python3-typing-extensions all 4.12.2-2 [73.0 kB] Get: 59 http://deb.debian.org/debian trixie/main amd64 python3-typeguard all 4.3.0-1 [36.5 kB] Get: 60 http://deb.debian.org/debian trixie/main amd64 python3-inflect all 7.3.1-2 [32.4 kB] Get: 61 http://deb.debian.org/debian trixie/main amd64 python3-jaraco.context all 6.0.0-1 [7984 B] Get: 62 http://deb.debian.org/debian trixie/main amd64 python3-jaraco.functools all 4.1.0-1 [12.0 kB] Get: 63 http://deb.debian.org/debian trixie/main amd64 python3-pkg-resources all 74.1.2-2 [213 kB] Get: 64 http://deb.debian.org/debian trixie/main amd64 python3-zipp all 3.20.2-1 [10.3 kB] Get: 65 http://deb.debian.org/debian trixie/main amd64 python3-setuptools all 74.1.2-2 [736 kB] Get: 66 http://deb.debian.org/debian trixie/main amd64 dh-python all 6.20240824 [109 kB] Get: 67 http://deb.debian.org/debian trixie/main amd64 xml-core all 0.19 [20.1 kB] Get: 68 http://deb.debian.org/debian trixie/main amd64 docutils-common all 0.21.2+dfsg-2 [128 kB] Get: 69 http://deb.debian.org/debian trixie/main amd64 fonts-font-awesome all 5.0.10+really4.7.0~dfsg-4.1 [517 kB] Get: 70 http://deb.debian.org/debian trixie/main amd64 fonts-glyphicons-halflings all 1.009~3.4.1+dfsg-3 [162 kB] Get: 71 http://deb.debian.org/debian trixie/main amd64 fonts-mathjax all 2.7.9+dfsg-1 [2210 kB] Get: 72 http://deb.debian.org/debian trixie/main amd64 libargon2-1 amd64 0~20190702+dfsg-4+b1 [21.1 kB] Get: 73 http://deb.debian.org/debian trixie/main amd64 libblas3 amd64 3.12.0-3 [148 kB] Get: 74 http://deb.debian.org/debian trixie/main amd64 libbrotli1 amd64 1.1.0-2+b4 [300 kB] Get: 75 http://deb.debian.org/debian trixie/main amd64 libcares2 amd64 1.33.1-2 [80.3 kB] Get: 76 http://deb.debian.org/debian trixie/main amd64 libgfortran5 amd64 14.2.0-6 [836 kB] Get: 77 http://deb.debian.org/debian trixie/main amd64 libjs-bootstrap all 3.4.1+dfsg-3 [172 kB] Get: 78 http://deb.debian.org/debian trixie/main amd64 libjs-jquery all 3.6.1+dfsg+~3.5.14-1 [326 kB] Get: 79 http://deb.debian.org/debian trixie/main amd64 libjs-mathjax all 2.7.9+dfsg-1 [5667 kB] Get: 80 http://deb.debian.org/debian trixie/main amd64 libjs-sortable-tablesort all 3.1.0+ds-2 [13.7 kB] Get: 81 http://deb.debian.org/debian trixie/main amd64 libjs-underscore all 1.13.4~dfsg+~1.11.4-3 [116 kB] Get: 82 http://deb.debian.org/debian trixie/main amd64 libjs-sphinxdoc all 7.4.7-3 [158 kB] Get: 83 http://deb.debian.org/debian trixie/main amd64 libjs-sprintf-js all 1.1.2+ds1+~1.1.2-1 [12.8 kB] Get: 84 http://deb.debian.org/debian trixie/main amd64 libjson-perl all 4.10000-1 [87.5 kB] Get: 85 http://deb.debian.org/debian trixie/main amd64 liblapack3 amd64 3.12.0-3 [2407 kB] Get: 86 http://deb.debian.org/debian trixie/main amd64 libnghttp2-14 amd64 1.63.0-1 [74.8 kB] Get: 87 http://deb.debian.org/debian trixie/main amd64 libuv1t64 amd64 1.48.0-6 [148 kB] Get: 88 http://deb.debian.org/debian trixie/main amd64 node-xtend all 4.0.2-3 [3932 B] Get: 89 http://deb.debian.org/debian trixie/main amd64 nodejs amd64 20.17.0+dfsg-2 [347 kB] Get: 90 http://deb.debian.org/debian trixie/main amd64 node-acorn all 8.8.1+ds+~cs25.17.7-2 [128 kB] Get: 91 http://deb.debian.org/debian trixie/main amd64 node-cjs-module-lexer all 1.2.3+dfsg-1 [30.6 kB] Get: 92 http://deb.debian.org/debian trixie/main amd64 node-balanced-match all 2.0.0-1 [4872 B] Get: 93 http://deb.debian.org/debian trixie/main amd64 node-brace-expansion all 2.0.1+~1.1.0-1 [7912 B] Get: 94 http://deb.debian.org/debian trixie/main amd64 node-minimatch all 9.0.3-5 [76.7 kB] Get: 95 http://deb.debian.org/debian trixie/main amd64 node-undici all 5.28.4+dfsg1+~cs23.12.11-2 [314 kB] Get: 96 http://deb.debian.org/debian trixie/main amd64 libnode115 amd64 20.17.0+dfsg-2 [12.1 MB] Get: 97 http://deb.debian.org/debian trixie/main amd64 libnorm1t64 amd64 1.5.9+dfsg-3.1 [221 kB] Get: 98 http://deb.debian.org/debian trixie/main amd64 libpgm-5.3-0t64 amd64 5.3.128~dfsg-2.1 [161 kB] Get: 99 http://deb.debian.org/debian trixie/main amd64 libsodium23 amd64 1.0.18-1+b1 [163 kB] Get: 100 http://deb.debian.org/debian trixie/main amd64 libyaml-0-2 amd64 0.2.5-1+b1 [52.6 kB] Get: 101 http://deb.debian.org/debian trixie/main amd64 libzmq5 amd64 4.3.5-1+b2 [281 kB] Get: 102 http://deb.debian.org/debian trixie/main amd64 node-sprintf-js all 1.1.2+ds1+~1.1.2-1 [4024 B] Get: 103 http://deb.debian.org/debian trixie/main amd64 node-argparse all 2.0.1-2 [34.6 kB] Get: 104 http://deb.debian.org/debian trixie/main amd64 node-esprima all 4.0.1+ds+~4.0.3-2 [69.3 kB] Get: 105 http://deb.debian.org/debian trixie/main amd64 node-js-yaml all 4.1.0+dfsg+~4.0.5-7 [66.6 kB] Get: 106 http://deb.debian.org/debian trixie/main amd64 node-uglify-js all 3.17.4-2 [203 kB] Get: 107 http://deb.debian.org/debian trixie/main amd64 python3-packaging all 24.1-1 [45.8 kB] Get: 108 http://deb.debian.org/debian trixie/main amd64 python3-pyproject-hooks all 1.1.0-2 [11.3 kB] Get: 109 http://deb.debian.org/debian trixie/main amd64 python3-toml all 0.10.2-1 [16.2 kB] Get: 110 http://deb.debian.org/debian trixie/main amd64 python3-wheel all 0.44.0-2 [53.4 kB] Get: 111 http://deb.debian.org/debian trixie/main amd64 python3-build all 1.2.2-1 [36.0 kB] Get: 112 http://deb.debian.org/debian trixie/main amd64 python3-installer all 0.7.0+dfsg1-3 [18.6 kB] Get: 113 http://deb.debian.org/debian trixie/main amd64 pybuild-plugin-pyproject all 6.20240824 [11.2 kB] Get: 114 http://deb.debian.org/debian trixie/main amd64 python-babel-localedata all 2.14.0-1 [5701 kB] Get: 115 http://deb.debian.org/debian trixie/main amd64 python-dask-doc all 2024.5.2+dfsg-1 [11.2 MB] Get: 116 http://deb.debian.org/debian trixie/main amd64 sphinx-rtd-theme-common all 2.0.0+dfsg-2 [1021 kB] Get: 117 http://deb.debian.org/debian trixie/main amd64 python-fsspec-doc all 2024.9.0-1 [344 kB] Get: 118 http://deb.debian.org/debian trixie/main amd64 python-numpy-doc all 1:1.26.4+ds-11 [9384 kB] Get: 119 http://deb.debian.org/debian trixie/main amd64 python-pandas-doc all 2.2.3+dfsg-4 [9100 kB] Get: 120 http://deb.debian.org/debian trixie/main amd64 python-tinycss2-common all 1.3.0-1 [31.9 kB] Get: 121 http://deb.debian.org/debian trixie/main amd64 python3-alabaster all 0.7.16-0.1 [27.9 kB] Get: 122 http://deb.debian.org/debian trixie/main amd64 python3-all amd64 3.12.6-1 [1040 B] Get: 123 http://deb.debian.org/debian trixie/main amd64 python3-idna all 3.8-2 [41.6 kB] Get: 124 http://deb.debian.org/debian trixie/main amd64 python3-sniffio all 1.3.1-1 [7120 B] Get: 125 http://deb.debian.org/debian trixie/main amd64 python3-anyio all 4.4.0-1 [57.3 kB] Get: 126 http://deb.debian.org/debian trixie/main amd64 python3-cffi-backend amd64 1.17.1-1 [87.1 kB] Get: 127 http://deb.debian.org/debian trixie/main amd64 python3-argon2 amd64 21.1.0-2 [22.8 kB] Get: 128 http://deb.debian.org/debian trixie/main amd64 python3-dateutil all 2.9.0-3 [79.3 kB] Get: 129 http://deb.debian.org/debian trixie/main amd64 python3-typeshed all 0.0~git20231111.6764465-3 [1258 kB] Get: 130 http://deb.debian.org/debian trixie/main amd64 python3-arrow all 1.3.0-1.1 [64.7 kB] Get: 131 http://deb.debian.org/debian trixie/main amd64 python3-async-generator all 1.10-4 [17.4 kB] Get: 132 http://deb.debian.org/debian trixie/main amd64 python3-attr all 23.2.0-2 [65.5 kB] Get: 133 http://deb.debian.org/debian trixie/main amd64 python3-tz all 2024.1-2 [30.9 kB] Get: 134 http://deb.debian.org/debian trixie/main amd64 python3-babel all 2.14.0-1 [111 kB] Get: 135 http://deb.debian.org/debian trixie/main amd64 python3-bcrypt amd64 4.2.0-2 [210 kB] Get: 136 http://deb.debian.org/debian trixie/main amd64 python3-webencodings all 0.5.1-5 [11.1 kB] Get: 137 http://deb.debian.org/debian trixie/main amd64 python3-html5lib all 1.2-1 [92.2 kB] Get: 138 http://deb.debian.org/debian trixie/main amd64 python3-bleach all 6.1.0-2 [50.7 kB] Get: 139 http://deb.debian.org/debian trixie/main amd64 python3-soupsieve all 2.6-1 [38.3 kB] Get: 140 http://deb.debian.org/debian trixie/main amd64 python3-bs4 all 4.12.3-3 [133 kB] Get: 141 http://deb.debian.org/debian trixie/main amd64 python3-certifi all 2024.8.30+dfsg-1 [9576 B] Get: 142 http://deb.debian.org/debian trixie/main amd64 python3-chardet all 5.2.0+dfsg-1 [107 kB] Get: 143 http://deb.debian.org/debian trixie/main amd64 python3-charset-normalizer amd64 3.3.2-4 [117 kB] Get: 144 http://deb.debian.org/debian trixie/main amd64 python3-colorama all 0.4.6-4 [36.2 kB] Get: 145 http://deb.debian.org/debian trixie/main amd64 python3-click all 8.1.7-2 [94.3 kB] Get: 146 http://deb.debian.org/debian trixie/main amd64 python3-cloudpickle all 3.0.0-2 [25.4 kB] Get: 147 http://deb.debian.org/debian trixie/main amd64 python3-cryptography amd64 43.0.0-1 [935 kB] Get: 148 http://deb.debian.org/debian trixie/main amd64 python3-fsspec all 2024.9.0-1 [206 kB] Get: 149 http://deb.debian.org/debian trixie/main amd64 python3-toolz all 1.0.0-1 [44.1 kB] Get: 150 http://deb.debian.org/debian trixie/main amd64 python3-locket all 1.0.0-2 [5820 B] Get: 151 http://deb.debian.org/debian trixie/main amd64 python3-partd all 1.4.2-1 [15.4 kB] Get: 152 http://deb.debian.org/debian trixie/main amd64 python3-yaml amd64 6.0.2-1 [134 kB] Get: 153 http://deb.debian.org/debian trixie/main amd64 python3-dask all 2024.5.2+dfsg-1 [946 kB] Get: 154 http://deb.debian.org/debian trixie/main amd64 python3-rpds-py amd64 0.12.0-3+b1 [225 kB] Get: 155 http://deb.debian.org/debian trixie/main amd64 python3-referencing all 0.35.1-1 [23.5 kB] Get: 156 http://deb.debian.org/debian trixie/main amd64 python3-jsonschema-specifications all 2023.12.1-2 [9020 B] Get: 157 http://deb.debian.org/debian trixie/main amd64 python3-jsonschema all 4.19.2-5 [71.4 kB] Get: 158 http://deb.debian.org/debian trixie/main amd64 python3-roman all 4.2-1 [10.4 kB] Get: 159 http://deb.debian.org/debian trixie/main amd64 python3-docutils all 0.21.2+dfsg-2 [403 kB] Get: 160 http://deb.debian.org/debian trixie/main amd64 python3-defusedxml all 0.7.1-2 [43.3 kB] Get: 161 http://deb.debian.org/debian trixie/main amd64 sphinx-common all 7.4.7-3 [731 kB] Get: 162 http://deb.debian.org/debian trixie/main amd64 python3-imagesize all 1.4.1-1 [6688 B] Get: 163 http://deb.debian.org/debian trixie/main amd64 python3-markupsafe amd64 2.1.5-1+b1 [13.6 kB] Get: 164 http://deb.debian.org/debian trixie/main amd64 python3-jinja2 all 3.1.3-1 [119 kB] Get: 165 http://deb.debian.org/debian trixie/main amd64 python3-pygments all 2.18.0+dfsg-1 [836 kB] Get: 166 http://deb.debian.org/debian trixie/main amd64 python3-urllib3 all 2.0.7-2 [111 kB] Get: 167 http://deb.debian.org/debian trixie/main amd64 python3-requests all 2.32.3+dfsg-1 [71.9 kB] Get: 168 http://deb.debian.org/debian trixie/main amd64 python3-snowballstemmer all 2.2.0-4 [58.0 kB] Get: 169 http://deb.debian.org/debian trixie/main amd64 python3-sphinx all 7.4.7-3 [588 kB] Get: 170 http://deb.debian.org/debian trixie/main amd64 python3-pydata-sphinx-theme all 0.15.2.is.0.7.2-1 [1076 kB] Get: 171 http://deb.debian.org/debian trixie/main amd64 python3-sphinx-book-theme all 0.1.7-3 [47.3 kB] Get: 172 http://deb.debian.org/debian trixie/main amd64 python3-dask-sphinx-theme all 3.0.5-2 [15.1 kB] Get: 173 http://deb.debian.org/debian trixie/main amd64 python3-decorator all 5.1.1-5 [15.1 kB] Get: 174 http://deb.debian.org/debian trixie/main amd64 python3.12-doc all 3.12.6-1 [13.1 MB] Get: 175 http://deb.debian.org/debian trixie/main amd64 python3-doc all 3.12.6-1 [9884 B] Get: 176 http://deb.debian.org/debian trixie/main amd64 python3-fastjsonschema all 2.20.0-1 [21.8 kB] Get: 177 http://deb.debian.org/debian trixie/main amd64 python3-fqdn all 1.5.1-2 [7592 B] Get: 178 http://deb.debian.org/debian trixie/main amd64 python3-iniconfig all 1.1.1-2 [6396 B] Get: 179 http://deb.debian.org/debian trixie/main amd64 python3-isoduration all 20.11.0+git20211126.ae0bd61-2 [12.3 kB] Get: 180 http://deb.debian.org/debian trixie/main amd64 python3-json-pointer all 2.4-2 [14.6 kB] Get: 181 http://deb.debian.org/debian trixie/main amd64 python3-platformdirs all 4.3.6-1 [16.6 kB] Get: 182 http://deb.debian.org/debian trixie/main amd64 python3-traitlets all 5.14.3+really5.14.3-1 [70.9 kB] Get: 183 http://deb.debian.org/debian trixie/main amd64 python3-jupyter-core all 5.7.2-4 [35.1 kB] Get: 184 http://deb.debian.org/debian trixie/main amd64 python3-psutil amd64 5.9.8-2 [225 kB] Get: 185 http://deb.debian.org/debian trixie/main amd64 python3-tornado amd64 6.4.1-3 [349 kB] Get: 186 http://deb.debian.org/debian trixie/main amd64 python3-py all 1.11.0-2 [88.7 kB] Get: 187 http://deb.debian.org/debian trixie/main amd64 python3-zmq amd64 24.0.1-5+b2 [269 kB] Get: 188 http://deb.debian.org/debian trixie/main amd64 python3-jupyter-client all 8.6.2-3 [91.5 kB] Get: 189 http://deb.debian.org/debian trixie/main amd64 python3-pythonjsonlogger all 3.0.0-2 [13.5 kB] Get: 190 http://deb.debian.org/debian trixie/main amd64 python3-rfc3339-validator all 0.1.4-2 [4060 B] Get: 191 http://deb.debian.org/debian trixie/main amd64 python3-rfc3986-validator all 0.1.1-2 [4884 B] Get: 192 http://deb.debian.org/debian trixie/main amd64 python3-mdurl all 0.1.2-1 [9444 B] Get: 193 http://deb.debian.org/debian trixie/main amd64 python3-markdown-it all 3.0.0-2 [62.5 kB] Get: 194 http://deb.debian.org/debian trixie/main amd64 python3-rich all 13.7.1-1 [201 kB] Get: 195 http://deb.debian.org/debian trixie/main amd64 python3-uritemplate all 4.1.1-3 [10.6 kB] Get: 196 http://deb.debian.org/debian trixie/main amd64 python3-webcolors all 1.13-1 [15.3 kB] Get: 197 http://deb.debian.org/debian trixie/main amd64 python3-jupyter-events all 0.10.0-2 [18.1 kB] Get: 198 http://deb.debian.org/debian trixie/main amd64 python3-ptyprocess all 0.7.0-5 [14.7 kB] Get: 199 http://deb.debian.org/debian trixie/main amd64 python3-terminado all 0.18.1-1 [17.3 kB] Get: 200 http://deb.debian.org/debian trixie/main amd64 python3-jupyter-server-terminals all 0.5.3-2 [13.9 kB] Get: 201 http://deb.debian.org/debian trixie/main amd64 python3-jupyterlab-pygments all 0.2.2-3 [6096 B] Get: 202 http://deb.debian.org/debian trixie/main amd64 python3-mistune all 3.0.2-2 [33.2 kB] Get: 203 http://deb.debian.org/debian trixie/main amd64 python3-nbformat all 5.9.1-1 [45.1 kB] Get: 204 http://deb.debian.org/debian trixie/main amd64 python3-nbclient all 0.8.0-1 [64.0 kB] Get: 205 http://deb.debian.org/debian trixie/main amd64 python3-pandocfilters all 1.5.1-1 [23.7 kB] Get: 206 http://deb.debian.org/debian trixie/main amd64 python3-tinycss2 all 1.3.0-1 [22.1 kB] Get: 207 http://deb.debian.org/debian trixie/main amd64 python3-nbconvert all 7.16.4-2 [158 kB] Get: 208 http://deb.debian.org/debian trixie/main amd64 python3-overrides all 7.7.0-2 [14.2 kB] Get: 209 http://deb.debian.org/debian trixie/main amd64 python3-prometheus-client all 0.19.0+ds1-1 [35.0 kB] Get: 210 http://deb.debian.org/debian trixie/main amd64 python3-send2trash all 1.8.2-1 [16.5 kB] Get: 211 http://deb.debian.org/debian trixie/main amd64 python3-websocket all 1.8.0-2 [45.5 kB] Get: 212 http://deb.debian.org/debian trixie/main amd64 python3-jupyter-server all 2.14.2-4 [212 kB] Get: 213 http://deb.debian.org/debian trixie/main amd64 python3-msgpack amd64 1.0.3-3+b2 [80.4 kB] Get: 214 http://deb.debian.org/debian trixie/main amd64 python3-numpy amd64 1:1.26.4+ds-11 [4236 kB] Get: 215 http://deb.debian.org/debian trixie/main amd64 python3-tabulate all 0.9.0-1 [45.9 kB] Get: 216 http://deb.debian.org/debian trixie/main amd64 python3-numpydoc all 1.6.0-2 [49.6 kB] Get: 217 http://deb.debian.org/debian trixie/main amd64 python3-pandas-lib amd64 2.2.3+dfsg-4 [4388 kB] Get: 218 http://deb.debian.org/debian trixie/main amd64 python3-pandas all 2.2.3+dfsg-4 [3096 kB] Get: 219 http://deb.debian.org/debian trixie/main amd64 python3-pluggy all 1.5.0-1 [26.9 kB] Get: 220 http://deb.debian.org/debian trixie/main amd64 python3-pytest all 8.3.3-1 [249 kB] Get: 221 http://deb.debian.org/debian trixie/main amd64 python3-pytest-asyncio all 0.20.3-1.3 [13.5 kB] Get: 222 http://deb.debian.org/debian trixie/main amd64 python3-pytest-rerunfailures all 12.0-1 [16.6 kB] Get: 223 http://deb.debian.org/debian trixie/main amd64 python3-pytest-timeout all 2.3.1-1 [21.9 kB] Get: 224 http://deb.debian.org/debian trixie/main amd64 python3-sortedcontainers all 2.4.0-2 [31.9 kB] Get: 225 http://deb.debian.org/debian trixie/main amd64 python3-sortedcollections all 2.1.0-2 [9620 B] Get: 226 http://deb.debian.org/debian trixie/main amd64 python3-sphinx-click all 5.1.0-1 [10.0 kB] Get: 227 http://deb.debian.org/debian trixie/main amd64 python3-sphinx-design all 0.6.1-1 [883 kB] Get: 228 http://deb.debian.org/debian trixie/main amd64 python3-tblib all 3.0.0-5 [13.7 kB] Get: 229 http://deb.debian.org/debian trixie/main amd64 python3-versioneer all 0.29-1 [43.0 kB] Get: 230 http://deb.debian.org/debian trixie/main amd64 python3-zict all 3.0.0-2 [29.7 kB] Get: 231 http://deb.debian.org/debian trixie/main amd64 tzdata-legacy all 2024a-4 [170 kB] Get: 232 http://deb.debian.org/debian trixie/main amd64 uglifyjs all 3.17.4-2 [12.1 kB] Fetched 136 MB in 6s (22.1 MB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package fonts-lato. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 19799 files and directories currently installed.) Preparing to unpack .../fonts-lato_2.015-1_all.deb ... Unpacking fonts-lato (2.015-1) ... Selecting previously unselected package libpython3.12-minimal:amd64. Preparing to unpack .../libpython3.12-minimal_3.12.6-1_amd64.deb ... Unpacking libpython3.12-minimal:amd64 (3.12.6-1) ... Selecting previously unselected package libexpat1:amd64. Preparing to unpack .../libexpat1_2.6.3-1_amd64.deb ... Unpacking libexpat1:amd64 (2.6.3-1) ... Selecting previously unselected package python3.12-minimal. Preparing to unpack .../python3.12-minimal_3.12.6-1_amd64.deb ... Unpacking python3.12-minimal (3.12.6-1) ... Setting up libpython3.12-minimal:amd64 (3.12.6-1) ... Setting up libexpat1:amd64 (2.6.3-1) ... Setting up python3.12-minimal (3.12.6-1) ... Selecting previously unselected package python3-minimal. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 20145 files and directories currently installed.) Preparing to unpack .../00-python3-minimal_3.12.6-1_amd64.deb ... Unpacking python3-minimal (3.12.6-1) ... Selecting previously unselected package media-types. Preparing to unpack .../01-media-types_10.1.0_all.deb ... Unpacking media-types (10.1.0) ... Selecting previously unselected package netbase. Preparing to unpack .../02-netbase_6.4_all.deb ... Unpacking netbase (6.4) ... Selecting previously unselected package tzdata. Preparing to unpack .../03-tzdata_2024a-4_all.deb ... Unpacking tzdata (2024a-4) ... Selecting previously unselected package libkrb5support0:amd64. Preparing to unpack .../04-libkrb5support0_1.21.3-3_amd64.deb ... Unpacking libkrb5support0:amd64 (1.21.3-3) ... Selecting previously unselected package libcom-err2:amd64. Preparing to unpack .../05-libcom-err2_1.47.1-1_amd64.deb ... Unpacking libcom-err2:amd64 (1.47.1-1) ... Selecting previously unselected package libk5crypto3:amd64. Preparing to unpack .../06-libk5crypto3_1.21.3-3_amd64.deb ... Unpacking libk5crypto3:amd64 (1.21.3-3) ... Selecting previously unselected package libkeyutils1:amd64. Preparing to unpack .../07-libkeyutils1_1.6.3-3_amd64.deb ... Unpacking libkeyutils1:amd64 (1.6.3-3) ... Selecting previously unselected package libkrb5-3:amd64. Preparing to unpack .../08-libkrb5-3_1.21.3-3_amd64.deb ... Unpacking libkrb5-3:amd64 (1.21.3-3) ... Selecting previously unselected package libgssapi-krb5-2:amd64. Preparing to unpack .../09-libgssapi-krb5-2_1.21.3-3_amd64.deb ... Unpacking libgssapi-krb5-2:amd64 (1.21.3-3) ... Selecting previously unselected package libtirpc-common. Preparing to unpack .../10-libtirpc-common_1.3.4+ds-1.3_all.deb ... Unpacking libtirpc-common (1.3.4+ds-1.3) ... Selecting previously unselected package libtirpc3t64:amd64. Preparing to unpack .../11-libtirpc3t64_1.3.4+ds-1.3_amd64.deb ... Adding 'diversion of /lib/x86_64-linux-gnu/libtirpc.so.3 to /lib/x86_64-linux-gnu/libtirpc.so.3.usr-is-merged by libtirpc3t64' Adding 'diversion of /lib/x86_64-linux-gnu/libtirpc.so.3.0.0 to /lib/x86_64-linux-gnu/libtirpc.so.3.0.0.usr-is-merged by libtirpc3t64' Unpacking libtirpc3t64:amd64 (1.3.4+ds-1.3) ... Selecting previously unselected package libnsl2:amd64. Preparing to unpack .../12-libnsl2_1.3.0-3+b2_amd64.deb ... Unpacking libnsl2:amd64 (1.3.0-3+b2) ... Selecting previously unselected package readline-common. Preparing to unpack .../13-readline-common_8.2-5_all.deb ... Unpacking readline-common (8.2-5) ... Selecting previously unselected package libreadline8t64:amd64. Preparing to unpack .../14-libreadline8t64_8.2-5_amd64.deb ... Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8 to /lib/x86_64-linux-gnu/libhistory.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libhistory.so.8.2 to /lib/x86_64-linux-gnu/libhistory.so.8.2.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8 to /lib/x86_64-linux-gnu/libreadline.so.8.usr-is-merged by libreadline8t64' Adding 'diversion of /lib/x86_64-linux-gnu/libreadline.so.8.2 to /lib/x86_64-linux-gnu/libreadline.so.8.2.usr-is-merged by libreadline8t64' Unpacking libreadline8t64:amd64 (8.2-5) ... Selecting previously unselected package libpython3.12-stdlib:amd64. Preparing to unpack .../15-libpython3.12-stdlib_3.12.6-1_amd64.deb ... Unpacking libpython3.12-stdlib:amd64 (3.12.6-1) ... Selecting previously unselected package python3.12. Preparing to unpack .../16-python3.12_3.12.6-1_amd64.deb ... Unpacking python3.12 (3.12.6-1) ... Selecting previously unselected package libpython3-stdlib:amd64. Preparing to unpack .../17-libpython3-stdlib_3.12.6-1_amd64.deb ... Unpacking libpython3-stdlib:amd64 (3.12.6-1) ... Setting up python3-minimal (3.12.6-1) ... Selecting previously unselected package python3. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 21217 files and directories currently installed.) Preparing to unpack .../000-python3_3.12.6-1_amd64.deb ... Unpacking python3 (3.12.6-1) ... Selecting previously unselected package sgml-base. Preparing to unpack .../001-sgml-base_1.31_all.deb ... Unpacking sgml-base (1.31) ... Selecting previously unselected package sensible-utils. Preparing to unpack .../002-sensible-utils_0.0.24_all.deb ... Unpacking sensible-utils (0.0.24) ... Selecting previously unselected package openssl. Preparing to unpack .../003-openssl_3.3.2-1_amd64.deb ... Unpacking openssl (3.3.2-1) ... Selecting previously unselected package ca-certificates. Preparing to unpack .../004-ca-certificates_20240203_all.deb ... Unpacking ca-certificates (20240203) ... Selecting previously unselected package libmagic-mgc. Preparing to unpack .../005-libmagic-mgc_1%3a5.45-3_amd64.deb ... Unpacking libmagic-mgc (1:5.45-3) ... Selecting previously unselected package libmagic1t64:amd64. Preparing to unpack .../006-libmagic1t64_1%3a5.45-3_amd64.deb ... Unpacking libmagic1t64:amd64 (1:5.45-3) ... Selecting previously unselected package file. Preparing to unpack .../007-file_1%3a5.45-3_amd64.deb ... Unpacking file (1:5.45-3) ... Selecting previously unselected package gettext-base. Preparing to unpack .../008-gettext-base_0.22.5-2_amd64.deb ... Unpacking gettext-base (0.22.5-2) ... Selecting previously unselected package libuchardet0:amd64. Preparing to unpack .../009-libuchardet0_0.0.8-1+b1_amd64.deb ... Unpacking libuchardet0:amd64 (0.0.8-1+b1) ... Selecting previously unselected package groff-base. Preparing to unpack .../010-groff-base_1.23.0-5_amd64.deb ... Unpacking groff-base (1.23.0-5) ... Selecting previously unselected package bsdextrautils. Preparing to unpack .../011-bsdextrautils_2.40.2-9_amd64.deb ... Unpacking bsdextrautils (2.40.2-9) ... Selecting previously unselected package libpipeline1:amd64. Preparing to unpack .../012-libpipeline1_1.5.8-1_amd64.deb ... Unpacking libpipeline1:amd64 (1.5.8-1) ... Selecting previously unselected package man-db. Preparing to unpack .../013-man-db_2.13.0-1_amd64.deb ... Unpacking man-db (2.13.0-1) ... Selecting previously unselected package m4. Preparing to unpack .../014-m4_1.4.19-4_amd64.deb ... Unpacking m4 (1.4.19-4) ... Selecting previously unselected package autoconf. Preparing to unpack .../015-autoconf_2.72-3_all.deb ... Unpacking autoconf (2.72-3) ... Selecting previously unselected package autotools-dev. Preparing to unpack .../016-autotools-dev_20220109.1_all.deb ... Unpacking autotools-dev (20220109.1) ... Selecting previously unselected package automake. Preparing to unpack .../017-automake_1%3a1.16.5-1.3_all.deb ... Unpacking automake (1:1.16.5-1.3) ... Selecting previously unselected package autopoint. Preparing to unpack .../018-autopoint_0.22.5-2_all.deb ... Unpacking autopoint (0.22.5-2) ... Selecting previously unselected package libdebhelper-perl. Preparing to unpack .../019-libdebhelper-perl_13.20_all.deb ... Unpacking libdebhelper-perl (13.20) ... Selecting previously unselected package libtool. Preparing to unpack .../020-libtool_2.4.7-7_all.deb ... Unpacking libtool (2.4.7-7) ... Selecting previously unselected package dh-autoreconf. Preparing to unpack .../021-dh-autoreconf_20_all.deb ... Unpacking dh-autoreconf (20) ... Selecting previously unselected package libarchive-zip-perl. Preparing to unpack .../022-libarchive-zip-perl_1.68-1_all.deb ... Unpacking libarchive-zip-perl (1.68-1) ... Selecting previously unselected package libfile-stripnondeterminism-perl. Preparing to unpack .../023-libfile-stripnondeterminism-perl_1.14.0-1_all.deb ... Unpacking libfile-stripnondeterminism-perl (1.14.0-1) ... Selecting previously unselected package dh-strip-nondeterminism. Preparing to unpack .../024-dh-strip-nondeterminism_1.14.0-1_all.deb ... Unpacking dh-strip-nondeterminism (1.14.0-1) ... Selecting previously unselected package libelf1t64:amd64. Preparing to unpack .../025-libelf1t64_0.191-2_amd64.deb ... Unpacking libelf1t64:amd64 (0.191-2) ... Selecting previously unselected package dwz. Preparing to unpack .../026-dwz_0.15-1+b1_amd64.deb ... Unpacking dwz (0.15-1+b1) ... Selecting previously unselected package libicu72:amd64. Preparing to unpack .../027-libicu72_72.1-5_amd64.deb ... Unpacking libicu72:amd64 (72.1-5) ... Selecting previously unselected package libxml2:amd64. Preparing to unpack .../028-libxml2_2.12.7+dfsg+really2.9.14-0.1_amd64.deb ... Unpacking libxml2:amd64 (2.12.7+dfsg+really2.9.14-0.1) ... Selecting previously unselected package gettext. Preparing to unpack .../029-gettext_0.22.5-2_amd64.deb ... Unpacking gettext (0.22.5-2) ... Selecting previously unselected package intltool-debian. Preparing to unpack .../030-intltool-debian_0.35.0+20060710.6_all.deb ... Unpacking intltool-debian (0.35.0+20060710.6) ... Selecting previously unselected package po-debconf. Preparing to unpack .../031-po-debconf_1.0.21+nmu1_all.deb ... Unpacking po-debconf (1.0.21+nmu1) ... Selecting previously unselected package debhelper. Preparing to unpack .../032-debhelper_13.20_all.deb ... Unpacking debhelper (13.20) ... Selecting previously unselected package python3-autocommand. Preparing to unpack .../033-python3-autocommand_2.2.2-3_all.deb ... Unpacking python3-autocommand (2.2.2-3) ... Selecting previously unselected package python3-more-itertools. Preparing to unpack .../034-python3-more-itertools_10.5.0-1_all.deb ... Unpacking python3-more-itertools (10.5.0-1) ... Selecting previously unselected package python3-typing-extensions. Preparing to unpack .../035-python3-typing-extensions_4.12.2-2_all.deb ... Unpacking python3-typing-extensions (4.12.2-2) ... Selecting previously unselected package python3-typeguard. Preparing to unpack .../036-python3-typeguard_4.3.0-1_all.deb ... Unpacking python3-typeguard (4.3.0-1) ... Selecting previously unselected package python3-inflect. Preparing to unpack .../037-python3-inflect_7.3.1-2_all.deb ... Unpacking python3-inflect (7.3.1-2) ... Selecting previously unselected package python3-jaraco.context. Preparing to unpack .../038-python3-jaraco.context_6.0.0-1_all.deb ... Unpacking python3-jaraco.context (6.0.0-1) ... Selecting previously unselected package python3-jaraco.functools. Preparing to unpack .../039-python3-jaraco.functools_4.1.0-1_all.deb ... Unpacking python3-jaraco.functools (4.1.0-1) ... Selecting previously unselected package python3-pkg-resources. Preparing to unpack .../040-python3-pkg-resources_74.1.2-2_all.deb ... Unpacking python3-pkg-resources (74.1.2-2) ... Selecting previously unselected package python3-zipp. Preparing to unpack .../041-python3-zipp_3.20.2-1_all.deb ... Unpacking python3-zipp (3.20.2-1) ... Selecting previously unselected package python3-setuptools. Preparing to unpack .../042-python3-setuptools_74.1.2-2_all.deb ... Unpacking python3-setuptools (74.1.2-2) ... Selecting previously unselected package dh-python. Preparing to unpack .../043-dh-python_6.20240824_all.deb ... Unpacking dh-python (6.20240824) ... Selecting previously unselected package xml-core. Preparing to unpack .../044-xml-core_0.19_all.deb ... Unpacking xml-core (0.19) ... Selecting previously unselected package docutils-common. Preparing to unpack .../045-docutils-common_0.21.2+dfsg-2_all.deb ... Unpacking docutils-common (0.21.2+dfsg-2) ... Selecting previously unselected package fonts-font-awesome. Preparing to unpack .../046-fonts-font-awesome_5.0.10+really4.7.0~dfsg-4.1_all.deb ... Unpacking fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Selecting previously unselected package fonts-glyphicons-halflings. Preparing to unpack .../047-fonts-glyphicons-halflings_1.009~3.4.1+dfsg-3_all.deb ... Unpacking fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Selecting previously unselected package fonts-mathjax. Preparing to unpack .../048-fonts-mathjax_2.7.9+dfsg-1_all.deb ... Unpacking fonts-mathjax (2.7.9+dfsg-1) ... Selecting previously unselected package libargon2-1:amd64. Preparing to unpack .../049-libargon2-1_0~20190702+dfsg-4+b1_amd64.deb ... Unpacking libargon2-1:amd64 (0~20190702+dfsg-4+b1) ... Selecting previously unselected package libblas3:amd64. Preparing to unpack .../050-libblas3_3.12.0-3_amd64.deb ... Unpacking libblas3:amd64 (3.12.0-3) ... Selecting previously unselected package libbrotli1:amd64. Preparing to unpack .../051-libbrotli1_1.1.0-2+b4_amd64.deb ... Unpacking libbrotli1:amd64 (1.1.0-2+b4) ... Selecting previously unselected package libcares2:amd64. Preparing to unpack .../052-libcares2_1.33.1-2_amd64.deb ... Unpacking libcares2:amd64 (1.33.1-2) ... Selecting previously unselected package libgfortran5:amd64. Preparing to unpack .../053-libgfortran5_14.2.0-6_amd64.deb ... Unpacking libgfortran5:amd64 (14.2.0-6) ... Selecting previously unselected package libjs-bootstrap. Preparing to unpack .../054-libjs-bootstrap_3.4.1+dfsg-3_all.deb ... Unpacking libjs-bootstrap (3.4.1+dfsg-3) ... Selecting previously unselected package libjs-jquery. Preparing to unpack .../055-libjs-jquery_3.6.1+dfsg+~3.5.14-1_all.deb ... Unpacking libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Selecting previously unselected package libjs-mathjax. Preparing to unpack .../056-libjs-mathjax_2.7.9+dfsg-1_all.deb ... Unpacking libjs-mathjax (2.7.9+dfsg-1) ... Selecting previously unselected package libjs-sortable-tablesort. Preparing to unpack .../057-libjs-sortable-tablesort_3.1.0+ds-2_all.deb ... Unpacking libjs-sortable-tablesort (3.1.0+ds-2) ... Selecting previously unselected package libjs-underscore. Preparing to unpack .../058-libjs-underscore_1.13.4~dfsg+~1.11.4-3_all.deb ... Unpacking libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Selecting previously unselected package libjs-sphinxdoc. Preparing to unpack .../059-libjs-sphinxdoc_7.4.7-3_all.deb ... Unpacking libjs-sphinxdoc (7.4.7-3) ... Selecting previously unselected package libjs-sprintf-js. Preparing to unpack .../060-libjs-sprintf-js_1.1.2+ds1+~1.1.2-1_all.deb ... Unpacking libjs-sprintf-js (1.1.2+ds1+~1.1.2-1) ... Selecting previously unselected package libjson-perl. Preparing to unpack .../061-libjson-perl_4.10000-1_all.deb ... Unpacking libjson-perl (4.10000-1) ... Selecting previously unselected package liblapack3:amd64. Preparing to unpack .../062-liblapack3_3.12.0-3_amd64.deb ... Unpacking liblapack3:amd64 (3.12.0-3) ... Selecting previously unselected package libnghttp2-14:amd64. Preparing to unpack .../063-libnghttp2-14_1.63.0-1_amd64.deb ... Unpacking libnghttp2-14:amd64 (1.63.0-1) ... Selecting previously unselected package libuv1t64:amd64. Preparing to unpack .../064-libuv1t64_1.48.0-6_amd64.deb ... Unpacking libuv1t64:amd64 (1.48.0-6) ... Selecting previously unselected package node-xtend. Preparing to unpack .../065-node-xtend_4.0.2-3_all.deb ... Unpacking node-xtend (4.0.2-3) ... Selecting previously unselected package nodejs. Preparing to unpack .../066-nodejs_20.17.0+dfsg-2_amd64.deb ... Unpacking nodejs (20.17.0+dfsg-2) ... Selecting previously unselected package node-acorn. Preparing to unpack .../067-node-acorn_8.8.1+ds+~cs25.17.7-2_all.deb ... Unpacking node-acorn (8.8.1+ds+~cs25.17.7-2) ... Selecting previously unselected package node-cjs-module-lexer. Preparing to unpack .../068-node-cjs-module-lexer_1.2.3+dfsg-1_all.deb ... Unpacking node-cjs-module-lexer (1.2.3+dfsg-1) ... Selecting previously unselected package node-balanced-match. Preparing to unpack .../069-node-balanced-match_2.0.0-1_all.deb ... Unpacking node-balanced-match (2.0.0-1) ... Selecting previously unselected package node-brace-expansion. Preparing to unpack .../070-node-brace-expansion_2.0.1+~1.1.0-1_all.deb ... Unpacking node-brace-expansion (2.0.1+~1.1.0-1) ... Selecting previously unselected package node-minimatch. Preparing to unpack .../071-node-minimatch_9.0.3-5_all.deb ... Unpacking node-minimatch (9.0.3-5) ... Selecting previously unselected package node-undici. Preparing to unpack .../072-node-undici_5.28.4+dfsg1+~cs23.12.11-2_all.deb ... Unpacking node-undici (5.28.4+dfsg1+~cs23.12.11-2) ... Selecting previously unselected package libnode115:amd64. Preparing to unpack .../073-libnode115_20.17.0+dfsg-2_amd64.deb ... Unpacking libnode115:amd64 (20.17.0+dfsg-2) ... Selecting previously unselected package libnorm1t64:amd64. Preparing to unpack .../074-libnorm1t64_1.5.9+dfsg-3.1_amd64.deb ... Unpacking libnorm1t64:amd64 (1.5.9+dfsg-3.1) ... Selecting previously unselected package libpgm-5.3-0t64:amd64. Preparing to unpack .../075-libpgm-5.3-0t64_5.3.128~dfsg-2.1_amd64.deb ... Unpacking libpgm-5.3-0t64:amd64 (5.3.128~dfsg-2.1) ... Selecting previously unselected package libsodium23:amd64. Preparing to unpack .../076-libsodium23_1.0.18-1+b1_amd64.deb ... Unpacking libsodium23:amd64 (1.0.18-1+b1) ... Selecting previously unselected package libyaml-0-2:amd64. Preparing to unpack .../077-libyaml-0-2_0.2.5-1+b1_amd64.deb ... Unpacking libyaml-0-2:amd64 (0.2.5-1+b1) ... Selecting previously unselected package libzmq5:amd64. Preparing to unpack .../078-libzmq5_4.3.5-1+b2_amd64.deb ... Unpacking libzmq5:amd64 (4.3.5-1+b2) ... Selecting previously unselected package node-sprintf-js. Preparing to unpack .../079-node-sprintf-js_1.1.2+ds1+~1.1.2-1_all.deb ... Unpacking node-sprintf-js (1.1.2+ds1+~1.1.2-1) ... Selecting previously unselected package node-argparse. Preparing to unpack .../080-node-argparse_2.0.1-2_all.deb ... Unpacking node-argparse (2.0.1-2) ... Selecting previously unselected package node-esprima. Preparing to unpack .../081-node-esprima_4.0.1+ds+~4.0.3-2_all.deb ... Unpacking node-esprima (4.0.1+ds+~4.0.3-2) ... Selecting previously unselected package node-js-yaml. Preparing to unpack .../082-node-js-yaml_4.1.0+dfsg+~4.0.5-7_all.deb ... Unpacking node-js-yaml (4.1.0+dfsg+~4.0.5-7) ... Selecting previously unselected package node-uglify-js. Preparing to unpack .../083-node-uglify-js_3.17.4-2_all.deb ... Unpacking node-uglify-js (3.17.4-2) ... Selecting previously unselected package python3-packaging. Preparing to unpack .../084-python3-packaging_24.1-1_all.deb ... Unpacking python3-packaging (24.1-1) ... Selecting previously unselected package python3-pyproject-hooks. Preparing to unpack .../085-python3-pyproject-hooks_1.1.0-2_all.deb ... Unpacking python3-pyproject-hooks (1.1.0-2) ... Selecting previously unselected package python3-toml. Preparing to unpack .../086-python3-toml_0.10.2-1_all.deb ... Unpacking python3-toml (0.10.2-1) ... Selecting previously unselected package python3-wheel. Preparing to unpack .../087-python3-wheel_0.44.0-2_all.deb ... Unpacking python3-wheel (0.44.0-2) ... Selecting previously unselected package python3-build. Preparing to unpack .../088-python3-build_1.2.2-1_all.deb ... Unpacking python3-build (1.2.2-1) ... Selecting previously unselected package python3-installer. Preparing to unpack .../089-python3-installer_0.7.0+dfsg1-3_all.deb ... Unpacking python3-installer (0.7.0+dfsg1-3) ... Selecting previously unselected package pybuild-plugin-pyproject. Preparing to unpack .../090-pybuild-plugin-pyproject_6.20240824_all.deb ... Unpacking pybuild-plugin-pyproject (6.20240824) ... Selecting previously unselected package python-babel-localedata. Preparing to unpack .../091-python-babel-localedata_2.14.0-1_all.deb ... Unpacking python-babel-localedata (2.14.0-1) ... Selecting previously unselected package python-dask-doc. Preparing to unpack .../092-python-dask-doc_2024.5.2+dfsg-1_all.deb ... Unpacking python-dask-doc (2024.5.2+dfsg-1) ... Selecting previously unselected package sphinx-rtd-theme-common. Preparing to unpack .../093-sphinx-rtd-theme-common_2.0.0+dfsg-2_all.deb ... Unpacking sphinx-rtd-theme-common (2.0.0+dfsg-2) ... Selecting previously unselected package python-fsspec-doc. Preparing to unpack .../094-python-fsspec-doc_2024.9.0-1_all.deb ... Unpacking python-fsspec-doc (2024.9.0-1) ... Selecting previously unselected package python-numpy-doc. Preparing to unpack .../095-python-numpy-doc_1%3a1.26.4+ds-11_all.deb ... Unpacking python-numpy-doc (1:1.26.4+ds-11) ... Selecting previously unselected package python-pandas-doc. Preparing to unpack .../096-python-pandas-doc_2.2.3+dfsg-4_all.deb ... Unpacking python-pandas-doc (2.2.3+dfsg-4) ... Selecting previously unselected package python-tinycss2-common. Preparing to unpack .../097-python-tinycss2-common_1.3.0-1_all.deb ... Unpacking python-tinycss2-common (1.3.0-1) ... Selecting previously unselected package python3-alabaster. Preparing to unpack .../098-python3-alabaster_0.7.16-0.1_all.deb ... Unpacking python3-alabaster (0.7.16-0.1) ... Selecting previously unselected package python3-all. Preparing to unpack .../099-python3-all_3.12.6-1_amd64.deb ... Unpacking python3-all (3.12.6-1) ... Selecting previously unselected package python3-idna. Preparing to unpack .../100-python3-idna_3.8-2_all.deb ... Unpacking python3-idna (3.8-2) ... Selecting previously unselected package python3-sniffio. Preparing to unpack .../101-python3-sniffio_1.3.1-1_all.deb ... Unpacking python3-sniffio (1.3.1-1) ... Selecting previously unselected package python3-anyio. Preparing to unpack .../102-python3-anyio_4.4.0-1_all.deb ... Unpacking python3-anyio (4.4.0-1) ... Selecting previously unselected package python3-cffi-backend:amd64. Preparing to unpack .../103-python3-cffi-backend_1.17.1-1_amd64.deb ... Unpacking python3-cffi-backend:amd64 (1.17.1-1) ... Selecting previously unselected package python3-argon2. Preparing to unpack .../104-python3-argon2_21.1.0-2_amd64.deb ... Unpacking python3-argon2 (21.1.0-2) ... Selecting previously unselected package python3-dateutil. Preparing to unpack .../105-python3-dateutil_2.9.0-3_all.deb ... Unpacking python3-dateutil (2.9.0-3) ... Selecting previously unselected package python3-typeshed. Preparing to unpack .../106-python3-typeshed_0.0~git20231111.6764465-3_all.deb ... Unpacking python3-typeshed (0.0~git20231111.6764465-3) ... Selecting previously unselected package python3-arrow. Preparing to unpack .../107-python3-arrow_1.3.0-1.1_all.deb ... Unpacking python3-arrow (1.3.0-1.1) ... Selecting previously unselected package python3-async-generator. Preparing to unpack .../108-python3-async-generator_1.10-4_all.deb ... Unpacking python3-async-generator (1.10-4) ... Selecting previously unselected package python3-attr. Preparing to unpack .../109-python3-attr_23.2.0-2_all.deb ... Unpacking python3-attr (23.2.0-2) ... Selecting previously unselected package python3-tz. Preparing to unpack .../110-python3-tz_2024.1-2_all.deb ... Unpacking python3-tz (2024.1-2) ... Selecting previously unselected package python3-babel. Preparing to unpack .../111-python3-babel_2.14.0-1_all.deb ... Unpacking python3-babel (2.14.0-1) ... Selecting previously unselected package python3-bcrypt. Preparing to unpack .../112-python3-bcrypt_4.2.0-2_amd64.deb ... Unpacking python3-bcrypt (4.2.0-2) ... Selecting previously unselected package python3-webencodings. Preparing to unpack .../113-python3-webencodings_0.5.1-5_all.deb ... Unpacking python3-webencodings (0.5.1-5) ... Selecting previously unselected package python3-html5lib. Preparing to unpack .../114-python3-html5lib_1.2-1_all.deb ... Unpacking python3-html5lib (1.2-1) ... Selecting previously unselected package python3-bleach. Preparing to unpack .../115-python3-bleach_6.1.0-2_all.deb ... Unpacking python3-bleach (6.1.0-2) ... Selecting previously unselected package python3-soupsieve. Preparing to unpack .../116-python3-soupsieve_2.6-1_all.deb ... Unpacking python3-soupsieve (2.6-1) ... Selecting previously unselected package python3-bs4. Preparing to unpack .../117-python3-bs4_4.12.3-3_all.deb ... Unpacking python3-bs4 (4.12.3-3) ... Selecting previously unselected package python3-certifi. Preparing to unpack .../118-python3-certifi_2024.8.30+dfsg-1_all.deb ... Unpacking python3-certifi (2024.8.30+dfsg-1) ... Selecting previously unselected package python3-chardet. Preparing to unpack .../119-python3-chardet_5.2.0+dfsg-1_all.deb ... Unpacking python3-chardet (5.2.0+dfsg-1) ... Selecting previously unselected package python3-charset-normalizer. Preparing to unpack .../120-python3-charset-normalizer_3.3.2-4_amd64.deb ... Unpacking python3-charset-normalizer (3.3.2-4) ... Selecting previously unselected package python3-colorama. Preparing to unpack .../121-python3-colorama_0.4.6-4_all.deb ... Unpacking python3-colorama (0.4.6-4) ... Selecting previously unselected package python3-click. Preparing to unpack .../122-python3-click_8.1.7-2_all.deb ... Unpacking python3-click (8.1.7-2) ... Selecting previously unselected package python3-cloudpickle. Preparing to unpack .../123-python3-cloudpickle_3.0.0-2_all.deb ... Unpacking python3-cloudpickle (3.0.0-2) ... Selecting previously unselected package python3-cryptography. Preparing to unpack .../124-python3-cryptography_43.0.0-1_amd64.deb ... Unpacking python3-cryptography (43.0.0-1) ... Selecting previously unselected package python3-fsspec. Preparing to unpack .../125-python3-fsspec_2024.9.0-1_all.deb ... Unpacking python3-fsspec (2024.9.0-1) ... Selecting previously unselected package python3-toolz. Preparing to unpack .../126-python3-toolz_1.0.0-1_all.deb ... Unpacking python3-toolz (1.0.0-1) ... Selecting previously unselected package python3-locket. Preparing to unpack .../127-python3-locket_1.0.0-2_all.deb ... Unpacking python3-locket (1.0.0-2) ... Selecting previously unselected package python3-partd. Preparing to unpack .../128-python3-partd_1.4.2-1_all.deb ... Unpacking python3-partd (1.4.2-1) ... Selecting previously unselected package python3-yaml. Preparing to unpack .../129-python3-yaml_6.0.2-1_amd64.deb ... Unpacking python3-yaml (6.0.2-1) ... Selecting previously unselected package python3-dask. Preparing to unpack .../130-python3-dask_2024.5.2+dfsg-1_all.deb ... Unpacking python3-dask (2024.5.2+dfsg-1) ... Selecting previously unselected package python3-rpds-py. Preparing to unpack .../131-python3-rpds-py_0.12.0-3+b1_amd64.deb ... Unpacking python3-rpds-py (0.12.0-3+b1) ... Selecting previously unselected package python3-referencing. Preparing to unpack .../132-python3-referencing_0.35.1-1_all.deb ... Unpacking python3-referencing (0.35.1-1) ... Selecting previously unselected package python3-jsonschema-specifications. Preparing to unpack .../133-python3-jsonschema-specifications_2023.12.1-2_all.deb ... Unpacking python3-jsonschema-specifications (2023.12.1-2) ... Selecting previously unselected package python3-jsonschema. Preparing to unpack .../134-python3-jsonschema_4.19.2-5_all.deb ... Unpacking python3-jsonschema (4.19.2-5) ... Selecting previously unselected package python3-roman. Preparing to unpack .../135-python3-roman_4.2-1_all.deb ... Unpacking python3-roman (4.2-1) ... Selecting previously unselected package python3-docutils. Preparing to unpack .../136-python3-docutils_0.21.2+dfsg-2_all.deb ... Unpacking python3-docutils (0.21.2+dfsg-2) ... Selecting previously unselected package python3-defusedxml. Preparing to unpack .../137-python3-defusedxml_0.7.1-2_all.deb ... Unpacking python3-defusedxml (0.7.1-2) ... Selecting previously unselected package sphinx-common. Preparing to unpack .../138-sphinx-common_7.4.7-3_all.deb ... Unpacking sphinx-common (7.4.7-3) ... Selecting previously unselected package python3-imagesize. Preparing to unpack .../139-python3-imagesize_1.4.1-1_all.deb ... Unpacking python3-imagesize (1.4.1-1) ... Selecting previously unselected package python3-markupsafe. Preparing to unpack .../140-python3-markupsafe_2.1.5-1+b1_amd64.deb ... Unpacking python3-markupsafe (2.1.5-1+b1) ... Selecting previously unselected package python3-jinja2. Preparing to unpack .../141-python3-jinja2_3.1.3-1_all.deb ... Unpacking python3-jinja2 (3.1.3-1) ... Selecting previously unselected package python3-pygments. Preparing to unpack .../142-python3-pygments_2.18.0+dfsg-1_all.deb ... Unpacking python3-pygments (2.18.0+dfsg-1) ... Selecting previously unselected package python3-urllib3. Preparing to unpack .../143-python3-urllib3_2.0.7-2_all.deb ... Unpacking python3-urllib3 (2.0.7-2) ... Selecting previously unselected package python3-requests. Preparing to unpack .../144-python3-requests_2.32.3+dfsg-1_all.deb ... Unpacking python3-requests (2.32.3+dfsg-1) ... Selecting previously unselected package python3-snowballstemmer. Preparing to unpack .../145-python3-snowballstemmer_2.2.0-4_all.deb ... Unpacking python3-snowballstemmer (2.2.0-4) ... Selecting previously unselected package python3-sphinx. Preparing to unpack .../146-python3-sphinx_7.4.7-3_all.deb ... Unpacking python3-sphinx (7.4.7-3) ... Selecting previously unselected package python3-pydata-sphinx-theme. Preparing to unpack .../147-python3-pydata-sphinx-theme_0.15.2.is.0.7.2-1_all.deb ... Unpacking python3-pydata-sphinx-theme (0.15.2.is.0.7.2-1) ... Selecting previously unselected package python3-sphinx-book-theme. Preparing to unpack .../148-python3-sphinx-book-theme_0.1.7-3_all.deb ... Unpacking python3-sphinx-book-theme (0.1.7-3) ... Selecting previously unselected package python3-dask-sphinx-theme. Preparing to unpack .../149-python3-dask-sphinx-theme_3.0.5-2_all.deb ... Unpacking python3-dask-sphinx-theme (3.0.5-2) ... Selecting previously unselected package python3-decorator. Preparing to unpack .../150-python3-decorator_5.1.1-5_all.deb ... Unpacking python3-decorator (5.1.1-5) ... Selecting previously unselected package python3.12-doc. Preparing to unpack .../151-python3.12-doc_3.12.6-1_all.deb ... Unpacking python3.12-doc (3.12.6-1) ... Selecting previously unselected package python3-doc. Preparing to unpack .../152-python3-doc_3.12.6-1_all.deb ... Unpacking python3-doc (3.12.6-1) ... Selecting previously unselected package python3-fastjsonschema. Preparing to unpack .../153-python3-fastjsonschema_2.20.0-1_all.deb ... Unpacking python3-fastjsonschema (2.20.0-1) ... Selecting previously unselected package python3-fqdn. Preparing to unpack .../154-python3-fqdn_1.5.1-2_all.deb ... Unpacking python3-fqdn (1.5.1-2) ... Selecting previously unselected package python3-iniconfig. Preparing to unpack .../155-python3-iniconfig_1.1.1-2_all.deb ... Unpacking python3-iniconfig (1.1.1-2) ... Selecting previously unselected package python3-isoduration. Preparing to unpack .../156-python3-isoduration_20.11.0+git20211126.ae0bd61-2_all.deb ... Unpacking python3-isoduration (20.11.0+git20211126.ae0bd61-2) ... Selecting previously unselected package python3-json-pointer. Preparing to unpack .../157-python3-json-pointer_2.4-2_all.deb ... Unpacking python3-json-pointer (2.4-2) ... Selecting previously unselected package python3-platformdirs. Preparing to unpack .../158-python3-platformdirs_4.3.6-1_all.deb ... Unpacking python3-platformdirs (4.3.6-1) ... Selecting previously unselected package python3-traitlets. Preparing to unpack .../159-python3-traitlets_5.14.3+really5.14.3-1_all.deb ... Unpacking python3-traitlets (5.14.3+really5.14.3-1) ... Selecting previously unselected package python3-jupyter-core. Preparing to unpack .../160-python3-jupyter-core_5.7.2-4_all.deb ... Unpacking python3-jupyter-core (5.7.2-4) ... Selecting previously unselected package python3-psutil. Preparing to unpack .../161-python3-psutil_5.9.8-2_amd64.deb ... Unpacking python3-psutil (5.9.8-2) ... Selecting previously unselected package python3-tornado. Preparing to unpack .../162-python3-tornado_6.4.1-3_amd64.deb ... Unpacking python3-tornado (6.4.1-3) ... Selecting previously unselected package python3-py. Preparing to unpack .../163-python3-py_1.11.0-2_all.deb ... Unpacking python3-py (1.11.0-2) ... Selecting previously unselected package python3-zmq. Preparing to unpack .../164-python3-zmq_24.0.1-5+b2_amd64.deb ... Unpacking python3-zmq (24.0.1-5+b2) ... Selecting previously unselected package python3-jupyter-client. Preparing to unpack .../165-python3-jupyter-client_8.6.2-3_all.deb ... Unpacking python3-jupyter-client (8.6.2-3) ... Selecting previously unselected package python3-pythonjsonlogger. Preparing to unpack .../166-python3-pythonjsonlogger_3.0.0-2_all.deb ... Unpacking python3-pythonjsonlogger (3.0.0-2) ... Selecting previously unselected package python3-rfc3339-validator. Preparing to unpack .../167-python3-rfc3339-validator_0.1.4-2_all.deb ... Unpacking python3-rfc3339-validator (0.1.4-2) ... Selecting previously unselected package python3-rfc3986-validator. Preparing to unpack .../168-python3-rfc3986-validator_0.1.1-2_all.deb ... Unpacking python3-rfc3986-validator (0.1.1-2) ... Selecting previously unselected package python3-mdurl. Preparing to unpack .../169-python3-mdurl_0.1.2-1_all.deb ... Unpacking python3-mdurl (0.1.2-1) ... Selecting previously unselected package python3-markdown-it. Preparing to unpack .../170-python3-markdown-it_3.0.0-2_all.deb ... Unpacking python3-markdown-it (3.0.0-2) ... Selecting previously unselected package python3-rich. Preparing to unpack .../171-python3-rich_13.7.1-1_all.deb ... Unpacking python3-rich (13.7.1-1) ... Selecting previously unselected package python3-uritemplate. Preparing to unpack .../172-python3-uritemplate_4.1.1-3_all.deb ... Unpacking python3-uritemplate (4.1.1-3) ... Selecting previously unselected package python3-webcolors. Preparing to unpack .../173-python3-webcolors_1.13-1_all.deb ... Unpacking python3-webcolors (1.13-1) ... Selecting previously unselected package python3-jupyter-events. Preparing to unpack .../174-python3-jupyter-events_0.10.0-2_all.deb ... Unpacking python3-jupyter-events (0.10.0-2) ... Selecting previously unselected package python3-ptyprocess. Preparing to unpack .../175-python3-ptyprocess_0.7.0-5_all.deb ... Unpacking python3-ptyprocess (0.7.0-5) ... Selecting previously unselected package python3-terminado. Preparing to unpack .../176-python3-terminado_0.18.1-1_all.deb ... Unpacking python3-terminado (0.18.1-1) ... Selecting previously unselected package python3-jupyter-server-terminals. Preparing to unpack .../177-python3-jupyter-server-terminals_0.5.3-2_all.deb ... Unpacking python3-jupyter-server-terminals (0.5.3-2) ... Selecting previously unselected package python3-jupyterlab-pygments. Preparing to unpack .../178-python3-jupyterlab-pygments_0.2.2-3_all.deb ... Unpacking python3-jupyterlab-pygments (0.2.2-3) ... Selecting previously unselected package python3-mistune. Preparing to unpack .../179-python3-mistune_3.0.2-2_all.deb ... Unpacking python3-mistune (3.0.2-2) ... Selecting previously unselected package python3-nbformat. Preparing to unpack .../180-python3-nbformat_5.9.1-1_all.deb ... Unpacking python3-nbformat (5.9.1-1) ... Selecting previously unselected package python3-nbclient. Preparing to unpack .../181-python3-nbclient_0.8.0-1_all.deb ... Unpacking python3-nbclient (0.8.0-1) ... Selecting previously unselected package python3-pandocfilters. Preparing to unpack .../182-python3-pandocfilters_1.5.1-1_all.deb ... Unpacking python3-pandocfilters (1.5.1-1) ... Selecting previously unselected package python3-tinycss2. Preparing to unpack .../183-python3-tinycss2_1.3.0-1_all.deb ... Unpacking python3-tinycss2 (1.3.0-1) ... Selecting previously unselected package python3-nbconvert. Preparing to unpack .../184-python3-nbconvert_7.16.4-2_all.deb ... Unpacking python3-nbconvert (7.16.4-2) ... Selecting previously unselected package python3-overrides. Preparing to unpack .../185-python3-overrides_7.7.0-2_all.deb ... Unpacking python3-overrides (7.7.0-2) ... Selecting previously unselected package python3-prometheus-client. Preparing to unpack .../186-python3-prometheus-client_0.19.0+ds1-1_all.deb ... Unpacking python3-prometheus-client (0.19.0+ds1-1) ... Selecting previously unselected package python3-send2trash. Preparing to unpack .../187-python3-send2trash_1.8.2-1_all.deb ... Unpacking python3-send2trash (1.8.2-1) ... Selecting previously unselected package python3-websocket. Preparing to unpack .../188-python3-websocket_1.8.0-2_all.deb ... Unpacking python3-websocket (1.8.0-2) ... Selecting previously unselected package python3-jupyter-server. Preparing to unpack .../189-python3-jupyter-server_2.14.2-4_all.deb ... Unpacking python3-jupyter-server (2.14.2-4) ... Selecting previously unselected package python3-msgpack. Preparing to unpack .../190-python3-msgpack_1.0.3-3+b2_amd64.deb ... Unpacking python3-msgpack (1.0.3-3+b2) ... Selecting previously unselected package python3-numpy. Preparing to unpack .../191-python3-numpy_1%3a1.26.4+ds-11_amd64.deb ... Unpacking python3-numpy (1:1.26.4+ds-11) ... Selecting previously unselected package python3-tabulate. Preparing to unpack .../192-python3-tabulate_0.9.0-1_all.deb ... Unpacking python3-tabulate (0.9.0-1) ... Selecting previously unselected package python3-numpydoc. Preparing to unpack .../193-python3-numpydoc_1.6.0-2_all.deb ... Unpacking python3-numpydoc (1.6.0-2) ... Selecting previously unselected package python3-pandas-lib:amd64. Preparing to unpack .../194-python3-pandas-lib_2.2.3+dfsg-4_amd64.deb ... Unpacking python3-pandas-lib:amd64 (2.2.3+dfsg-4) ... Selecting previously unselected package python3-pandas. Preparing to unpack .../195-python3-pandas_2.2.3+dfsg-4_all.deb ... Unpacking python3-pandas (2.2.3+dfsg-4) ... Selecting previously unselected package python3-pluggy. Preparing to unpack .../196-python3-pluggy_1.5.0-1_all.deb ... Unpacking python3-pluggy (1.5.0-1) ... Selecting previously unselected package python3-pytest. Preparing to unpack .../197-python3-pytest_8.3.3-1_all.deb ... Unpacking python3-pytest (8.3.3-1) ... Selecting previously unselected package python3-pytest-asyncio. Preparing to unpack .../198-python3-pytest-asyncio_0.20.3-1.3_all.deb ... Unpacking python3-pytest-asyncio (0.20.3-1.3) ... Selecting previously unselected package python3-pytest-rerunfailures. Preparing to unpack .../199-python3-pytest-rerunfailures_12.0-1_all.deb ... Unpacking python3-pytest-rerunfailures (12.0-1) ... Selecting previously unselected package python3-pytest-timeout. Preparing to unpack .../200-python3-pytest-timeout_2.3.1-1_all.deb ... Unpacking python3-pytest-timeout (2.3.1-1) ... Selecting previously unselected package python3-sortedcontainers. Preparing to unpack .../201-python3-sortedcontainers_2.4.0-2_all.deb ... Unpacking python3-sortedcontainers (2.4.0-2) ... Selecting previously unselected package python3-sortedcollections. Preparing to unpack .../202-python3-sortedcollections_2.1.0-2_all.deb ... Unpacking python3-sortedcollections (2.1.0-2) ... Selecting previously unselected package python3-sphinx-click. Preparing to unpack .../203-python3-sphinx-click_5.1.0-1_all.deb ... Unpacking python3-sphinx-click (5.1.0-1) ... Selecting previously unselected package python3-sphinx-design. Preparing to unpack .../204-python3-sphinx-design_0.6.1-1_all.deb ... Unpacking python3-sphinx-design (0.6.1-1) ... Selecting previously unselected package python3-tblib. Preparing to unpack .../205-python3-tblib_3.0.0-5_all.deb ... Unpacking python3-tblib (3.0.0-5) ... Selecting previously unselected package python3-versioneer. Preparing to unpack .../206-python3-versioneer_0.29-1_all.deb ... Unpacking python3-versioneer (0.29-1) ... Selecting previously unselected package python3-zict. Preparing to unpack .../207-python3-zict_3.0.0-2_all.deb ... Unpacking python3-zict (3.0.0-2) ... Selecting previously unselected package tzdata-legacy. Preparing to unpack .../208-tzdata-legacy_2024a-4_all.deb ... Unpacking tzdata-legacy (2024a-4) ... Selecting previously unselected package uglifyjs. Preparing to unpack .../209-uglifyjs_3.17.4-2_all.deb ... Unpacking uglifyjs (3.17.4-2) ... Setting up media-types (10.1.0) ... Setting up libpipeline1:amd64 (1.5.8-1) ... Setting up node-uglify-js (3.17.4-2) ... Setting up libnorm1t64:amd64 (1.5.9+dfsg-3.1) ... Setting up libkeyutils1:amd64 (1.6.3-3) ... Setting up fonts-lato (2.015-1) ... Setting up libjs-sprintf-js (1.1.2+ds1+~1.1.2-1) ... Setting up fonts-mathjax (2.7.9+dfsg-1) ... Setting up libsodium23:amd64 (1.0.18-1+b1) ... Setting up libicu72:amd64 (72.1-5) ... Setting up bsdextrautils (2.40.2-9) ... Setting up libjs-mathjax (2.7.9+dfsg-1) ... Setting up libmagic-mgc (1:5.45-3) ... Setting up libarchive-zip-perl (1.68-1) ... Setting up libyaml-0-2:amd64 (0.2.5-1+b1) ... Setting up fonts-glyphicons-halflings (1.009~3.4.1+dfsg-3) ... Setting up libtirpc-common (1.3.4+ds-1.3) ... Setting up libargon2-1:amd64 (0~20190702+dfsg-4+b1) ... Setting up libdebhelper-perl (13.20) ... Setting up libbrotli1:amd64 (1.1.0-2+b4) ... Setting up libuv1t64:amd64 (1.48.0-6) ... Setting up libmagic1t64:amd64 (1:5.45-3) ... Setting up libnghttp2-14:amd64 (1.63.0-1) ... Setting up gettext-base (0.22.5-2) ... Setting up m4 (1.4.19-4) ... Setting up node-cjs-module-lexer (1.2.3+dfsg-1) ... Setting up libcom-err2:amd64 (1.47.1-1) ... Setting up file (1:5.45-3) ... Setting up libelf1t64:amd64 (0.191-2) ... Setting up python-babel-localedata (2.14.0-1) ... Setting up libkrb5support0:amd64 (1.21.3-3) ... Setting up tzdata (2024a-4) ... Current default time zone: 'Etc/UTC' Local time is now: Mon Oct 21 16:19:20 UTC 2024. Universal Time is now: Mon Oct 21 16:19:20 UTC 2024. Run 'dpkg-reconfigure tzdata' if you wish to change it. Setting up libpgm-5.3-0t64:amd64 (5.3.128~dfsg-2.1) ... Setting up autotools-dev (20220109.1) ... Setting up libblas3:amd64 (3.12.0-3) ... update-alternatives: using /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in auto mode Setting up node-sprintf-js (1.1.2+ds1+~1.1.2-1) ... Setting up node-balanced-match (2.0.0-1) ... Setting up node-brace-expansion (2.0.1+~1.1.0-1) ... Setting up libcares2:amd64 (1.33.1-2) ... Setting up autopoint (0.22.5-2) ... Setting up libk5crypto3:amd64 (1.21.3-3) ... Setting up libgfortran5:amd64 (14.2.0-6) ... Setting up autoconf (2.72-3) ... Setting up node-undici (5.28.4+dfsg1+~cs23.12.11-2) ... Setting up dwz (0.15-1+b1) ... Setting up sensible-utils (0.0.24) ... Setting up libuchardet0:amd64 (0.0.8-1+b1) ... Setting up python-tinycss2-common (1.3.0-1) ... Setting up libjson-perl (4.10000-1) ... Setting up netbase (6.4) ... Setting up sgml-base (1.31) ... Setting up libkrb5-3:amd64 (1.21.3-3) ... Setting up node-minimatch (9.0.3-5) ... Setting up libjs-jquery (3.6.1+dfsg+~3.5.14-1) ... Setting up node-xtend (4.0.2-3) ... Setting up openssl (3.3.2-1) ... Setting up readline-common (8.2-5) ... Setting up libxml2:amd64 (2.12.7+dfsg+really2.9.14-0.1) ... Setting up fonts-font-awesome (5.0.10+really4.7.0~dfsg-4.1) ... Setting up sphinx-rtd-theme-common (2.0.0+dfsg-2) ... Setting up libjs-sortable-tablesort (3.1.0+ds-2) ... Setting up libjs-bootstrap (3.4.1+dfsg-3) ... Setting up libjs-underscore (1.13.4~dfsg+~1.11.4-3) ... Setting up automake (1:1.16.5-1.3) ... update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode Setting up libfile-stripnondeterminism-perl (1.14.0-1) ... Setting up liblapack3:amd64 (3.12.0-3) ... update-alternatives: using /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in auto mode Setting up gettext (0.22.5-2) ... Setting up libtool (2.4.7-7) ... Setting up tzdata-legacy (2024a-4) ... Setting up intltool-debian (0.35.0+20060710.6) ... Setting up dh-autoreconf (20) ... Setting up ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 146 added, 0 removed; done. Setting up libgssapi-krb5-2:amd64 (1.21.3-3) ... Setting up libjs-sphinxdoc (7.4.7-3) ... Setting up libreadline8t64:amd64 (8.2-5) ... Setting up dh-strip-nondeterminism (1.14.0-1) ... Setting up groff-base (1.23.0-5) ... Setting up xml-core (0.19) ... Setting up python3.12-doc (3.12.6-1) ... Setting up python3-doc (3.12.6-1) ... Setting up python-numpy-doc (1:1.26.4+ds-11) ... Setting up libzmq5:amd64 (4.3.5-1+b2) ... Setting up libtirpc3t64:amd64 (1.3.4+ds-1.3) ... Setting up python-pandas-doc (2.2.3+dfsg-4) ... Setting up po-debconf (1.0.21+nmu1) ... Setting up man-db (2.13.0-1) ... Not building database; man-db/auto-update is not 'true'. Setting up sphinx-common (7.4.7-3) ... Setting up python-fsspec-doc (2024.9.0-1) ... Setting up libnsl2:amd64 (1.3.0-3+b2) ... Setting up libpython3.12-stdlib:amd64 (3.12.6-1) ... Setting up python3.12 (3.12.6-1) ... Setting up debhelper (13.20) ... Setting up libpython3-stdlib:amd64 (3.12.6-1) ... Setting up python3 (3.12.6-1) ... Setting up python3-sortedcontainers (2.4.0-2) ... Setting up python3-pandocfilters (1.5.1-1) ... Setting up python3-zipp (3.20.2-1) ... Setting up python3-autocommand (2.2.2-3) ... Setting up python3-markupsafe (2.1.5-1+b1) ... Setting up python3-wheel (0.44.0-2) ... Setting up python3-webencodings (0.5.1-5) ... Setting up python3-platformdirs (4.3.6-1) ... Setting up python3-psutil (5.9.8-2) ... Setting up python3-tz (2024.1-2) ... Setting up python3-cloudpickle (3.0.0-2) ... Setting up python3-uritemplate (4.1.1-3) ... Setting up python3-roman (4.2-1) ... Setting up python3-decorator (5.1.1-5) ... Setting up python3-fqdn (1.5.1-2) ... Setting up python3-jinja2 (3.1.3-1) ... Setting up python3-webcolors (1.13-1) ... Setting up python3-packaging (24.1-1) ... Setting up python3-pyproject-hooks (1.1.0-2) ... Setting up python3-certifi (2024.8.30+dfsg-1) ... Setting up python3-typeshed (0.0~git20231111.6764465-3) ... Setting up python3-snowballstemmer (2.2.0-4) ... Setting up python3-mdurl (0.1.2-1) ... Setting up python3-rfc3986-validator (0.1.1-2) ... Setting up python3-idna (3.8-2) ... Setting up python3-typing-extensions (4.12.2-2) ... Setting up python3-html5lib (1.2-1) ... Setting up python3-toml (0.10.2-1) ... Setting up python3-installer (0.7.0+dfsg1-3) ... Setting up python3-urllib3 (2.0.7-2) ... Setting up python3-pluggy (1.5.0-1) ... Setting up python3-markdown-it (3.0.0-2) ... Setting up python3-rpds-py (0.12.0-3+b1) ... Setting up python3-fastjsonschema (2.20.0-1) ... Setting up python3-json-pointer (2.4-2) ... Setting up python3-traitlets (5.14.3+really5.14.3-1) ... Setting up python3-toolz (1.0.0-1) ... Setting up python3-dateutil (2.9.0-3) ... Setting up python3-msgpack (1.0.3-3+b2) ... Setting up python3-versioneer (0.29-1) ... Setting up python3-build (1.2.2-1) ... Setting up python3-zict (3.0.0-2) ... Setting up python3-locket (1.0.0-2) ... Setting up python3-mistune (3.0.2-2) ... Setting up python3-soupsieve (2.6-1) ... Setting up python3-websocket (1.8.0-2) ... Setting up python3-jupyter-core (5.7.2-4) ... Setting up python3-cffi-backend:amd64 (1.17.1-1) ... Setting up python3-imagesize (1.4.1-1) ... Setting up python3-ptyprocess (0.7.0-5) ... Setting up python3-partd (1.4.2-1) ... Setting up python3-more-itertools (10.5.0-1) ... Setting up python3-iniconfig (1.1.1-2) ... Setting up python3-sniffio (1.3.1-1) ... Setting up python3-sortedcollections (2.1.0-2) ... Setting up python3-attr (23.2.0-2) ... Setting up python3-tornado (6.4.1-3) ... Setting up python3-send2trash (1.8.2-1) ... Setting up python3-tinycss2 (1.3.0-1) ... Setting up python3-jaraco.functools (4.1.0-1) ... Setting up python3-async-generator (1.10-4) ... Setting up python3-arrow (1.3.0-1.1) ... Setting up python3-tblib (3.0.0-5) ... Setting up python3-jaraco.context (6.0.0-1) ... Setting up python3-anyio (4.4.0-1) ... Setting up python3-pythonjsonlogger (3.0.0-2) ... Setting up python3-colorama (0.4.6-4) ... Setting up python3-defusedxml (0.7.1-2) ... Setting up python3-charset-normalizer (3.3.2-4) ... Setting up python3-pytest (8.3.3-1) ... Setting up python3-alabaster (0.7.16-0.1) ... Setting up python3-overrides (7.7.0-2) ... Setting up python3-bcrypt (4.2.0-2) ... Setting up python3-fsspec (2024.9.0-1) ... Setting up python3-typeguard (4.3.0-1) ... Setting up python3-rfc3339-validator (0.1.4-2) ... Setting up python3-tabulate (0.9.0-1) ... Setting up python3-isoduration (20.11.0+git20211126.ae0bd61-2) ... Setting up python3-all (3.12.6-1) ... Setting up python3-yaml (6.0.2-1) ... Setting up python3-click (8.1.7-2) ... Setting up python3-bs4 (4.12.3-3) ... Setting up python3-bleach (6.1.0-2) ... Setting up python3-inflect (7.3.1-2) ... Setting up python3-prometheus-client (0.19.0+ds1-1) ... Setting up python3-cryptography (43.0.0-1) ... Setting up python3-pytest-asyncio (0.20.3-1.3) ... Setting up python3-terminado (0.18.1-1) ... Setting up python3-argon2 (21.1.0-2) ... Setting up python3-jupyter-server-terminals (0.5.3-2) ... Setting up python3-referencing (0.35.1-1) ... Setting up python3-pkg-resources (74.1.2-2) ... Setting up python3-pytest-rerunfailures (12.0-1) ... Setting up python3-setuptools (74.1.2-2) ... Setting up python3-pytest-timeout (2.3.1-1) ... Setting up python3-py (1.11.0-2) ... Setting up python3-babel (2.14.0-1) ... update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode Setting up python3-dask (2024.5.2+dfsg-1) ... Setting up python3-pygments (2.18.0+dfsg-1) ... Setting up python3-chardet (5.2.0+dfsg-1) ... Setting up python3-zmq (24.0.1-5+b2) ... Setting up python3-requests (2.32.3+dfsg-1) ... Setting up python3-numpy (1:1.26.4+ds-11) ... Setting up python3-jupyter-client (8.6.2-3) ... Setting up python3-jsonschema-specifications (2023.12.1-2) ... Setting up dh-python (6.20240824) ... Setting up python3-rich (13.7.1-1) ... Setting up pybuild-plugin-pyproject (6.20240824) ... Setting up python3-pandas-lib:amd64 (2.2.3+dfsg-4) ... Setting up python3-jupyterlab-pygments (0.2.2-3) ... Setting up python3-pandas (2.2.3+dfsg-4) ... Setting up python3-jsonschema (4.19.2-5) ... Setting up python3-jupyter-events (0.10.0-2) ... Setting up python3-nbformat (5.9.1-1) ... Setting up python3-nbclient (0.8.0-1) ... Setting up python3-nbconvert (7.16.4-2) ... Setting up python3-jupyter-server (2.14.2-4) ... Setting up node-acorn (8.8.1+ds+~cs25.17.7-2) ... Setting up libnode115:amd64 (20.17.0+dfsg-2) ... Setting up nodejs (20.17.0+dfsg-2) ... update-alternatives: using /usr/bin/nodejs to provide /usr/bin/js (js) in auto mode Setting up node-argparse (2.0.1-2) ... Setting up node-esprima (4.0.1+ds+~4.0.3-2) ... Setting up node-js-yaml (4.1.0+dfsg+~4.0.5-7) ... Setting up uglifyjs (3.17.4-2) ... Setting up python-dask-doc (2024.5.2+dfsg-1) ... Processing triggers for libc-bin (2.40-3) ... Processing triggers for sgml-base (1.31) ... Setting up docutils-common (0.21.2+dfsg-2) ... Processing triggers for sgml-base (1.31) ... Setting up python3-docutils (0.21.2+dfsg-2) ... Setting up python3-sphinx (7.4.7-3) ... Setting up python3-sphinx-click (5.1.0-1) ... Setting up python3-sphinx-design (0.6.1-1) ... Setting up python3-numpydoc (1.6.0-2) ... Setting up python3-pydata-sphinx-theme (0.15.2.is.0.7.2-1) ... Setting up python3-sphinx-book-theme (0.1.7-3) ... Setting up python3-dask-sphinx-theme (3.0.5-2) ... Processing triggers for ca-certificates (20240203) ... Updating certificates in /etc/ssl/certs... 0 added, 0 removed; done. Running hooks in /etc/ca-certificates/update.d... done. Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Writing extended state information... Building tag database... -> Finished parsing the build-deps I: Building the package I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/A99_set_merged_usr starting Not re-configuring usrmerge for trixie I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/A99_set_merged_usr finished hostname: Name or service not known I: Running cd /build/reproducible-path/dask.distributed-2024.5.2+ds.1/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-genchanges -S > ../dask.distributed_2024.5.2+ds.1-7_source.changes dpkg-buildpackage: info: source package dask.distributed dpkg-buildpackage: info: source version 2024.5.2+ds.1-7 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Julian Gilbey dpkg-source --before-build . dpkg-buildpackage: info: host architecture amd64 dpkg-source: info: using options from dask.distributed-2024.5.2+ds.1/debian/source/options: --extend-diff-ignore=^[^/]+\.egg-info/ debian/rules clean dh clean --buildsystem=pybuild debian/rules execute_after_dh_testdir make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' # Guard against version mismatch in debian/patches/fix-version.patch. # This patch needs to be updated to newer upstream version every time. grep -q 2024.5.2 distributed/_version.py make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' debian/rules override_dh_auto_clean make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' touch distributed/comm/tests/__init__.py dh_auto_clean rm distributed/http/static/js/anime.min.js distributed/http/static/js/reconnecting-websocket.min.js rm: cannot remove 'distributed/http/static/js/anime.min.js': No such file or directory rm: cannot remove 'distributed/http/static/js/reconnecting-websocket.min.js': No such file or directory make[1]: [debian/rules:92: override_dh_auto_clean] Error 1 (ignored) make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_autoreconf_clean -O--buildsystem=pybuild dh_clean -O--buildsystem=pybuild debian/rules binary dh binary --buildsystem=pybuild debian/rules execute_after_dh_testdir make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' # Guard against version mismatch in debian/patches/fix-version.patch. # This patch needs to be updated to newer upstream version every time. grep -q 2024.5.2 distributed/_version.py make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_update_autotools_config -O--buildsystem=pybuild dh_autoreconf -O--buildsystem=pybuild dh_auto_configure -O--buildsystem=pybuild debian/rules override_dh_auto_build make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' rm -f distributed/comm/tests/__init__.py set -e; \ for p in distributed/http/static/js/anime.min.js distributed/http/static/js/reconnecting-websocket.min.js; do \ uglifyjs -o $p debian/missing-sources/${p%.min.js}.js ; \ done cp /usr/share/javascript/sortable-tablesort/sortable.min.js distributed/http/static/js cp /usr/share/javascript/sortable-tablesort/sortable.min.css* distributed/http/static/css chmod a-x distributed/tests/test_utils_test.py dh_auto_build I: pybuild plugin_pyproject:129: Building wheel for python3.12 with "build" module I: pybuild base:311: python3.12 -m build --skip-dependency-check --no-isolation --wheel --outdir /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed * Building wheel... running bdist_wheel running build running build_py creating build creating build/lib creating build/lib/distributed copying distributed/node.py -> build/lib/distributed copying distributed/nanny.py -> build/lib/distributed copying distributed/metrics.py -> build/lib/distributed copying distributed/sizeof.py -> build/lib/distributed copying distributed/semaphore.py -> build/lib/distributed copying distributed/utils_comm.py -> build/lib/distributed copying distributed/_stories.py -> build/lib/distributed copying distributed/worker_state_machine.py -> build/lib/distributed copying distributed/worker_client.py -> build/lib/distributed copying distributed/spill.py -> build/lib/distributed copying distributed/event.py -> build/lib/distributed copying distributed/lock.py -> build/lib/distributed copying distributed/threadpoolexecutor.py -> build/lib/distributed copying distributed/pubsub.py -> build/lib/distributed copying distributed/chaos.py -> build/lib/distributed copying distributed/client.py -> build/lib/distributed copying distributed/worker_memory.py -> build/lib/distributed copying distributed/counter.py -> build/lib/distributed copying distributed/stealing.py -> build/lib/distributed copying distributed/worker.py -> build/lib/distributed copying distributed/pytest_resourceleaks.py -> build/lib/distributed copying distributed/utils_test.py -> build/lib/distributed copying distributed/scheduler.py -> build/lib/distributed copying distributed/multi_lock.py -> build/lib/distributed copying distributed/core.py -> build/lib/distributed copying distributed/bokeh.py -> build/lib/distributed copying distributed/preloading.py -> build/lib/distributed copying distributed/publish.py -> build/lib/distributed copying distributed/exceptions.py -> build/lib/distributed copying distributed/__init__.py -> build/lib/distributed copying distributed/profile.py -> build/lib/distributed copying distributed/actor.py -> build/lib/distributed copying distributed/diskutils.py -> build/lib/distributed copying distributed/process.py -> build/lib/distributed copying distributed/variable.py -> build/lib/distributed copying distributed/compatibility.py -> build/lib/distributed copying distributed/utils_perf.py -> build/lib/distributed copying distributed/config.py -> build/lib/distributed copying distributed/system_monitor.py -> build/lib/distributed copying distributed/batched.py -> build/lib/distributed copying distributed/security.py -> build/lib/distributed copying distributed/cfexecutor.py -> build/lib/distributed copying distributed/objects.py -> build/lib/distributed copying distributed/itertools.py -> build/lib/distributed copying distributed/proctitle.py -> build/lib/distributed copying distributed/system.py -> build/lib/distributed copying distributed/cluster_dump.py -> build/lib/distributed copying distributed/_concurrent_futures_thread.py -> build/lib/distributed copying distributed/_signals.py -> build/lib/distributed copying distributed/active_memory_manager.py -> build/lib/distributed copying distributed/spans.py -> build/lib/distributed copying distributed/utils.py -> build/lib/distributed copying distributed/_asyncio.py -> build/lib/distributed copying distributed/recreate_tasks.py -> build/lib/distributed copying distributed/_version.py -> build/lib/distributed copying distributed/versions.py -> build/lib/distributed copying distributed/collections.py -> build/lib/distributed copying distributed/queues.py -> build/lib/distributed creating build/lib/distributed/shuffle copying distributed/shuffle/_pickle.py -> build/lib/distributed/shuffle copying distributed/shuffle/_arrow.py -> build/lib/distributed/shuffle copying distributed/shuffle/_merge.py -> build/lib/distributed/shuffle copying distributed/shuffle/_disk.py -> build/lib/distributed/shuffle copying distributed/shuffle/_shuffle.py -> build/lib/distributed/shuffle copying distributed/shuffle/__init__.py -> build/lib/distributed/shuffle copying distributed/shuffle/_memory.py -> build/lib/distributed/shuffle copying distributed/shuffle/_core.py -> build/lib/distributed/shuffle copying distributed/shuffle/_rechunk.py -> build/lib/distributed/shuffle copying distributed/shuffle/_worker_plugin.py -> build/lib/distributed/shuffle copying distributed/shuffle/_comms.py -> build/lib/distributed/shuffle copying distributed/shuffle/_buffer.py -> build/lib/distributed/shuffle copying distributed/shuffle/_exceptions.py -> build/lib/distributed/shuffle copying distributed/shuffle/_limiter.py -> build/lib/distributed/shuffle copying distributed/shuffle/_scheduler_plugin.py -> build/lib/distributed/shuffle creating build/lib/distributed/http copying distributed/http/proxy.py -> build/lib/distributed/http copying distributed/http/prometheus.py -> build/lib/distributed/http copying distributed/http/__init__.py -> build/lib/distributed/http copying distributed/http/statics.py -> build/lib/distributed/http copying distributed/http/health.py -> build/lib/distributed/http copying distributed/http/utils.py -> build/lib/distributed/http copying distributed/http/routing.py -> build/lib/distributed/http creating build/lib/distributed/deploy copying distributed/deploy/local.py -> build/lib/distributed/deploy copying distributed/deploy/adaptive.py -> build/lib/distributed/deploy copying distributed/deploy/subprocess.py -> build/lib/distributed/deploy copying distributed/deploy/old_ssh.py -> build/lib/distributed/deploy copying distributed/deploy/__init__.py -> build/lib/distributed/deploy copying distributed/deploy/adaptive_core.py -> build/lib/distributed/deploy copying distributed/deploy/ssh.py -> build/lib/distributed/deploy copying distributed/deploy/utils.py -> build/lib/distributed/deploy copying distributed/deploy/cluster.py -> build/lib/distributed/deploy copying distributed/deploy/spec.py -> build/lib/distributed/deploy creating build/lib/distributed/diagnostics copying distributed/diagnostics/nvml.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/rmm.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/progressbar.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/task_stream.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/__init__.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/websocket.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/memray.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/plugin.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/graph_layout.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/cluster_dump.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/eventstream.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/progress.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/cudf.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/memory_sampler.py -> build/lib/distributed/diagnostics copying distributed/diagnostics/progress_stream.py -> build/lib/distributed/diagnostics creating build/lib/distributed/widgets copying distributed/widgets/__init__.py -> build/lib/distributed/widgets creating build/lib/distributed/protocol copying distributed/protocol/sparse.py -> build/lib/distributed/protocol copying distributed/protocol/cuda.py -> build/lib/distributed/protocol copying distributed/protocol/h5py.py -> build/lib/distributed/protocol copying distributed/protocol/keras.py -> build/lib/distributed/protocol copying distributed/protocol/serialize.py -> build/lib/distributed/protocol copying distributed/protocol/rmm.py -> build/lib/distributed/protocol copying distributed/protocol/utils_test.py -> build/lib/distributed/protocol copying distributed/protocol/core.py -> build/lib/distributed/protocol copying distributed/protocol/torch.py -> build/lib/distributed/protocol copying distributed/protocol/netcdf4.py -> build/lib/distributed/protocol copying distributed/protocol/__init__.py -> build/lib/distributed/protocol copying distributed/protocol/compression.py -> build/lib/distributed/protocol copying distributed/protocol/scipy.py -> build/lib/distributed/protocol copying distributed/protocol/arrow.py -> build/lib/distributed/protocol copying distributed/protocol/numpy.py -> build/lib/distributed/protocol copying distributed/protocol/utils.py -> build/lib/distributed/protocol copying distributed/protocol/pickle.py -> build/lib/distributed/protocol copying distributed/protocol/cupy.py -> build/lib/distributed/protocol copying distributed/protocol/numba.py -> build/lib/distributed/protocol creating build/lib/distributed/dashboard copying distributed/dashboard/worker.py -> build/lib/distributed/dashboard copying distributed/dashboard/scheduler.py -> build/lib/distributed/dashboard copying distributed/dashboard/core.py -> build/lib/distributed/dashboard copying distributed/dashboard/__init__.py -> build/lib/distributed/dashboard copying distributed/dashboard/export_tool.py -> build/lib/distributed/dashboard copying distributed/dashboard/utils.py -> build/lib/distributed/dashboard creating build/lib/distributed/comm copying distributed/comm/registry.py -> build/lib/distributed/comm copying distributed/comm/core.py -> build/lib/distributed/comm copying distributed/comm/addressing.py -> build/lib/distributed/comm copying distributed/comm/__init__.py -> build/lib/distributed/comm copying distributed/comm/inproc.py -> build/lib/distributed/comm copying distributed/comm/ucx.py -> build/lib/distributed/comm copying distributed/comm/tcp.py -> build/lib/distributed/comm copying distributed/comm/utils.py -> build/lib/distributed/comm copying distributed/comm/ws.py -> build/lib/distributed/comm creating build/lib/distributed/cli copying distributed/cli/dask_worker.py -> build/lib/distributed/cli copying distributed/cli/dask_ssh.py -> build/lib/distributed/cli copying distributed/cli/dask_scheduler.py -> build/lib/distributed/cli copying distributed/cli/__init__.py -> build/lib/distributed/cli copying distributed/cli/utils.py -> build/lib/distributed/cli copying distributed/cli/dask_spec.py -> build/lib/distributed/cli creating build/lib/distributed/http/scheduler copying distributed/http/scheduler/json.py -> build/lib/distributed/http/scheduler copying distributed/http/scheduler/__init__.py -> build/lib/distributed/http/scheduler copying distributed/http/scheduler/api.py -> build/lib/distributed/http/scheduler copying distributed/http/scheduler/info.py -> build/lib/distributed/http/scheduler copying distributed/http/scheduler/missing_bokeh.py -> build/lib/distributed/http/scheduler creating build/lib/distributed/http/static copying distributed/http/static/__init__.py -> build/lib/distributed/http/static creating build/lib/distributed/http/templates copying distributed/http/templates/__init__.py -> build/lib/distributed/http/templates creating build/lib/distributed/http/worker copying distributed/http/worker/__init__.py -> build/lib/distributed/http/worker creating build/lib/distributed/http/scheduler/prometheus copying distributed/http/scheduler/prometheus/semaphore.py -> build/lib/distributed/http/scheduler/prometheus copying distributed/http/scheduler/prometheus/stealing.py -> build/lib/distributed/http/scheduler/prometheus copying distributed/http/scheduler/prometheus/core.py -> build/lib/distributed/http/scheduler/prometheus copying distributed/http/scheduler/prometheus/__init__.py -> build/lib/distributed/http/scheduler/prometheus creating build/lib/distributed/http/static/images copying distributed/http/static/images/__init__.py -> build/lib/distributed/http/static/images creating build/lib/distributed/http/static/css copying distributed/http/static/css/__init__.py -> build/lib/distributed/http/static/css creating build/lib/distributed/http/static/js copying distributed/http/static/js/__init__.py -> build/lib/distributed/http/static/js creating build/lib/distributed/http/worker/prometheus copying distributed/http/worker/prometheus/core.py -> build/lib/distributed/http/worker/prometheus copying distributed/http/worker/prometheus/__init__.py -> build/lib/distributed/http/worker/prometheus creating build/lib/distributed/widgets/templates copying distributed/widgets/templates/__init__.py -> build/lib/distributed/widgets/templates creating build/lib/distributed/dashboard/components copying distributed/dashboard/components/nvml.py -> build/lib/distributed/dashboard/components copying distributed/dashboard/components/rmm.py -> build/lib/distributed/dashboard/components copying distributed/dashboard/components/worker.py -> build/lib/distributed/dashboard/components copying distributed/dashboard/components/scheduler.py -> build/lib/distributed/dashboard/components copying distributed/dashboard/components/__init__.py -> build/lib/distributed/dashboard/components copying distributed/dashboard/components/shared.py -> build/lib/distributed/dashboard/components creating build/lib/distributed/dashboard/templates copying distributed/dashboard/templates/__init__.py -> build/lib/distributed/dashboard/templates running egg_info creating distributed.egg-info writing distributed.egg-info/PKG-INFO writing dependency_links to distributed.egg-info/dependency_links.txt writing entry points to distributed.egg-info/entry_points.txt writing requirements to distributed.egg-info/requires.txt writing top-level names to distributed.egg-info/top_level.txt writing manifest file 'distributed.egg-info/SOURCES.txt' reading manifest file 'distributed.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching '*.coffee' under directory 'distributed' no previously-included directories found matching 'docs/_build' adding license file 'LICENSE.txt' adding license file 'AUTHORS.md' writing manifest file 'distributed.egg-info/SOURCES.txt' /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.cli.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.cli.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.cli.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.cli.tests' to be distributed and are already explicitly excluding 'distributed.cli.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.comm.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.comm.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.comm.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.comm.tests' to be distributed and are already explicitly excluding 'distributed.comm.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.dashboard.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.dashboard.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.dashboard.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.dashboard.tests' to be distributed and are already explicitly excluding 'distributed.dashboard.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.deploy.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.deploy.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.deploy.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.deploy.tests' to be distributed and are already explicitly excluding 'distributed.deploy.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.diagnostics.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.diagnostics.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.diagnostics.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.diagnostics.tests' to be distributed and are already explicitly excluding 'distributed.diagnostics.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.http.scheduler.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.http.scheduler.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.http.scheduler.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.http.scheduler.tests' to be distributed and are already explicitly excluding 'distributed.http.scheduler.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.http.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.http.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.http.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.http.tests' to be distributed and are already explicitly excluding 'distributed.http.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.http.worker.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.http.worker.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.http.worker.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.http.worker.tests' to be distributed and are already explicitly excluding 'distributed.http.worker.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.protocol.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.protocol.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.protocol.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.protocol.tests' to be distributed and are already explicitly excluding 'distributed.protocol.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.shuffle.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.shuffle.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.shuffle.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.shuffle.tests' to be distributed and are already explicitly excluding 'distributed.shuffle.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) /usr/lib/python3/dist-packages/setuptools/command/build_py.py:218: _Warning: Package 'distributed.tests' is absent from the `packages` configuration. !! ******************************************************************************** ############################ # Package would be ignored # ############################ Python recognizes 'distributed.tests' as an importable package[^1], but it is absent from setuptools' `packages` configuration. This leads to an ambiguous overall configuration. If you want to distribute this package, please make sure that 'distributed.tests' is explicitly added to the `packages` configuration field. Alternatively, you can also rely on setuptools' discovery methods (for example by using `find_namespace_packages(...)`/`find_namespace:` instead of `find_packages(...)`/`find:`). You can read more about "package discovery" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html If you don't want 'distributed.tests' to be distributed and are already explicitly excluding 'distributed.tests' via `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`, you can try to use `exclude_package_data`, or `include-package-data=False` in combination with a more fine grained `package-data` configuration. You can read more about "package data files" on setuptools documentation page: - https://setuptools.pypa.io/en/latest/userguide/datafiles.html [^1]: For Python, any directory (with suitable naming) can be imported, even if it does not contain any `.py` files. On the other hand, currently there is no concept of package data directory, all directories are treated like packages. ******************************************************************************** !! check.warn(importable) copying distributed/distributed-schema.yaml -> build/lib/distributed copying distributed/distributed.yaml -> build/lib/distributed copying distributed/py.typed -> build/lib/distributed creating build/lib/distributed/tests copying distributed/tests/make_tls_certs.py -> build/lib/distributed/tests copying distributed/tests/test_active_memory_manager.py -> build/lib/distributed/tests copying distributed/tests/test_actor.py -> build/lib/distributed/tests copying distributed/tests/test_as_completed.py -> build/lib/distributed/tests copying distributed/tests/test_asyncio.py -> build/lib/distributed/tests copying distributed/tests/test_asyncprocess.py -> build/lib/distributed/tests copying distributed/tests/test_batched.py -> build/lib/distributed/tests copying distributed/tests/test_cancelled_state.py -> build/lib/distributed/tests copying distributed/tests/test_chaos.py -> build/lib/distributed/tests copying distributed/tests/test_client.py -> build/lib/distributed/tests copying distributed/tests/test_client_executor.py -> build/lib/distributed/tests copying distributed/tests/test_client_loop.py -> build/lib/distributed/tests copying distributed/tests/test_cluster_dump.py -> build/lib/distributed/tests copying distributed/tests/test_collections.py -> build/lib/distributed/tests copying distributed/tests/test_compatibility.py -> build/lib/distributed/tests copying distributed/tests/test_computations.py -> build/lib/distributed/tests copying distributed/tests/test_config.py -> build/lib/distributed/tests copying distributed/tests/test_core.py -> build/lib/distributed/tests copying distributed/tests/test_counter.py -> build/lib/distributed/tests copying distributed/tests/test_dask_collections.py -> build/lib/distributed/tests copying distributed/tests/test_deadline.py -> build/lib/distributed/tests copying distributed/tests/test_diskutils.py -> build/lib/distributed/tests copying distributed/tests/test_events.py -> build/lib/distributed/tests copying distributed/tests/test_failed_workers.py -> build/lib/distributed/tests copying distributed/tests/test_imports.py -> build/lib/distributed/tests copying distributed/tests/test_init.py -> build/lib/distributed/tests copying distributed/tests/test_itertools.py -> build/lib/distributed/tests copying distributed/tests/test_jupyter.py -> build/lib/distributed/tests copying distributed/tests/test_locks.py -> build/lib/distributed/tests copying distributed/tests/test_metrics.py -> build/lib/distributed/tests copying distributed/tests/test_multi_locks.py -> build/lib/distributed/tests copying distributed/tests/test_nanny.py -> build/lib/distributed/tests copying distributed/tests/test_parse_stdout.py -> build/lib/distributed/tests copying distributed/tests/test_preload.py -> build/lib/distributed/tests copying distributed/tests/test_priorities.py -> build/lib/distributed/tests copying distributed/tests/test_profile.py -> build/lib/distributed/tests copying distributed/tests/test_publish.py -> build/lib/distributed/tests copying distributed/tests/test_pubsub.py -> build/lib/distributed/tests copying distributed/tests/test_queues.py -> build/lib/distributed/tests copying distributed/tests/test_reschedule.py -> build/lib/distributed/tests copying distributed/tests/test_resources.py -> build/lib/distributed/tests copying distributed/tests/test_scheduler.py -> build/lib/distributed/tests copying distributed/tests/test_security.py -> build/lib/distributed/tests copying distributed/tests/test_semaphore.py -> build/lib/distributed/tests copying distributed/tests/test_sizeof.py -> build/lib/distributed/tests copying distributed/tests/test_spans.py -> build/lib/distributed/tests copying distributed/tests/test_spec.py -> build/lib/distributed/tests copying distributed/tests/test_spill.py -> build/lib/distributed/tests copying distributed/tests/test_steal.py -> build/lib/distributed/tests copying distributed/tests/test_stories.py -> build/lib/distributed/tests copying distributed/tests/test_stress.py -> build/lib/distributed/tests copying distributed/tests/test_system.py -> build/lib/distributed/tests copying distributed/tests/test_system_monitor.py -> build/lib/distributed/tests copying distributed/tests/test_threadpoolexecutor.py -> build/lib/distributed/tests copying distributed/tests/test_tls_functional.py -> build/lib/distributed/tests copying distributed/tests/test_utils.py -> build/lib/distributed/tests copying distributed/tests/test_utils_comm.py -> build/lib/distributed/tests copying distributed/tests/test_utils_perf.py -> build/lib/distributed/tests copying distributed/tests/test_utils_test.py -> build/lib/distributed/tests copying distributed/tests/test_variable.py -> build/lib/distributed/tests copying distributed/tests/test_versions.py -> build/lib/distributed/tests copying distributed/tests/test_worker.py -> build/lib/distributed/tests copying distributed/tests/test_worker_client.py -> build/lib/distributed/tests copying distributed/tests/test_worker_memory.py -> build/lib/distributed/tests copying distributed/tests/test_worker_metrics.py -> build/lib/distributed/tests copying distributed/tests/test_worker_state_machine.py -> build/lib/distributed/tests copying distributed/tests/tls-ca-cert.pem -> build/lib/distributed/tests copying distributed/tests/tls-ca-key.pem -> build/lib/distributed/tests copying distributed/tests/tls-cert-chain.pem -> build/lib/distributed/tests copying distributed/tests/tls-cert.pem -> build/lib/distributed/tests copying distributed/tests/tls-key-cert.pem -> build/lib/distributed/tests copying distributed/tests/tls-key.pem -> build/lib/distributed/tests copying distributed/tests/tls-self-signed-cert.pem -> build/lib/distributed/tests copying distributed/tests/tls-self-signed-key.pem -> build/lib/distributed/tests copying distributed/http/templates/call-stack.html -> build/lib/distributed/http/templates copying distributed/http/templates/status.html -> build/lib/distributed/http/templates copying distributed/http/templates/json-index.html -> build/lib/distributed/http/templates copying distributed/http/templates/task.html -> build/lib/distributed/http/templates copying distributed/http/templates/simple.html -> build/lib/distributed/http/templates copying distributed/http/templates/base.html -> build/lib/distributed/http/templates copying distributed/http/templates/main.html -> build/lib/distributed/http/templates copying distributed/http/templates/worker-table.html -> build/lib/distributed/http/templates copying distributed/http/templates/gpu.html -> build/lib/distributed/http/templates copying distributed/http/templates/logs.html -> build/lib/distributed/http/templates copying distributed/http/templates/workers.html -> build/lib/distributed/http/templates copying distributed/http/templates/exceptions.html -> build/lib/distributed/http/templates copying distributed/http/templates/worker.html -> build/lib/distributed/http/templates creating build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/__init__.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_buffer.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_comm_buffer.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_core.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_disk_buffer.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_graph.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_limiter.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_memory_buffer.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_merge.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_merge_column_and_index.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_metrics.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_pickle.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_read_write_lock.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_rechunk.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_shuffle.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/test_shuffle_plugins.py -> build/lib/distributed/shuffle/tests copying distributed/shuffle/tests/utils.py -> build/lib/distributed/shuffle/tests creating build/lib/distributed/http/tests copying distributed/http/tests/__init__.py -> build/lib/distributed/http/tests copying distributed/http/tests/test_core.py -> build/lib/distributed/http/tests copying distributed/http/tests/test_routing.py -> build/lib/distributed/http/tests creating build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_adaptive.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_adaptive_core.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_cluster.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_deploy_utils.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_local.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_old_ssh.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_slow_adaptive.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_spec_cluster.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_ssh.py -> build/lib/distributed/deploy/tests copying distributed/deploy/tests/test_subprocess.py -> build/lib/distributed/deploy/tests creating build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_cluster_dump_plugin.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_cudf_diagnostics.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_eventstream.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_graph_layout.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_install_plugin.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_memory_sampler.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_memray.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_nanny_plugin.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_nvml.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_progress.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_progress_stream.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_progress_widgets.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_progressbar.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_rmm_diagnostics.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_scheduler_plugin.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_task_stream.py -> build/lib/distributed/diagnostics/tests copying distributed/diagnostics/tests/test_worker_plugin.py -> build/lib/distributed/diagnostics/tests creating build/lib/distributed/protocol/tests copying distributed/protocol/tests/__init__.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_arrow.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_collection.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_collection_cuda.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_compression.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_cupy.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_h5py.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_highlevelgraph.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_keras.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_netcdf4.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_numba.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_numpy.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_pandas.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_pickle.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_protocol.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_protocol_utils.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_rmm.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_scipy.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_serialize.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_sparse.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_to_pickle.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_torch.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_utils.py -> build/lib/distributed/protocol/tests copying distributed/protocol/tests/test_utils_test.py -> build/lib/distributed/protocol/tests copying distributed/dashboard/theme.yaml -> build/lib/distributed/dashboard creating build/lib/distributed/dashboard/tests copying distributed/dashboard/tests/test_bokeh.py -> build/lib/distributed/dashboard/tests copying distributed/dashboard/tests/test_components.py -> build/lib/distributed/dashboard/tests copying distributed/dashboard/tests/test_scheduler_bokeh.py -> build/lib/distributed/dashboard/tests copying distributed/dashboard/tests/test_worker_bokeh.py -> build/lib/distributed/dashboard/tests creating build/lib/distributed/comm/tests copying distributed/comm/tests/test_comms.py -> build/lib/distributed/comm/tests copying distributed/comm/tests/test_tcp.py -> build/lib/distributed/comm/tests copying distributed/comm/tests/test_ucx.py -> build/lib/distributed/comm/tests copying distributed/comm/tests/test_ucx_config.py -> build/lib/distributed/comm/tests copying distributed/comm/tests/test_ws.py -> build/lib/distributed/comm/tests creating build/lib/distributed/cli/tests copying distributed/cli/tests/__init__.py -> build/lib/distributed/cli/tests copying distributed/cli/tests/test_dask_scheduler.py -> build/lib/distributed/cli/tests copying distributed/cli/tests/test_dask_spec.py -> build/lib/distributed/cli/tests copying distributed/cli/tests/test_dask_ssh.py -> build/lib/distributed/cli/tests copying distributed/cli/tests/test_dask_worker.py -> build/lib/distributed/cli/tests copying distributed/cli/tests/test_tls_cli.py -> build/lib/distributed/cli/tests creating build/lib/distributed/http/scheduler/tests copying distributed/http/scheduler/tests/test_missing_bokeh.py -> build/lib/distributed/http/scheduler/tests copying distributed/http/scheduler/tests/test_scheduler_http.py -> build/lib/distributed/http/scheduler/tests copying distributed/http/scheduler/tests/test_semaphore_http.py -> build/lib/distributed/http/scheduler/tests copying distributed/http/scheduler/tests/test_stealing_http.py -> build/lib/distributed/http/scheduler/tests copying distributed/http/static/individual-cluster-map.html -> build/lib/distributed/http/static creating build/lib/distributed/http/worker/tests copying distributed/http/worker/tests/test_worker_http.py -> build/lib/distributed/http/worker/tests copying distributed/http/static/images/dask-logo.svg -> build/lib/distributed/http/static/images copying distributed/http/static/images/fa-bars.svg -> build/lib/distributed/http/static/images copying distributed/http/static/images/favicon.ico -> build/lib/distributed/http/static/images copying distributed/http/static/images/jupyter.svg -> build/lib/distributed/http/static/images copying distributed/http/static/images/numpy.png -> build/lib/distributed/http/static/images copying distributed/http/static/images/pandas.png -> build/lib/distributed/http/static/images copying distributed/http/static/images/python.png -> build/lib/distributed/http/static/images copying distributed/http/static/css/base.css -> build/lib/distributed/http/static/css copying distributed/http/static/css/gpu.css -> build/lib/distributed/http/static/css copying distributed/http/static/css/individual-cluster-map.css -> build/lib/distributed/http/static/css copying distributed/http/static/css/sortable.min.css -> build/lib/distributed/http/static/css copying distributed/http/static/css/status.css -> build/lib/distributed/http/static/css copying distributed/http/static/js/anime.min.js -> build/lib/distributed/http/static/js copying distributed/http/static/js/individual-cluster-map.js -> build/lib/distributed/http/static/js copying distributed/http/static/js/reconnecting-websocket.min.js -> build/lib/distributed/http/static/js copying distributed/http/static/js/sortable.min.js -> build/lib/distributed/http/static/js copying distributed/widgets/templates/client.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/cluster.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/computation.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/future.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/has_what.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/local_cluster.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/log.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/logs.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/process_interface.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/scheduler.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/scheduler_info.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/security.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/task_state.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/who_has.html.j2 -> build/lib/distributed/widgets/templates copying distributed/widgets/templates/worker_state.html.j2 -> build/lib/distributed/widgets/templates copying distributed/dashboard/templates/performance_report.html -> build/lib/distributed/dashboard/templates UPDATING build/lib/distributed/_version.py set build/lib/distributed/_version.py to '2024.5.2' installing to build/bdist.linux-x86_64/wheel running install running install_lib creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/wheel creating build/bdist.linux-x86_64/wheel/distributed creating build/bdist.linux-x86_64/wheel/distributed/shuffle copying build/lib/distributed/shuffle/_pickle.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_arrow.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_merge.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_disk.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_shuffle.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_memory.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle creating build/bdist.linux-x86_64/wheel/distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_core.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_comm_buffer.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_limiter.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_pickle.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_merge_column_and_index.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_disk_buffer.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_buffer.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_read_write_lock.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_shuffle.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_memory_buffer.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_metrics.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_graph.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_shuffle_plugins.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_merge.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/tests/test_rechunk.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle/tests copying build/lib/distributed/shuffle/_core.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_rechunk.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_worker_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_comms.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_buffer.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_exceptions.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_limiter.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/shuffle/_scheduler_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/shuffle copying build/lib/distributed/node.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/http creating build/bdist.linux-x86_64/wheel/distributed/http/scheduler copying build/lib/distributed/http/scheduler/json.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler copying build/lib/distributed/http/scheduler/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler creating build/bdist.linux-x86_64/wheel/distributed/http/scheduler/tests copying build/lib/distributed/http/scheduler/tests/test_missing_bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/tests copying build/lib/distributed/http/scheduler/tests/test_stealing_http.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/tests copying build/lib/distributed/http/scheduler/tests/test_scheduler_http.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/tests copying build/lib/distributed/http/scheduler/tests/test_semaphore_http.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/tests copying build/lib/distributed/http/scheduler/api.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler copying build/lib/distributed/http/scheduler/info.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler copying build/lib/distributed/http/scheduler/missing_bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler creating build/bdist.linux-x86_64/wheel/distributed/http/scheduler/prometheus copying build/lib/distributed/http/scheduler/prometheus/semaphore.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/prometheus copying build/lib/distributed/http/scheduler/prometheus/stealing.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/prometheus copying build/lib/distributed/http/scheduler/prometheus/core.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/prometheus copying build/lib/distributed/http/scheduler/prometheus/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/scheduler/prometheus copying build/lib/distributed/http/proxy.py -> build/bdist.linux-x86_64/wheel/./distributed/http copying build/lib/distributed/http/prometheus.py -> build/bdist.linux-x86_64/wheel/./distributed/http copying build/lib/distributed/http/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http creating build/bdist.linux-x86_64/wheel/distributed/http/tests copying build/lib/distributed/http/tests/test_core.py -> build/bdist.linux-x86_64/wheel/./distributed/http/tests copying build/lib/distributed/http/tests/test_routing.py -> build/bdist.linux-x86_64/wheel/./distributed/http/tests copying build/lib/distributed/http/tests/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/tests copying build/lib/distributed/http/statics.py -> build/bdist.linux-x86_64/wheel/./distributed/http copying build/lib/distributed/http/health.py -> build/bdist.linux-x86_64/wheel/./distributed/http copying build/lib/distributed/http/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/http creating build/bdist.linux-x86_64/wheel/distributed/http/static creating build/bdist.linux-x86_64/wheel/distributed/http/static/images copying build/lib/distributed/http/static/images/jupyter.svg -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/favicon.ico -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/pandas.png -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/numpy.png -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/fa-bars.svg -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/python.png -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images copying build/lib/distributed/http/static/images/dask-logo.svg -> build/bdist.linux-x86_64/wheel/./distributed/http/static/images creating build/bdist.linux-x86_64/wheel/distributed/http/static/css copying build/lib/distributed/http/static/css/gpu.css -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/css/base.css -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/css/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/css/status.css -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/css/sortable.min.css -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/css/individual-cluster-map.css -> build/bdist.linux-x86_64/wheel/./distributed/http/static/css copying build/lib/distributed/http/static/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/static copying build/lib/distributed/http/static/individual-cluster-map.html -> build/bdist.linux-x86_64/wheel/./distributed/http/static creating build/bdist.linux-x86_64/wheel/distributed/http/static/js copying build/lib/distributed/http/static/js/individual-cluster-map.js -> build/bdist.linux-x86_64/wheel/./distributed/http/static/js copying build/lib/distributed/http/static/js/sortable.min.js -> build/bdist.linux-x86_64/wheel/./distributed/http/static/js copying build/lib/distributed/http/static/js/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/static/js copying build/lib/distributed/http/static/js/reconnecting-websocket.min.js -> build/bdist.linux-x86_64/wheel/./distributed/http/static/js copying build/lib/distributed/http/static/js/anime.min.js -> build/bdist.linux-x86_64/wheel/./distributed/http/static/js creating build/bdist.linux-x86_64/wheel/distributed/http/templates copying build/lib/distributed/http/templates/call-stack.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/status.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/json-index.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/task.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/simple.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/base.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/main.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/worker-table.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/gpu.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/logs.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/workers.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/exceptions.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/templates/worker.html -> build/bdist.linux-x86_64/wheel/./distributed/http/templates copying build/lib/distributed/http/routing.py -> build/bdist.linux-x86_64/wheel/./distributed/http creating build/bdist.linux-x86_64/wheel/distributed/http/worker copying build/lib/distributed/http/worker/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/worker creating build/bdist.linux-x86_64/wheel/distributed/http/worker/tests copying build/lib/distributed/http/worker/tests/test_worker_http.py -> build/bdist.linux-x86_64/wheel/./distributed/http/worker/tests creating build/bdist.linux-x86_64/wheel/distributed/http/worker/prometheus copying build/lib/distributed/http/worker/prometheus/core.py -> build/bdist.linux-x86_64/wheel/./distributed/http/worker/prometheus copying build/lib/distributed/http/worker/prometheus/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/http/worker/prometheus copying build/lib/distributed/nanny.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/metrics.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/sizeof.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/semaphore.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/utils_comm.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/deploy copying build/lib/distributed/deploy/local.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/adaptive.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/subprocess.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/old_ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/adaptive_core.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy creating build/bdist.linux-x86_64/wheel/distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_slow_adaptive.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_adaptive_core.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_deploy_utils.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_adaptive.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_spec_cluster.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_local.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_cluster.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_subprocess.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/tests/test_old_ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy/tests copying build/lib/distributed/deploy/ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/cluster.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/deploy/spec.py -> build/bdist.linux-x86_64/wheel/./distributed/deploy copying build/lib/distributed/_stories.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/worker_state_machine.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/worker_client.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/spill.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/event.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/diagnostics copying build/lib/distributed/diagnostics/nvml.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/rmm.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/progressbar.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/task_stream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/websocket.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/memray.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics creating build/bdist.linux-x86_64/wheel/distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_install_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_worker_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_graph_layout.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_task_stream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_nvml.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_cluster_dump_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_memory_sampler.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_nanny_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_progress.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_memray.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_scheduler_plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_progressbar.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_progress_widgets.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_progress_stream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_cudf_diagnostics.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_eventstream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/tests/test_rmm_diagnostics.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics/tests copying build/lib/distributed/diagnostics/plugin.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/graph_layout.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/cluster_dump.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/eventstream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/progress.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/cudf.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/memory_sampler.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/diagnostics/progress_stream.py -> build/bdist.linux-x86_64/wheel/./distributed/diagnostics copying build/lib/distributed/lock.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/threadpoolexecutor.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/pubsub.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/chaos.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/client.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/worker_memory.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/counter.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/stealing.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/widgets copying build/lib/distributed/widgets/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/widgets creating build/bdist.linux-x86_64/wheel/distributed/widgets/templates copying build/lib/distributed/widgets/templates/log.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/who_has.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/local_cluster.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/security.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/client.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/computation.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/task_state.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/has_what.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/future.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/cluster.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/logs.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/scheduler.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/worker_state.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/scheduler_info.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/widgets/templates/process_interface.html.j2 -> build/bdist.linux-x86_64/wheel/./distributed/widgets/templates copying build/lib/distributed/worker.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/protocol copying build/lib/distributed/protocol/sparse.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/cuda.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/h5py.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/keras.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/serialize.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/rmm.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/utils_test.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/core.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/torch.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/netcdf4.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/compression.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol creating build/bdist.linux-x86_64/wheel/distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_serialize.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_scipy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_sparse.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_protocol.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_collection.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_arrow.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_pickle.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_numba.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_highlevelgraph.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_utils_test.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_cupy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_numpy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_pandas.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_netcdf4.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_compression.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_protocol_utils.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_rmm.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_to_pickle.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_keras.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_utils.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_torch.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_collection_cuda.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/tests/test_h5py.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol/tests copying build/lib/distributed/protocol/scipy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/arrow.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/numpy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/pickle.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/cupy.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/protocol/numba.py -> build/bdist.linux-x86_64/wheel/./distributed/protocol copying build/lib/distributed/pytest_resourceleaks.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/utils_test.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/multi_lock.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/core.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/preloading.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/publish.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/exceptions.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/profile.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/actor.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/diskutils.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/py.typed -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/process.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/tests copying build/lib/distributed/tests/test_events.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_asyncio.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_counter.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-self-signed-key.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_system_monitor.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_steal.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_threadpoolexecutor.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_security.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_semaphore.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_jupyter.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_worker_metrics.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_stories.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_queues.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_failed_workers.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_worker_client.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_core.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_stress.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_nanny.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_cancelled_state.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_system.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_utils_test.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_dask_collections.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_imports.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_profile.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_collections.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-self-signed-cert.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_worker_state_machine.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_active_memory_manager.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_parse_stdout.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_batched.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-cert-chain.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_deadline.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_cluster_dump.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_diskutils.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_spans.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_pubsub.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_publish.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_sizeof.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_as_completed.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-ca-key.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_spill.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-key.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_client_loop.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_variable.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-ca-cert.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_utils_perf.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_versions.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_computations.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_utils_comm.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_locks.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_chaos.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_worker_memory.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_spec.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_client_executor.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-key-cert.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_actor.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_reschedule.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_client.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_utils.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_init.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/tls-cert.pem -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_worker.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_itertools.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_tls_functional.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_asyncprocess.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_config.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_resources.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_metrics.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_preload.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_compatibility.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/make_tls_certs.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_priorities.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/tests/test_multi_locks.py -> build/bdist.linux-x86_64/wheel/./distributed/tests copying build/lib/distributed/variable.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/compatibility.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/utils_perf.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/distributed.yaml -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/config.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/system_monitor.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/batched.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/security.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/cfexecutor.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/objects.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/itertools.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/proctitle.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/system.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/cluster_dump.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/_concurrent_futures_thread.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/_signals.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/active_memory_manager.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/spans.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/distributed-schema.yaml -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/dashboard creating build/bdist.linux-x86_64/wheel/distributed/dashboard/components copying build/lib/distributed/dashboard/components/nvml.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/components/rmm.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/components/worker.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/components/scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/components/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/components/shared.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/components copying build/lib/distributed/dashboard/worker.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard copying build/lib/distributed/dashboard/scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard copying build/lib/distributed/dashboard/core.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard copying build/lib/distributed/dashboard/theme.yaml -> build/bdist.linux-x86_64/wheel/./distributed/dashboard copying build/lib/distributed/dashboard/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard creating build/bdist.linux-x86_64/wheel/distributed/dashboard/tests copying build/lib/distributed/dashboard/tests/test_worker_bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/tests copying build/lib/distributed/dashboard/tests/test_scheduler_bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/tests copying build/lib/distributed/dashboard/tests/test_bokeh.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/tests copying build/lib/distributed/dashboard/tests/test_components.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/tests copying build/lib/distributed/dashboard/export_tool.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard copying build/lib/distributed/dashboard/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard creating build/bdist.linux-x86_64/wheel/distributed/dashboard/templates copying build/lib/distributed/dashboard/templates/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/templates copying build/lib/distributed/dashboard/templates/performance_report.html -> build/bdist.linux-x86_64/wheel/./distributed/dashboard/templates copying build/lib/distributed/utils.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/comm copying build/lib/distributed/comm/registry.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/core.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/addressing.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/inproc.py -> build/bdist.linux-x86_64/wheel/./distributed/comm creating build/bdist.linux-x86_64/wheel/distributed/comm/tests copying build/lib/distributed/comm/tests/test_ucx_config.py -> build/bdist.linux-x86_64/wheel/./distributed/comm/tests copying build/lib/distributed/comm/tests/test_tcp.py -> build/bdist.linux-x86_64/wheel/./distributed/comm/tests copying build/lib/distributed/comm/tests/test_ws.py -> build/bdist.linux-x86_64/wheel/./distributed/comm/tests copying build/lib/distributed/comm/tests/test_ucx.py -> build/bdist.linux-x86_64/wheel/./distributed/comm/tests copying build/lib/distributed/comm/tests/test_comms.py -> build/bdist.linux-x86_64/wheel/./distributed/comm/tests copying build/lib/distributed/comm/ucx.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/tcp.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/comm/ws.py -> build/bdist.linux-x86_64/wheel/./distributed/comm copying build/lib/distributed/_asyncio.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/recreate_tasks.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/_version.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/versions.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/collections.py -> build/bdist.linux-x86_64/wheel/./distributed copying build/lib/distributed/queues.py -> build/bdist.linux-x86_64/wheel/./distributed creating build/bdist.linux-x86_64/wheel/distributed/cli copying build/lib/distributed/cli/dask_worker.py -> build/bdist.linux-x86_64/wheel/./distributed/cli copying build/lib/distributed/cli/dask_ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/cli copying build/lib/distributed/cli/dask_scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed/cli copying build/lib/distributed/cli/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/cli creating build/bdist.linux-x86_64/wheel/distributed/cli/tests copying build/lib/distributed/cli/tests/test_dask_worker.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/tests/__init__.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/tests/test_dask_scheduler.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/tests/test_dask_ssh.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/tests/test_tls_cli.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/tests/test_dask_spec.py -> build/bdist.linux-x86_64/wheel/./distributed/cli/tests copying build/lib/distributed/cli/utils.py -> build/bdist.linux-x86_64/wheel/./distributed/cli copying build/lib/distributed/cli/dask_spec.py -> build/bdist.linux-x86_64/wheel/./distributed/cli running install_egg_info Copying distributed.egg-info to build/bdist.linux-x86_64/wheel/./distributed-2024.5.2.egg-info running install_scripts creating build/bdist.linux-x86_64/wheel/distributed-2024.5.2.dist-info/WHEEL creating '/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/.tmp-lpusj2qc/distributed-2024.5.2-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it adding 'distributed/__init__.py' adding 'distributed/_asyncio.py' adding 'distributed/_concurrent_futures_thread.py' adding 'distributed/_signals.py' adding 'distributed/_stories.py' adding 'distributed/_version.py' adding 'distributed/active_memory_manager.py' adding 'distributed/actor.py' adding 'distributed/batched.py' adding 'distributed/bokeh.py' adding 'distributed/cfexecutor.py' adding 'distributed/chaos.py' adding 'distributed/client.py' adding 'distributed/cluster_dump.py' adding 'distributed/collections.py' adding 'distributed/compatibility.py' adding 'distributed/config.py' adding 'distributed/core.py' adding 'distributed/counter.py' adding 'distributed/diskutils.py' adding 'distributed/distributed-schema.yaml' adding 'distributed/distributed.yaml' adding 'distributed/event.py' adding 'distributed/exceptions.py' adding 'distributed/itertools.py' adding 'distributed/lock.py' adding 'distributed/metrics.py' adding 'distributed/multi_lock.py' adding 'distributed/nanny.py' adding 'distributed/node.py' adding 'distributed/objects.py' adding 'distributed/preloading.py' adding 'distributed/process.py' adding 'distributed/proctitle.py' adding 'distributed/profile.py' adding 'distributed/publish.py' adding 'distributed/pubsub.py' adding 'distributed/py.typed' adding 'distributed/pytest_resourceleaks.py' adding 'distributed/queues.py' adding 'distributed/recreate_tasks.py' adding 'distributed/scheduler.py' adding 'distributed/security.py' adding 'distributed/semaphore.py' adding 'distributed/sizeof.py' adding 'distributed/spans.py' adding 'distributed/spill.py' adding 'distributed/stealing.py' adding 'distributed/system.py' adding 'distributed/system_monitor.py' adding 'distributed/threadpoolexecutor.py' adding 'distributed/utils.py' adding 'distributed/utils_comm.py' adding 'distributed/utils_perf.py' adding 'distributed/utils_test.py' adding 'distributed/variable.py' adding 'distributed/versions.py' adding 'distributed/worker.py' adding 'distributed/worker_client.py' adding 'distributed/worker_memory.py' adding 'distributed/worker_state_machine.py' adding 'distributed/cli/__init__.py' adding 'distributed/cli/dask_scheduler.py' adding 'distributed/cli/dask_spec.py' adding 'distributed/cli/dask_ssh.py' adding 'distributed/cli/dask_worker.py' adding 'distributed/cli/utils.py' adding 'distributed/cli/tests/__init__.py' adding 'distributed/cli/tests/test_dask_scheduler.py' adding 'distributed/cli/tests/test_dask_spec.py' adding 'distributed/cli/tests/test_dask_ssh.py' adding 'distributed/cli/tests/test_dask_worker.py' adding 'distributed/cli/tests/test_tls_cli.py' adding 'distributed/comm/__init__.py' adding 'distributed/comm/addressing.py' adding 'distributed/comm/core.py' adding 'distributed/comm/inproc.py' adding 'distributed/comm/registry.py' adding 'distributed/comm/tcp.py' adding 'distributed/comm/ucx.py' adding 'distributed/comm/utils.py' adding 'distributed/comm/ws.py' adding 'distributed/comm/tests/test_comms.py' adding 'distributed/comm/tests/test_tcp.py' adding 'distributed/comm/tests/test_ucx.py' adding 'distributed/comm/tests/test_ucx_config.py' adding 'distributed/comm/tests/test_ws.py' adding 'distributed/dashboard/__init__.py' adding 'distributed/dashboard/core.py' adding 'distributed/dashboard/export_tool.py' adding 'distributed/dashboard/scheduler.py' adding 'distributed/dashboard/theme.yaml' adding 'distributed/dashboard/utils.py' adding 'distributed/dashboard/worker.py' adding 'distributed/dashboard/components/__init__.py' adding 'distributed/dashboard/components/nvml.py' adding 'distributed/dashboard/components/rmm.py' adding 'distributed/dashboard/components/scheduler.py' adding 'distributed/dashboard/components/shared.py' adding 'distributed/dashboard/components/worker.py' adding 'distributed/dashboard/templates/__init__.py' adding 'distributed/dashboard/templates/performance_report.html' adding 'distributed/dashboard/tests/test_bokeh.py' adding 'distributed/dashboard/tests/test_components.py' adding 'distributed/dashboard/tests/test_scheduler_bokeh.py' adding 'distributed/dashboard/tests/test_worker_bokeh.py' adding 'distributed/deploy/__init__.py' adding 'distributed/deploy/adaptive.py' adding 'distributed/deploy/adaptive_core.py' adding 'distributed/deploy/cluster.py' adding 'distributed/deploy/local.py' adding 'distributed/deploy/old_ssh.py' adding 'distributed/deploy/spec.py' adding 'distributed/deploy/ssh.py' adding 'distributed/deploy/subprocess.py' adding 'distributed/deploy/utils.py' adding 'distributed/deploy/tests/test_adaptive.py' adding 'distributed/deploy/tests/test_adaptive_core.py' adding 'distributed/deploy/tests/test_cluster.py' adding 'distributed/deploy/tests/test_deploy_utils.py' adding 'distributed/deploy/tests/test_local.py' adding 'distributed/deploy/tests/test_old_ssh.py' adding 'distributed/deploy/tests/test_slow_adaptive.py' adding 'distributed/deploy/tests/test_spec_cluster.py' adding 'distributed/deploy/tests/test_ssh.py' adding 'distributed/deploy/tests/test_subprocess.py' adding 'distributed/diagnostics/__init__.py' adding 'distributed/diagnostics/cluster_dump.py' adding 'distributed/diagnostics/cudf.py' adding 'distributed/diagnostics/eventstream.py' adding 'distributed/diagnostics/graph_layout.py' adding 'distributed/diagnostics/memory_sampler.py' adding 'distributed/diagnostics/memray.py' adding 'distributed/diagnostics/nvml.py' adding 'distributed/diagnostics/plugin.py' adding 'distributed/diagnostics/progress.py' adding 'distributed/diagnostics/progress_stream.py' adding 'distributed/diagnostics/progressbar.py' adding 'distributed/diagnostics/rmm.py' adding 'distributed/diagnostics/task_stream.py' adding 'distributed/diagnostics/websocket.py' adding 'distributed/diagnostics/tests/test_cluster_dump_plugin.py' adding 'distributed/diagnostics/tests/test_cudf_diagnostics.py' adding 'distributed/diagnostics/tests/test_eventstream.py' adding 'distributed/diagnostics/tests/test_graph_layout.py' adding 'distributed/diagnostics/tests/test_install_plugin.py' adding 'distributed/diagnostics/tests/test_memory_sampler.py' adding 'distributed/diagnostics/tests/test_memray.py' adding 'distributed/diagnostics/tests/test_nanny_plugin.py' adding 'distributed/diagnostics/tests/test_nvml.py' adding 'distributed/diagnostics/tests/test_progress.py' adding 'distributed/diagnostics/tests/test_progress_stream.py' adding 'distributed/diagnostics/tests/test_progress_widgets.py' adding 'distributed/diagnostics/tests/test_progressbar.py' adding 'distributed/diagnostics/tests/test_rmm_diagnostics.py' adding 'distributed/diagnostics/tests/test_scheduler_plugin.py' adding 'distributed/diagnostics/tests/test_task_stream.py' adding 'distributed/diagnostics/tests/test_worker_plugin.py' adding 'distributed/http/__init__.py' adding 'distributed/http/health.py' adding 'distributed/http/prometheus.py' adding 'distributed/http/proxy.py' adding 'distributed/http/routing.py' adding 'distributed/http/statics.py' adding 'distributed/http/utils.py' adding 'distributed/http/scheduler/__init__.py' adding 'distributed/http/scheduler/api.py' adding 'distributed/http/scheduler/info.py' adding 'distributed/http/scheduler/json.py' adding 'distributed/http/scheduler/missing_bokeh.py' adding 'distributed/http/scheduler/prometheus/__init__.py' adding 'distributed/http/scheduler/prometheus/core.py' adding 'distributed/http/scheduler/prometheus/semaphore.py' adding 'distributed/http/scheduler/prometheus/stealing.py' adding 'distributed/http/scheduler/tests/test_missing_bokeh.py' adding 'distributed/http/scheduler/tests/test_scheduler_http.py' adding 'distributed/http/scheduler/tests/test_semaphore_http.py' adding 'distributed/http/scheduler/tests/test_stealing_http.py' adding 'distributed/http/static/__init__.py' adding 'distributed/http/static/individual-cluster-map.html' adding 'distributed/http/static/css/__init__.py' adding 'distributed/http/static/css/base.css' adding 'distributed/http/static/css/gpu.css' adding 'distributed/http/static/css/individual-cluster-map.css' adding 'distributed/http/static/css/sortable.min.css' adding 'distributed/http/static/css/status.css' adding 'distributed/http/static/images/__init__.py' adding 'distributed/http/static/images/dask-logo.svg' adding 'distributed/http/static/images/fa-bars.svg' adding 'distributed/http/static/images/favicon.ico' adding 'distributed/http/static/images/jupyter.svg' adding 'distributed/http/static/images/numpy.png' adding 'distributed/http/static/images/pandas.png' adding 'distributed/http/static/images/python.png' adding 'distributed/http/static/js/__init__.py' adding 'distributed/http/static/js/anime.min.js' adding 'distributed/http/static/js/individual-cluster-map.js' adding 'distributed/http/static/js/reconnecting-websocket.min.js' adding 'distributed/http/static/js/sortable.min.js' adding 'distributed/http/templates/__init__.py' adding 'distributed/http/templates/base.html' adding 'distributed/http/templates/call-stack.html' adding 'distributed/http/templates/exceptions.html' adding 'distributed/http/templates/gpu.html' adding 'distributed/http/templates/json-index.html' adding 'distributed/http/templates/logs.html' adding 'distributed/http/templates/main.html' adding 'distributed/http/templates/simple.html' adding 'distributed/http/templates/status.html' adding 'distributed/http/templates/task.html' adding 'distributed/http/templates/worker-table.html' adding 'distributed/http/templates/worker.html' adding 'distributed/http/templates/workers.html' adding 'distributed/http/tests/__init__.py' adding 'distributed/http/tests/test_core.py' adding 'distributed/http/tests/test_routing.py' adding 'distributed/http/worker/__init__.py' adding 'distributed/http/worker/prometheus/__init__.py' adding 'distributed/http/worker/prometheus/core.py' adding 'distributed/http/worker/tests/test_worker_http.py' adding 'distributed/protocol/__init__.py' adding 'distributed/protocol/arrow.py' adding 'distributed/protocol/compression.py' adding 'distributed/protocol/core.py' adding 'distributed/protocol/cuda.py' adding 'distributed/protocol/cupy.py' adding 'distributed/protocol/h5py.py' adding 'distributed/protocol/keras.py' adding 'distributed/protocol/netcdf4.py' adding 'distributed/protocol/numba.py' adding 'distributed/protocol/numpy.py' adding 'distributed/protocol/pickle.py' adding 'distributed/protocol/rmm.py' adding 'distributed/protocol/scipy.py' adding 'distributed/protocol/serialize.py' adding 'distributed/protocol/sparse.py' adding 'distributed/protocol/torch.py' adding 'distributed/protocol/utils.py' adding 'distributed/protocol/utils_test.py' adding 'distributed/protocol/tests/__init__.py' adding 'distributed/protocol/tests/test_arrow.py' adding 'distributed/protocol/tests/test_collection.py' adding 'distributed/protocol/tests/test_collection_cuda.py' adding 'distributed/protocol/tests/test_compression.py' adding 'distributed/protocol/tests/test_cupy.py' adding 'distributed/protocol/tests/test_h5py.py' adding 'distributed/protocol/tests/test_highlevelgraph.py' adding 'distributed/protocol/tests/test_keras.py' adding 'distributed/protocol/tests/test_netcdf4.py' adding 'distributed/protocol/tests/test_numba.py' adding 'distributed/protocol/tests/test_numpy.py' adding 'distributed/protocol/tests/test_pandas.py' adding 'distributed/protocol/tests/test_pickle.py' adding 'distributed/protocol/tests/test_protocol.py' adding 'distributed/protocol/tests/test_protocol_utils.py' adding 'distributed/protocol/tests/test_rmm.py' adding 'distributed/protocol/tests/test_scipy.py' adding 'distributed/protocol/tests/test_serialize.py' adding 'distributed/protocol/tests/test_sparse.py' adding 'distributed/protocol/tests/test_to_pickle.py' adding 'distributed/protocol/tests/test_torch.py' adding 'distributed/protocol/tests/test_utils.py' adding 'distributed/protocol/tests/test_utils_test.py' adding 'distributed/shuffle/__init__.py' adding 'distributed/shuffle/_arrow.py' adding 'distributed/shuffle/_buffer.py' adding 'distributed/shuffle/_comms.py' adding 'distributed/shuffle/_core.py' adding 'distributed/shuffle/_disk.py' adding 'distributed/shuffle/_exceptions.py' adding 'distributed/shuffle/_limiter.py' adding 'distributed/shuffle/_memory.py' adding 'distributed/shuffle/_merge.py' adding 'distributed/shuffle/_pickle.py' adding 'distributed/shuffle/_rechunk.py' adding 'distributed/shuffle/_scheduler_plugin.py' adding 'distributed/shuffle/_shuffle.py' adding 'distributed/shuffle/_worker_plugin.py' adding 'distributed/shuffle/tests/__init__.py' adding 'distributed/shuffle/tests/test_buffer.py' adding 'distributed/shuffle/tests/test_comm_buffer.py' adding 'distributed/shuffle/tests/test_core.py' adding 'distributed/shuffle/tests/test_disk_buffer.py' adding 'distributed/shuffle/tests/test_graph.py' adding 'distributed/shuffle/tests/test_limiter.py' adding 'distributed/shuffle/tests/test_memory_buffer.py' adding 'distributed/shuffle/tests/test_merge.py' adding 'distributed/shuffle/tests/test_merge_column_and_index.py' adding 'distributed/shuffle/tests/test_metrics.py' adding 'distributed/shuffle/tests/test_pickle.py' adding 'distributed/shuffle/tests/test_read_write_lock.py' adding 'distributed/shuffle/tests/test_rechunk.py' adding 'distributed/shuffle/tests/test_shuffle.py' adding 'distributed/shuffle/tests/test_shuffle_plugins.py' adding 'distributed/shuffle/tests/utils.py' adding 'distributed/tests/make_tls_certs.py' adding 'distributed/tests/test_active_memory_manager.py' adding 'distributed/tests/test_actor.py' adding 'distributed/tests/test_as_completed.py' adding 'distributed/tests/test_asyncio.py' adding 'distributed/tests/test_asyncprocess.py' adding 'distributed/tests/test_batched.py' adding 'distributed/tests/test_cancelled_state.py' adding 'distributed/tests/test_chaos.py' adding 'distributed/tests/test_client.py' adding 'distributed/tests/test_client_executor.py' adding 'distributed/tests/test_client_loop.py' adding 'distributed/tests/test_cluster_dump.py' adding 'distributed/tests/test_collections.py' adding 'distributed/tests/test_compatibility.py' adding 'distributed/tests/test_computations.py' adding 'distributed/tests/test_config.py' adding 'distributed/tests/test_core.py' adding 'distributed/tests/test_counter.py' adding 'distributed/tests/test_dask_collections.py' adding 'distributed/tests/test_deadline.py' adding 'distributed/tests/test_diskutils.py' adding 'distributed/tests/test_events.py' adding 'distributed/tests/test_failed_workers.py' adding 'distributed/tests/test_imports.py' adding 'distributed/tests/test_init.py' adding 'distributed/tests/test_itertools.py' adding 'distributed/tests/test_jupyter.py' adding 'distributed/tests/test_locks.py' adding 'distributed/tests/test_metrics.py' adding 'distributed/tests/test_multi_locks.py' adding 'distributed/tests/test_nanny.py' adding 'distributed/tests/test_parse_stdout.py' adding 'distributed/tests/test_preload.py' adding 'distributed/tests/test_priorities.py' adding 'distributed/tests/test_profile.py' adding 'distributed/tests/test_publish.py' adding 'distributed/tests/test_pubsub.py' adding 'distributed/tests/test_queues.py' adding 'distributed/tests/test_reschedule.py' adding 'distributed/tests/test_resources.py' adding 'distributed/tests/test_scheduler.py' adding 'distributed/tests/test_security.py' adding 'distributed/tests/test_semaphore.py' adding 'distributed/tests/test_sizeof.py' adding 'distributed/tests/test_spans.py' adding 'distributed/tests/test_spec.py' adding 'distributed/tests/test_spill.py' adding 'distributed/tests/test_steal.py' adding 'distributed/tests/test_stories.py' adding 'distributed/tests/test_stress.py' adding 'distributed/tests/test_system.py' adding 'distributed/tests/test_system_monitor.py' adding 'distributed/tests/test_threadpoolexecutor.py' adding 'distributed/tests/test_tls_functional.py' adding 'distributed/tests/test_utils.py' adding 'distributed/tests/test_utils_comm.py' adding 'distributed/tests/test_utils_perf.py' adding 'distributed/tests/test_utils_test.py' adding 'distributed/tests/test_variable.py' adding 'distributed/tests/test_versions.py' adding 'distributed/tests/test_worker.py' adding 'distributed/tests/test_worker_client.py' adding 'distributed/tests/test_worker_memory.py' adding 'distributed/tests/test_worker_metrics.py' adding 'distributed/tests/test_worker_state_machine.py' adding 'distributed/tests/tls-ca-cert.pem' adding 'distributed/tests/tls-ca-key.pem' adding 'distributed/tests/tls-cert-chain.pem' adding 'distributed/tests/tls-cert.pem' adding 'distributed/tests/tls-key-cert.pem' adding 'distributed/tests/tls-key.pem' adding 'distributed/tests/tls-self-signed-cert.pem' adding 'distributed/tests/tls-self-signed-key.pem' adding 'distributed/widgets/__init__.py' adding 'distributed/widgets/templates/__init__.py' adding 'distributed/widgets/templates/client.html.j2' adding 'distributed/widgets/templates/cluster.html.j2' adding 'distributed/widgets/templates/computation.html.j2' adding 'distributed/widgets/templates/future.html.j2' adding 'distributed/widgets/templates/has_what.html.j2' adding 'distributed/widgets/templates/local_cluster.html.j2' adding 'distributed/widgets/templates/log.html.j2' adding 'distributed/widgets/templates/logs.html.j2' adding 'distributed/widgets/templates/process_interface.html.j2' adding 'distributed/widgets/templates/scheduler.html.j2' adding 'distributed/widgets/templates/scheduler_info.html.j2' adding 'distributed/widgets/templates/security.html.j2' adding 'distributed/widgets/templates/task_state.html.j2' adding 'distributed/widgets/templates/who_has.html.j2' adding 'distributed/widgets/templates/worker_state.html.j2' adding 'distributed-2024.5.2.dist-info/AUTHORS.md' adding 'distributed-2024.5.2.dist-info/LICENSE.txt' adding 'distributed-2024.5.2.dist-info/METADATA' adding 'distributed-2024.5.2.dist-info/WHEEL' adding 'distributed-2024.5.2.dist-info/entry_points.txt' adding 'distributed-2024.5.2.dist-info/top_level.txt' adding 'distributed-2024.5.2.dist-info/RECORD' removing build/bdist.linux-x86_64/wheel Successfully built distributed-2024.5.2-py3-none-any.whl I: pybuild plugin_pyproject:144: Unpacking wheel built for python3.12 with "installer" module I: pybuild pybuild:334: cp -v distributed/tests/testegg-*-py*.egg distributed/tests/mytest.pyz /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/tests 'distributed/tests/testegg-1.0.0-py3.4.egg' -> '/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/tests/testegg-1.0.0-py3.4.egg' 'distributed/tests/mytest.pyz' -> '/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/tests/mytest.pyz' chmod a+x distributed/tests/test_utils_test.py PYTHONPATH=. python3 -m sphinx -N -bhtml docs/source build/html # HTML generator Running Sphinx v7.4.7 loading translations [en]... done making output directory... done Converting `source_suffix = '.rst'` to `source_suffix = {'.rst': 'restructuredtext'}`. [autosummary] generating autosummary for: active_memory_manager.rst, actors.rst, api.rst, asynchronous.rst, changelog.rst, client.rst, communications.rst, develop.rst, diagnosing-performance.rst, efficiency.rst, ..., scheduling-policies.rst, scheduling-state.rst, serialization.rst, spans.rst, task-launch.rst, tls.rst, work-stealing.rst, worker-memory.rst, worker-state.rst, worker.rst loading intersphinx inventory 'python' from /usr/share/doc/python3/html/objects.inv... loading intersphinx inventory 'numpy' from /usr/share/doc/python-numpy/html/objects.inv... loading intersphinx inventory 'pandas' from /usr/share/doc/python-pandas-doc/html/objects.inv... loading intersphinx inventory 'dask' from /usr/share/doc/python-dask-doc/html/objects.inv... loading intersphinx inventory 'fsspec' from /usr/share/doc/python-fsspec-doc/html/objects.inv... building [mo]: targets for 0 po files that are out of date writing output... building [html]: targets for 45 source files that are out of date updating environment: [new config] 45 added, 0 changed, 0 removed reading sources... [ 2%] active_memory_manager reading sources... [ 4%] actors reading sources... [ 7%] api reading sources... [ 9%] asynchronous reading sources... [ 11%] changelog reading sources... [ 13%] client reading sources... [ 16%] communications reading sources... [ 18%] develop reading sources... [ 20%] diagnosing-performance reading sources... [ 22%] efficiency reading sources... [ 24%] examples-overview reading sources... [ 27%] examples/word-count reading sources... [ 29%] faq reading sources... [ 31%] fine-performance-metrics reading sources... [ 33%] foundations reading sources... [ 36%] http_services reading sources... [ 38%] index reading sources... [ 40%] install reading sources... [ 42%] journey reading sources... [ 44%] killed reading sources... [ 47%] limitations reading sources... [ 49%] locality reading sources... [ 51%] logging reading sources... [ 53%] manage-computation reading sources... [ 56%] memory reading sources... [ 58%] plugins reading sources... [ 60%] priority reading sources... [ 62%] prometheus reading sources... [ 64%] protocol reading sources... [ 67%] publish reading sources... [ 69%] queues reading sources... [ 71%] quickstart reading sources... [ 73%] related-work reading sources... [ 76%] resilience reading sources... [ 78%] resources reading sources... [ 80%] scheduling-policies reading sources... [ 82%] scheduling-state reading sources... [ 84%] serialization reading sources... [ 87%] spans reading sources... [ 89%] task-launch reading sources... [ 91%] tls reading sources... [ 93%] work-stealing reading sources... [ 96%] worker reading sources... [ 98%] worker-memory reading sources... [100%] worker-state WARNING: Summarised items should not include the current module. Replace 'distributed.diagnostics.progressbar.progress' with 'diagnostics.progressbar.progress'. WARNING: Summarised items should not include the current module. Replace 'distributed.utils.Log' with 'utils.Log'. WARNING: Summarised items should not include the current module. Replace 'distributed.utils.Logs' with 'utils.Logs'. looking for now-outdated files... none found pickling environment... done checking consistency... done preparing documents... done copying assets... copying static files... done copying extra files... done copying assets: done writing output... [ 2%] active_memory_manager writing output... [ 4%] actors writing output... [ 7%] api writing output... [ 9%] asynchronous writing output... [ 11%] changelog writing output... [ 13%] client writing output... [ 16%] communications writing output... [ 18%] develop writing output... [ 20%] diagnosing-performance writing output... [ 22%] efficiency writing output... [ 24%] examples-overview writing output... [ 27%] examples/word-count writing output... [ 29%] faq writing output... [ 31%] fine-performance-metrics writing output... [ 33%] foundations writing output... [ 36%] http_services writing output... [ 38%] index writing output... [ 40%] install writing output... [ 42%] journey writing output... [ 44%] killed writing output... [ 47%] limitations writing output... [ 49%] locality writing output... [ 51%] logging writing output... [ 53%] manage-computation writing output... [ 56%] memory writing output... [ 58%] plugins writing output... [ 60%] priority writing output... [ 62%] prometheus writing output... [ 64%] protocol writing output... [ 67%] publish writing output... [ 69%] queues writing output... [ 71%] quickstart writing output... [ 73%] related-work writing output... [ 76%] resilience writing output... [ 78%] resources writing output... [ 80%] scheduling-policies writing output... [ 82%] scheduling-state writing output... [ 84%] serialization writing output... [ 87%] spans writing output... [ 89%] task-launch writing output... [ 91%] tls writing output... [ 93%] work-stealing writing output... [ 96%] worker writing output... [ 98%] worker-memory writing output... [100%] worker-state /build/reproducible-path/dask.distributed-2024.5.2+ds.1/docs/source/active_memory_manager.rst:31: WARNING: unknown document: 'configuration' /build/reproducible-path/dask.distributed-2024.5.2+ds.1/docs/source/scheduling-policies.rst:270: WARNING: unknown document: 'configuration' /build/reproducible-path/dask.distributed-2024.5.2+ds.1/docs/source/tls.rst:55: WARNING: unknown document: 'configuration' generating indices... genindex done highlighting module code... [ 3%] distributed.active_memory_manager highlighting module code... [ 6%] distributed.client highlighting module code... [ 10%] distributed.comm.addressing highlighting module code... [ 13%] distributed.comm.core highlighting module code... [ 16%] distributed.comm.registry highlighting module code... [ 19%] distributed.core highlighting module code... [ 23%] distributed.deploy.adaptive highlighting module code... [ 26%] distributed.deploy.local highlighting module code... [ 29%] distributed.deploy.spec highlighting module code... [ 32%] distributed.diagnostics.memory_sampler highlighting module code... [ 35%] distributed.diagnostics.plugin highlighting module code... [ 39%] distributed.diagnostics.progressbar highlighting module code... [ 42%] distributed.event highlighting module code... [ 45%] distributed.exceptions highlighting module code... [ 48%] distributed.lock highlighting module code... [ 52%] distributed.multi_lock highlighting module code... [ 55%] distributed.nanny highlighting module code... [ 58%] distributed.protocol.serialize highlighting module code... [ 61%] distributed.queues highlighting module code... [ 65%] distributed.recreate_tasks highlighting module code... [ 68%] distributed.scheduler highlighting module code... [ 71%] distributed.security highlighting module code... [ 74%] distributed.semaphore highlighting module code... [ 77%] distributed.spans highlighting module code... [ 81%] distributed.spill highlighting module code... [ 84%] distributed.threadpoolexecutor highlighting module code... [ 87%] distributed.utils highlighting module code... [ 90%] distributed.variable highlighting module code... [ 94%] distributed.worker highlighting module code... [ 97%] distributed.worker_client highlighting module code... [100%] distributed.worker_state_machine writing additional pages... search done copying images... [ 6%] ../../debian/bokeh-task-stream.gif copying images... [ 12%] ../../debian/daskboard-profile.gif copying images... [ 19%] images/memory-sampler.svg copying images... [ 25%] images/fine-performance-metrics/seconds-full.png copying images... [ 31%] images/fine-performance-metrics/seconds-IO.png copying images... [ 38%] images/fine-performance-metrics/seconds-not-IO.png copying images... [ 44%] images/fine-performance-metrics/bytes.png copying images... [ 50%] images/task-state.svg copying images... [ 56%] images/fine-performance-metrics/spans.png copying images... [ 62%] images/worker-dep-state.svg copying images... [ 69%] images/worker-execute-state.svg copying images... [ 75%] images/worker-scatter-state.svg copying images... [ 81%] images/worker-forget-state.svg copying images... [ 88%] images/worker-cancel-state1.svg copying images... [ 94%] images/worker-cancel-state2.svg copying images... [100%] images/worker-state-machine.svg dumping search index in English (code: en)... done dumping object inventory... done /build/reproducible-path/dask.distributed-2024.5.2+ds.1/docs/source/conf.py:435: RemovedInSphinx80Warning: Sphinx 8 will drop support for representing paths as strings. Use "pathlib.Path" or "os.fspath" instead. target_path = app.outdir + "/" + html_src_path build succeeded, 6 warnings. The HTML pages are in build/html. rm -rf build/html/_static/__pycache__ make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_auto_test -O--buildsystem=pybuild I: pybuild pybuild:308: mkdir -pv /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/.config && printf '\173 "dataframe": \173 "query-planning": False \175 \175\n' > /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/.config/dask mkdir: created directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/.config' I: pybuild base:311: cd /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build; cd /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build; /build/reproducible-path/dask.distributed-2024.5.2+ds.1/debian/run-tests /build/reproducible-path/dask.distributed-2024.5.2+ds.1/debian/get-test-exclusions rules python3.12 -m pytest --pyargs distributed --verbose --color=no --timeout-method=signal --timeout=300 -m "not avoid_ci" *** STARTING RUN 1: python3.12 -m pytest --pyargs distributed --verbose --color=no --timeout-method=signal --timeout=300 -m not avoid_ci -rsfE --ignore=distributed/comm/tests/test_comms.py --ignore=distributed/comm/tests/test_ws.py --ignore=distributed/deploy/tests/test_adaptive.py --ignore=distributed/deploy/tests/test_local.py --ignore=distributed/deploy/tests/test_slow_adaptive.py --ignore=distributed/deploy/tests/test_spec_cluster.py --deselect=distributed/cli/tests/test_dask_scheduler.py::test_no_dashboard --deselect=distributed/deploy/tests/test_local.py::test_localcluster_get_client --deselect=distributed/deploy/tests/test_old_ssh.py::test_cluster --deselect=distributed/deploy/tests/test_old_ssh.py::test_old_ssh_nprocs_renamed_to_n_workers --deselect=distributed/deploy/tests/test_old_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/deploy/tests/test_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/http/tests/test_core.py::test_prometheus_api_doc --deselect=distributed/tests/test_init.py::test_git_revision --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout_returned --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_jupyter.py::test_shutsdown_cleanly --deselect=distributed/tests/test_profile.py::test_stack_overflow --deselect=distributed/tests/test_pubsub.py::test_client_worker --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_spill.py::test_spillbuffer_oserror --deselect=distributed/tests/test_steal.py::test_steal_twice --deselect=distributed/tests/test_utils_test.py::test_cluster --deselect=distributed/tests/test_variable.py::test_variable_in_task --deselect=distributed/tests/test_worker.py::test_process_executor_kills_process --deselect=distributed/tests/test_worker_memory.py::test_fail_to_pickle_execute_1 --deselect=distributed/tests/test_worker_state_machine.py::test_task_state_instance_are_garbage_collected --deselect=distributed/protocol/tests/test_protocol.py::test_deeply_nested_structures --deselect=distributed/protocol/tests/test_serialize.py::test_deeply_nested_structures --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_spec.py::test_errors --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/dashboard/tests/test_scheduler_bokeh.py::test_counters --deselect=distributed/dashboard/tests/test_worker_bokeh.py::test_counters --deselect=distributed/deploy/tests/test_local.py::test_adapt_then_manual --deselect=distributed/deploy/tests/test_local.py::test_async_with --deselect=distributed/deploy/tests/test_local.py::test_close_twice --deselect=distributed/deploy/tests/test_local.py::test_cluster_info_sync --deselect=distributed/deploy/tests/test_local.py::test_local_tls --deselect=distributed/deploy/tests/test_local.py::test_no_dangling_asyncio_tasks --deselect=distributed/deploy/tests/test_local.py::test_only_local_access --deselect=distributed/deploy/tests/test_local.py::test_remote_access --deselect=distributed/diagnostics/tests/test_progress_widgets.py::test_serializers --deselect=distributed/diagnostics/tests/test_scheduler_plugin.py::test_lifecycle --deselect=distributed/http/scheduler/tests/test_missing_bokeh.py::test_missing_bokeh --deselect=distributed/http/scheduler/tests/test_scheduler_http.py::test_metrics_when_prometheus_client_not_installed --deselect=distributed/protocol/tests/test_serialize.py::test_errors --deselect=distributed/tests/test_batched.py::test_BatchedSend --deselect=distributed/tests/test_batched.py::test_close_closed --deselect=distributed/tests/test_batched.py::test_close_twice --deselect=distributed/tests/test_batched.py::test_send_after_stream_start --deselect=distributed/tests/test_batched.py::test_send_before_close --deselect=distributed/tests/test_batched.py::test_send_before_start --deselect=distributed/tests/test_batched.py::test_sending_traffic_jam --deselect=distributed/tests/test_batched.py::test_serializers --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_async_with --deselect=distributed/tests/test_client.py::test_client_is_quiet_cluster_close --deselect=distributed/tests/test_client.py::test_dashboard_link_cluster --deselect=distributed/tests/test_client.py::test_dashboard_link_inproc --deselect=distributed/tests/test_client.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_client.py::test_mixing_clients_different_scheduler --deselect=distributed/tests/test_client.py::test_quiet_client_close --deselect=distributed/tests/test_client.py::test_rebalance_sync --deselect=distributed/tests/test_client.py::test_repr_localcluster --deselect=distributed/tests/test_client.py::test_security_loader --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_shutdown --deselect=distributed/tests/test_client.py::test_shutdown_is_quiet_with_cluster --deselect=distributed/tests/test_client.py::test_shutdown_localcluster --deselect=distributed/tests/test_client.py::test_shutdown_stops_callbacks --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_start_new_loop --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_use_running_loop --deselect=distributed/tests/test_core.py::test_close_fast_without_active_handlers --deselect=distributed/tests/test_core.py::test_close_grace_period_for_handlers --deselect=distributed/tests/test_core.py::test_close_properly --deselect=distributed/tests/test_core.py::test_compression --deselect=distributed/tests/test_core.py::test_connection_pool --deselect=distributed/tests/test_core.py::test_connection_pool_close_while_connecting --deselect=distributed/tests/test_core.py::test_connection_pool_detects_remote_close --deselect=distributed/tests/test_core.py::test_connection_pool_outside_cancellation --deselect=distributed/tests/test_core.py::test_connection_pool_remove --deselect=distributed/tests/test_core.py::test_connection_pool_respects_limit --deselect=distributed/tests/test_core.py::test_connection_pool_tls --deselect=distributed/tests/test_core.py::test_counters --deselect=distributed/tests/test_core.py::test_deserialize_error --deselect=distributed/tests/test_core.py::test_errors --deselect=distributed/tests/test_core.py::test_identity_inproc --deselect=distributed/tests/test_core.py::test_identity_tcp --deselect=distributed/tests/test_core.py::test_large_packets_inproc --deselect=distributed/tests/test_core.py::test_messages_are_ordered_bsend --deselect=distributed/tests/test_core.py::test_messages_are_ordered_raw --deselect=distributed/tests/test_core.py::test_ports --deselect=distributed/tests/test_core.py::test_rpc_default --deselect=distributed/tests/test_core.py::test_rpc_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_default --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_tcp --deselect=distributed/tests/test_core.py::test_rpc_serialization --deselect=distributed/tests/test_core.py::test_rpc_tcp --deselect=distributed/tests/test_core.py::test_rpc_tls --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_inproc --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_tcp --deselect=distributed/tests/test_core.py::test_send_recv_args --deselect=distributed/tests/test_core.py::test_send_recv_cancelled --deselect=distributed/tests/test_core.py::test_server --deselect=distributed/tests/test_core.py::test_server_comms_mark_active_handlers --deselect=distributed/tests/test_core.py::test_server_raises_on_blocked_handlers --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_locks.py::test_errors --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_worker_uses_same_host_as_nanny --deselect=distributed/tests/test_preload.py::test_failure_doesnt_crash_scheduler --deselect=distributed/tests/test_preload.py::test_preload_import_time --deselect=distributed/tests/test_preload.py::test_preload_manager_sequence --deselect=distributed/tests/test_preload.py::test_worker_preload_text --deselect=distributed/tests/test_scheduler.py::test_allowed_failures_config --deselect=distributed/tests/test_scheduler.py::test_async_context_manager --deselect=distributed/tests/test_scheduler.py::test_dashboard_host --deselect=distributed/tests/test_scheduler.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_scheduler.py::test_finished --deselect=distributed/tests/test_scheduler.py::test_multiple_listeners --deselect=distributed/tests/test_scheduler.py::test_no_dangling_asyncio_tasks --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_security.py::test_require_encryption --deselect=distributed/tests/test_security.py::test_tls_listen_connect --deselect=distributed/tests/test_security.py::test_tls_temporary_credentials_functional --deselect=distributed/tests/test_semaphore.py::test_threadpoolworkers_pick_correct_ioloop --deselect=distributed/tests/test_tls_functional.py::test_security_dict_input_no_security --deselect=distributed/tests/test_utils_test.py::test_ensure_no_new_clients --deselect=distributed/tests/test_utils_test.py::test_freeze_batched_send --deselect=distributed/tests/test_utils_test.py::test_locked_comm_drop_in_replacement --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_read --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_write --deselect=distributed/tests/test_worker.py::test_host_uses_scheduler_protocol --deselect=distributed/tests/test_worker.py::test_plugin_exception --deselect=distributed/tests/test_worker.py::test_plugin_internal_exception --deselect=distributed/tests/test_worker.py::test_plugin_multiple_exceptions --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker_client.py::test_dont_override_default_get --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_allowlist --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_protocols --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers_2 --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command_default --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_config --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_file --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_remote_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_scheduler_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_contact_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_error_during_startup --deselect=distributed/cli/tests/test_dask_worker.py::test_integer_names --deselect=distributed/cli/tests/test_dask_worker.py::test_listen_address_ipv6 --deselect=distributed/cli/tests/test_dask_worker.py::test_local_directory --deselect=distributed/cli/tests/test_dask_worker.py::test_memory_limit --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range_too_many_workers_raises --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_no_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_auto --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_expands_name --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_negative --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_requires_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_preload_config --deselect=distributed/cli/tests/test_dask_worker.py::test_resources --deselect=distributed/cli/tests/test_dask_worker.py::test_respect_host_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_address_env --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_restart_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_stagger_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_signal_handling --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_works --deselect=distributed/cli/tests/test_dask_worker.py::test_timeout --deselect=distributed/cli/tests/test_dask_worker.py::test_worker_class --deselect=distributed/tests/test_config.py::test_logging_extended --deselect=distributed/tests/test_config.py::test_logging_file_config --deselect=distributed/tests/test_config.py::test_logging_mutual_exclusive --deselect=distributed/tests/test_config.py::test_logging_simple --deselect=distributed/tests/test_config.py::test_logging_simple_under_distributed --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_variable.py::test_variable_in_task ============================= test session starts ============================== platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /usr/bin/python3.12 cachedir: .pytest_cache rootdir: /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build configfile: pyproject.toml plugins: rerunfailures-12.0, typeguard-4.3.0, timeout-2.3.1, anyio-4.4.0 timeout: 300.0s timeout method: signal timeout func_only: False collecting ... collected 3349 items / 221 deselected / 30 skipped / 3128 selected distributed/cli/tests/test_dask_scheduler.py::test_interface PASSED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_pid_file PASSED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_version_option PASSED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_idle_timeout SKIPPED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_restores_signal_handler SKIPPED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_signal_handling[2] SKIPPED [ 0%] distributed/cli/tests/test_dask_scheduler.py::test_signal_handling[15] SKIPPED [ 0%] distributed/cli/tests/test_dask_spec.py::test_text PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_file PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_worker[2-Nanny] PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_worker[2-Worker] PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_worker[15-Nanny] PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_worker[15-Worker] PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_scheduler[2] PASSED [ 0%] distributed/cli/tests/test_dask_spec.py::test_signal_handling_scheduler[15] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args0-expect0] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args1-expect1] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args2-expect2] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args3-expect3] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args4-expect4] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args5-expect5] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args6-expect6] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args7-expect7] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args8-expect8] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args9-expect9] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args10-expect10] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args11-expect11] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args12-expect12] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args13-expect13] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args14-expect14] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args15-expect15] PASSED [ 0%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args16-expect16] PASSED [ 1%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports[args17-expect17] PASSED [ 1%] distributed/cli/tests/test_dask_worker.py::test_apportion_ports_bad PASSED [ 1%] distributed/cli/tests/test_dask_worker.py::test_version_option PASSED [ 1%] distributed/cli/tests/test_dask_worker.py::test_worker_timeout[True] SKIPPED [ 1%] distributed/cli/tests/test_dask_worker.py::test_worker_timeout[False] SKIPPED [ 1%] distributed/cli/tests/test_tls_cli.py::test_basic PASSED [ 1%] distributed/cli/tests/test_tls_cli.py::test_sni PASSED [ 1%] distributed/cli/tests/test_tls_cli.py::test_nanny PASSED [ 1%] distributed/cli/tests/test_tls_cli.py::test_separate_key_cert PASSED [ 1%] distributed/cli/tests/test_tls_cli.py::test_use_config_file PASSED [ 1%] distributed/comm/tests/test_tcp.py::test_getaddrinfo_invalid_af PASSED [ 1%] distributed/dashboard/tests/test_bokeh.py::test_old_import PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_safe_target PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_scale_up PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_scale_down PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_interval PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_adapt_oserror_safe_target PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_adapt_oserror_scale PASSED [ 1%] distributed/deploy/tests/test_adaptive_core.py::test_adapt_stop_del PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_eq PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_repr PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_cluster_wait_for_worker PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_deprecated_loop_properties PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_exponential_backoff PASSED [ 1%] distributed/deploy/tests/test_cluster.py::test_sync_context_manager_used_with_async_cluster PASSED [ 1%] distributed/deploy/tests/test_deploy_utils.py::test_default_process_thread_breakdown PASSED [ 1%] distributed/deploy/tests/test_subprocess.py::test_basic PASSED [ 1%] distributed/deploy/tests/test_subprocess.py::test_n_workers PASSED [ 1%] distributed/deploy/tests/test_subprocess.py::test_scale_up_and_down PASSED [ 1%] distributed/deploy/tests/test_subprocess.py::test_raise_if_scheduler_fails_to_start PASSED [ 1%] distributed/deploy/tests/test_subprocess.py::test_subprocess_cluster_does_not_depend_on_logging SKIPPED [ 2%] distributed/deploy/tests/test_subprocess.py::test_raise_on_windows SKIPPED [ 2%] distributed/diagnostics/tests/test_cluster_dump_plugin.py::test_cluster_dump_plugin PASSED [ 2%] distributed/diagnostics/tests/test_eventstream.py::test_eventstream SKIPPED [ 2%] distributed/diagnostics/tests/test_eventstream.py::test_eventstream_remote PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_basic PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_construct_after_call PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_states PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_release_tasks PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_forget PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_unique_positions PASSED [ 2%] distributed/diagnostics/tests/test_graph_layout.py::test_layout_scatter PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_pip_install PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_conda_install PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_pip_install_fails PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_conda_install_fails_when_conda_not_found PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_conda_install_fails_when_conda_raises PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_conda_install_fails_on_returncode PASSED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_package_install_restarts_on_nanny SKIPPED [ 2%] distributed/diagnostics/tests/test_install_plugin.py::test_package_install_failing_does_not_restart_on_nanny PASSED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_async PASSED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_sync PASSED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_at_least_one_sample PASSED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_multi_sample SKIPPED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_pandas[False] SKIPPED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_pandas[True] SKIPPED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_pandas_multiseries[False] SKIPPED [ 2%] distributed/diagnostics/tests/test_memory_sampler.py::test_pandas_multiseries[True] SKIPPED [ 2%] distributed/diagnostics/tests/test_nanny_plugin.py::test_register_worker_plugin_is_deprecated PASSED [ 2%] distributed/diagnostics/tests/test_nanny_plugin.py::test_register_worker_plugin_typing_over_nanny_keyword PASSED [ 2%] distributed/diagnostics/tests/test_nanny_plugin.py::test_duck_typed_register_nanny_plugin_is_deprecated PASSED [ 2%] distributed/diagnostics/tests/test_nanny_plugin.py::test_register_idempotent_plugin PASSED [ 3%] distributed/diagnostics/tests/test_nanny_plugin.py::test_register_non_idempotent_plugin PASSED [ 3%] distributed/diagnostics/tests/test_nanny_plugin.py::test_register_plugin_with_idempotent_keyword_is_deprecated PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_many_Progress RERUN [ 3%] distributed/diagnostics/tests/test_progress.py::test_many_Progress PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_multiprogress PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_multiprogress_cancel PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_multiprogress_with_spans PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_multiprogress_with_prefix PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_multiprogress_warns PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_robust_to_bad_plugin PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_AllProgress PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_AllProgress_lost_key PASSED [ 3%] distributed/diagnostics/tests/test_progress.py::test_group_timing PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_text_progressbar PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_TextProgressBar_error PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_TextProgressBar_empty PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_progress_function PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_progress_function_w_kwargs PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_progress_function_warns PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_progress_function_raises PASSED [ 3%] distributed/diagnostics/tests/test_progressbar.py::test_deprecated_loop_properties PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_simple PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_add_remove_worker PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_remove_worker_renamed_kwargs_allowed PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_remove_worker_without_kwargs_deprecated PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_async_add_remove_worker PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_async_and_sync_add_remove_worker PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_failing_async_add_remove_worker PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_failing_sync_add_remove_worker PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_plugin PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_scheduler_plugin_deprecated PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_unregister_scheduler_plugin PASSED [ 3%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_unregister_scheduler_plugin_from_client PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_log_event_plugin PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_plugin_on_scheduler PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_closing_errors_ok PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_update_graph_hook_simple PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_update_graph_hook_complex PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_scheduler_plugin_in_register_worker_plugin_overrides PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_scheduler_plugin_in_register_worker_plugin_overrides_nanny PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_idempotent_plugin PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_non_idempotent_plugin PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_plugin_with_idempotent_keyword_is_deprecated PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_idempotent_plugins_directly PASSED [ 4%] distributed/diagnostics/tests/test_scheduler_plugin.py::test_register_non_idempotent_plugins_directly PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_TaskStreamPlugin PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_maxlen PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_collect PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_no_startstops PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_client PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_client_sync PASSED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_get_task_stream_plot SKIPPED [ 4%] distributed/diagnostics/tests/test_task_stream.py::test_get_task_stream_save SKIPPED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_create_with_client PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_remove_with_client PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_remove_with_client_raises PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_create_on_construction PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_normal_task_transitions_called PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_failing_task_transitions_called PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_superseding_task_transitions_called PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_dependent_tasks PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_empty_plugin PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_default_name PASSED [ 4%] distributed/diagnostics/tests/test_worker_plugin.py::test_assert_no_warning_no_overload PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_WorkerPlugin_overwrite PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_register_worker_plugin_is_deprecated PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_register_worker_plugin_typing_over_nanny_keyword PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_duck_typed_register_worker_plugin_is_deprecated PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_register_idempotent_plugin PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_register_non_idempotent_plugin PASSED [ 5%] distributed/diagnostics/tests/test_worker_plugin.py::test_register_plugin_with_idempotent_keyword_is_deprecated PASSED [ 5%] distributed/http/scheduler/tests/test_missing_bokeh.py::test_bokeh_version_too_low SKIPPED [ 5%] distributed/http/scheduler/tests/test_missing_bokeh.py::test_bokeh_version_too_high SKIPPED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_connect PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_worker_404 PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_prefix SKIPPED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_prometheus PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_prometheus_collect_task_states PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_prometheus_collect_task_prefix_counts PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_prometheus_collect_worker_states PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_health SKIPPED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_sitemap PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[1_0] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[1_1] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[x] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[a+b] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[a b] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key5] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key6] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key7] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key8] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[(] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[[] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page['] PASSED [ 5%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page["] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[123] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key14] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key15] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key16] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key17] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_task_page[key18] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[1_0] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[1_1] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[x] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[a+b] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[a b] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key5] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key6] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key7] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key8] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[(] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[[] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page['] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page["] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[123] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key14] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key15] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key16] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key17] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_call_stack_page[key18] PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_allow_websocket_origin SKIPPED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_eventstream PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_api_disabled_by_default PASSED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_api SKIPPED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_retire_workers SKIPPED [ 6%] distributed/http/scheduler/tests/test_scheduler_http.py::test_get_workers SKIPPED [ 7%] distributed/http/scheduler/tests/test_scheduler_http.py::test_adaptive_target SKIPPED [ 7%] distributed/http/scheduler/tests/test_scheduler_http.py::test_check_idle SKIPPED [ 7%] distributed/http/scheduler/tests/test_semaphore_http.py::test_prometheus PASSED [ 7%] distributed/http/scheduler/tests/test_semaphore_http.py::test_prometheus_without_semaphore_extension PASSED [ 7%] distributed/http/scheduler/tests/test_stealing_http.py::test_prometheus PASSED [ 7%] distributed/http/scheduler/tests/test_stealing_http.py::test_prometheus_collect_count_total_by_cost_multipliers PASSED [ 7%] distributed/http/scheduler/tests/test_stealing_http.py::test_prometheus_collect_cost_total_by_cost_multipliers PASSED [ 7%] distributed/http/scheduler/tests/test_stealing_http.py::test_prometheus_without_stealing_extension PASSED [ 7%] distributed/http/tests/test_core.py::test_scheduler PASSED [ 7%] distributed/http/tests/test_routing.py::test_basic PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_prometheus PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_metrics_when_prometheus_client_not_installed PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_prometheus_collect_task_states PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_health SKIPPED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_sitemap SKIPPED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_prometheus_collect_memory_metrics PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_prometheus_collect_memory_metrics_spill_disabled PASSED [ 7%] distributed/http/worker/tests/test_worker_http.py::test_prometheus_collect_memory_metrics_bogus_sizeof SKIPPED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y0-dask-tuple] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y0-dask-dict] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y0-dask-list] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y1-pickle-tuple] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y1-pickle-dict] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[y1-pickle-list] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[None-pickle-tuple] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[None-pickle-dict] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_serialize_collection[None-pickle-list] PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_large_collections_serialize_simply PASSED [ 7%] distributed/protocol/tests/test_collection.py::test_nested_types PASSED [ 7%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_cupy[50-cuda-tuple] SKIPPED [ 7%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_cupy[50-cuda-dict] SKIPPED [ 7%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_cupy[None-pickle-tuple] SKIPPED [ 8%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_cupy[None-pickle-dict] SKIPPED [ 8%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_pandas_pandas[df20-cuda-tuple] SKIPPED [ 8%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_pandas_pandas[df20-cuda-dict] SKIPPED [ 8%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_pandas_pandas[None-pickle-tuple] SKIPPED [ 8%] distributed/protocol/tests/test_collection_cuda.py::test_serialize_pandas_pandas[None-pickle-dict] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_get_compression_settings PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_auto_compression PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_compression_1 PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_compression_2 PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_compression_3 PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_compression_without_deserialization PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_lz4_decompression_avoids_deep_copy SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[None-bytes] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[None-memoryview] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[zlib-bytes] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[zlib-memoryview] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[lz4-bytes] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[lz4-memoryview] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[snappy-bytes] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[snappy-memoryview] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[zstd-bytes] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress[zstd-memoryview] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_sample[None] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_sample[zlib] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_sample[lz4] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_sample[snappy] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_sample[zstd] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_memoryviews[None] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_memoryviews[zlib] PASSED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_memoryviews[lz4] SKIPPED [ 8%] distributed/protocol/tests/test_compression.py::test_maybe_compress_memoryviews[snappy] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_maybe_compress_memoryviews[zstd] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_maybe_compress_auto PASSED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[None-bytes] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[None-memoryview] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[zlib-bytes] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[zlib-memoryview] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[lz4-bytes] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[lz4-memoryview] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[snappy-bytes] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[snappy-memoryview] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[zstd-bytes] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_compression_thread_safety[zstd-memoryview] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_large_messages[None] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_large_messages[zlib] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_large_messages[lz4] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_large_messages[snappy] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_large_messages[zstd] SKIPPED [ 9%] distributed/protocol/tests/test_compression.py::test_loads_without_deserialization_avoids_compression PASSED [ 9%] distributed/protocol/tests/test_compression.py::test_compress_remote_comms PASSED [ 9%] distributed/protocol/tests/test_compression.py::test_disable_compression_on_localhost[None] PASSED [ 9%] distributed/protocol/tests/test_compression.py::test_disable_compression_on_localhost[127.0.0.1] PASSED [ 9%] distributed/protocol/tests/test_compression.py::test_disable_compression_on_localhost[localhost] PASSED [ 9%] distributed/protocol/tests/test_highlevelgraph.py::test_combo_of_layer_types PASSED [ 9%] distributed/protocol/tests/test_highlevelgraph.py::test_blockwise PASSED [ 9%] distributed/protocol/tests/test_highlevelgraph.py::test_shuffle PASSED [ 9%] distributed/protocol/tests/test_highlevelgraph.py::test_array_annotations PASSED [ 9%] distributed/protocol/tests/test_highlevelgraph.py::test_dataframe_annotations PASSED [ 9%] distributed/protocol/tests/test_numpy.py::test_serialize PASSED [ 9%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x0] PASSED [ 9%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x1] PASSED [ 9%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x2] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x3] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x4] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x5] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x6] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x7] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x8] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x9] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x10] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x11] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x12] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x13] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x14] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x15] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x16] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x17] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x18] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x19] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x20] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x21] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x22] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x23] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x24] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x25] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x26] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x27] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x28] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x29] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x30] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x31] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x32] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x33] PASSED [ 10%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x34] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x35] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy[x36] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_numpy_writable[True] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_numpy_writable[False] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x0] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x1] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x2] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x3] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x4] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked_array[x5] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_serialize_numpy_ma_masked PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy_custom_dtype SKIPPEDre.test_rational') [ 11%] distributed/protocol/tests/test_numpy.py::test_memmap PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_serialize_numpy_large SKIPPED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[f8-8] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[i4-4] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[c16-16] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[b-1] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[S3-3] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[M8[us]-8] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[M8[s]-8] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[U3-12] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[dt8-12] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[dt9-4] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[dt10-8] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[dt11-88] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_itemsize[dt12-8] PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_compress_numpy PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_compress_memoryview PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_dumps_large PASSED [ 11%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[True-x0] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[True-x1] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[True-x2] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[True-x3] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[False-x0] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[False-x1] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[False-x2] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_zero_strided_numpy_array[False-x3] PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_non_zero_strided_array PASSED [ 12%] distributed/protocol/tests/test_numpy.py::test_serialize_writeable_array_readonly_base_object PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df0] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df1] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df2] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df3] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df4] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df5] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df6] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df7] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df8] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df9] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df10] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df11] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df12] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df13] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df14] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df15] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df16] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df17] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df18] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df19] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df20] PASSED [ 12%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df21] PASSED [ 13%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df22] PASSED [ 13%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df23] PASSED [ 13%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df24] PASSED [ 13%] distributed/protocol/tests/test_pandas.py::test_dumps_serialize_pandas[df25] PASSED [ 13%] distributed/protocol/tests/test_pandas.py::test_dumps_pandas_writable PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_data[4] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_data[5] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_out_of_band[4] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_out_of_band[5] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_empty[4] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_empty[5] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_numpy[4] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_numpy[5] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_functions[4] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_functions[5] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_by_value_when_registered PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_allow_pickle_if_registered_in_dask_serialize PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_nopickle_nested PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_functions_in_main SKIPPED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_zero_copy_read_only_flag[dask] PASSED [ 13%] distributed/protocol/tests/test_pickle.py::test_pickle_zero_copy_read_only_flag[pickle] PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_protocol PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_small PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_small_and_big PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_large_bytes[bytes] PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_large_bytes[memoryview] PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_large_messages_map PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_loads_deserialize_False PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_dumps_loads_Serialize PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_dumps_loads_Serialized PASSED [ 13%] distributed/protocol/tests/test_protocol.py::test_preserve_header[serializers0] PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_preserve_header[serializers1] PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_sizeof_serialize[Serialize-Serialized0] PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_sizeof_serialize[Serialize-Serialized1] PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_sizeof_serialize[ToPickle-Pickled] PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_fallback_to_pickle PASSED [ 14%] distributed/protocol/tests/test_protocol.py::test_large_payload[bytes] SKIPPED [ 14%] distributed/protocol/tests/test_protocol.py::test_large_payload[str] SKIPPED [ 14%] distributed/protocol/tests/test_protocol.py::test_large_payload[ext] SKIPPED [ 14%] distributed/protocol/tests/test_protocol_utils.py::test_pack_frames PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::test_unpack_frames_remainder[456] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::test_unpack_frames_remainder[] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::test_unpack_frames_partial PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_empty PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_one PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices0] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices1] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices2] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices3] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices4] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices5] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices6] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices7] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices8] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_parts[slices9] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_readonly_buffer PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_non_memoryview PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_gaps[slices0] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_gaps[slices1] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_gaps[slices2] PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_different_buffer PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_different_non_contiguous PASSED [ 14%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_multidimensional PASSED [ 15%] distributed/protocol/tests/test_protocol_utils.py::TestMergeMemroyviews::test_catch_different_formats PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_dumps_serialize PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_bytestrings PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_empty_array PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[b] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[B] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[h] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[H] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[i] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[I] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[l] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[L] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[q] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[Q] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[f] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_arrays[d] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_Serialize PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_Serialized PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_nested_deserialize PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_iterate_collection PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_object_in_graph PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_scatter PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_inter_worker_comms PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_empty PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_empty_loads PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_empty_loads_deep PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_bytes[kwargs0] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_bytes[kwargs1] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_bytes_numpy[kwargs0] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_serialize_bytes_numpy[kwargs1] PASSED [ 15%] distributed/protocol/tests/test_serialize.py::test_deserialize_bytes_zero_copy_read_only PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_deserialize_bytes_zero_copy_writeable PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_list_compress PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_malicious_exception PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_err_on_bad_deserializer PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_context_specific_serialization PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_context_specific_serialization_class PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_raises PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_different_compression_families PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_frame_split PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data0-False] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data1-False] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data2-False] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data3-False] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data4-False] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data5-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data6-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data7-True] XFAIL [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data8-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data9-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data10-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data11-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_check_dask_serializable[data12-True] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_lists[serializers0] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_lists[serializers1] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_lists[serializers2] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_serialize_lists[serializers3] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_deser_memoryview[data_in0] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_deser_memoryview[data_in1] PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_ser_memoryview_object PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_ser_empty_1d_memoryview PASSED [ 16%] distributed/protocol/tests/test_serialize.py::test_ser_empty_nd_memoryview PASSED [ 17%] distributed/protocol/tests/test_serialize.py::test_large_pickled_object PASSED [ 17%] distributed/protocol/tests/test_serialize.py::test__is_msgpack_serializable PASSED [ 17%] distributed/protocol/tests/test_to_pickle.py::test_ToPickle PASSED [ 17%] distributed/protocol/tests/test_utils.py::test_host_array PASSED [ 17%] distributed/protocol/tests/test_utils.py::test_host_array_from_buffers PASSED [ 17%] distributed/protocol/tests/test_utils.py::test_host_array_from_buffers_numpy PASSED [ 17%] distributed/protocol/tests/test_utils_test.py::test_get_host_array PASSED [ 17%] distributed/shuffle/tests/test_buffer.py::test_memory_limit[big_payloads0] PASSED [ 17%] distributed/shuffle/tests/test_buffer.py::test_memory_limit[big_payloads1] PASSED [ 17%] distributed/shuffle/tests/test_buffer.py::test_memory_limit[big_payloads2] PASSED [ 17%] distributed/shuffle/tests/test_buffer.py::test_memory_limit[big_payloads3] PASSED [ 17%] distributed/shuffle/tests/test_buffer.py::test_memory_limit_blocked_exception PASSED [ 17%] distributed/shuffle/tests/test_comm_buffer.py::test_basic PASSED [ 17%] distributed/shuffle/tests/test_comm_buffer.py::test_exceptions PASSED [ 17%] distributed/shuffle/tests/test_comm_buffer.py::test_slow_send PASSED [ 17%] distributed/shuffle/tests/test_comm_buffer.py::test_concurrent_puts PASSED [ 17%] distributed/shuffle/tests/test_comm_buffer.py::test_concurrent_puts_error PASSED [ 17%] distributed/shuffle/tests/test_core.py::test_mean_shard_size PASSED [ 17%] distributed/shuffle/tests/test_core.py::test_mean_shard_size_numpy PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_basic PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_read_before_flush PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_many[2] PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_many[100] PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_many[1000] PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_exceptions PASSED [ 17%] distributed/shuffle/tests/test_disk_buffer.py::test_high_pressure_flush_with_exception PASSED [ 17%] distributed/shuffle/tests/test_limiter.py::test_limiter_basic PASSED [ 17%] distributed/shuffle/tests/test_limiter.py::test_unlimited_limiter PASSED [ 17%] distributed/shuffle/tests/test_limiter.py::test_limiter_concurrent_decrease_releases_waiter PASSED [ 17%] distributed/shuffle/tests/test_limiter.py::test_limiter_statistics PASSED [ 17%] distributed/shuffle/tests/test_limiter.py::test_limiter_no_limit_no_statistics PASSED [ 17%] distributed/shuffle/tests/test_memory_buffer.py::test_basic PASSED [ 18%] distributed/shuffle/tests/test_memory_buffer.py::test_read_before_flush PASSED [ 18%] distributed/shuffle/tests/test_memory_buffer.py::test_many[2] PASSED [ 18%] distributed/shuffle/tests/test_memory_buffer.py::test_many[100] PASSED [ 18%] distributed/shuffle/tests/test_memory_buffer.py::test_many[1000] PASSED [ 18%] distributed/shuffle/tests/test_pickle.py::test_pickle PASSED [ 18%] distributed/shuffle/tests/test_pickle.py::test_pickle_numpy PASSED [ 18%] distributed/shuffle/tests/test_pickle.py::test_pickle_zero_copy PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_basic PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_lock_timeout PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_concurrent_reads_are_allowed PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_read_blocks_write PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_write_blocks_read PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_write_blocks_write PASSED [ 18%] distributed/shuffle/tests/test_read_write_lock.py::test_write_preferred_over_read PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[True-True-1] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[True-True-10] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[True-False-1] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[True-False-10] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[False-True-1] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[False-True-10] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[False-False-1] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_lowlevel_rechunk[False-False-10] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[tasks-tasks] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[tasks-p2p] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[tasks-None] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[p2p-tasks] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[p2p-p2p] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[p2p-None] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[None-tasks] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[None-p2p] PASSED [ 18%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_configuration[None-None] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_cull_p2p_rechunk_independent_partitions PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_cull_p2p_rechunk_overlapping_partitions PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_2d[True] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_2d[False] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_4d[True] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_4d[False] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_expand PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_expand2 PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_method FAILED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_blockshape PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_dtype PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_dict PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_empty_input PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_null_dimensions PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_integer PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_0d PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_empty_array[arr0] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_empty_array[arr1] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_empty_array[arr2] PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_empty PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_zero_dim_array PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_zero_dim_array_II PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_same PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_same_fully_unknown PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_same_fully_unknown_floats PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_same_partially_unknown PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_zero_placeholders PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_minus_one PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_warning PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_unknown_from_pandas PASSED [ 19%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_unknown_from_array PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x0-chunks0] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x1-chunks1] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x2-chunks2] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x3-chunks3] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x4-chunks4] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x5-chunks5] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x6-chunks6] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x7-chunks7] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x8-chunks8] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x9-chunks9] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x10-chunks10] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x11-chunks11] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x0-chunks0] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x1-chunks1] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x2-chunks2] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] FAILED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x4-chunks4] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] FAILED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x6-chunks6] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x7-chunks7] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x8-chunks8] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x9-chunks9] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x10-chunks10] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x11-chunks11] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension_explicit[new_chunks0] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension_explicit[new_chunks1] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension_explicit[new_chunks2] PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_unknown_raises PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_zero_dim PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_empty_chunks PASSED [ 20%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_avoid_needless_chunking PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_1d[100-1-10-expected0] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_1d[100-50-10-expected1] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_1d[100-100-10-expected2] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_1d[20-7-10-expected3] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_1d[20-chunks4-5-expected4] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_2d PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_3d PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_image_stack[100] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_auto_image_stack[1000] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_down PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_zero PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_bad_keys PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_zero PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_1 PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_2 PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_nan PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_nan_single PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_nan_long PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_with_nonzero PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_split_axes_with_zero PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_preserve_writeable_flag PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_in_memory_shards_dont_share_buffer PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_pick_worker_homogeneous_distribution[1] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_pick_worker_homogeneous_distribution[2] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_pick_worker_homogeneous_distribution[41] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_pick_worker_homogeneous_distribution[50] PASSED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_partial_rechunk_homogeneous_distribution SKIPPED [ 21%] distributed/shuffle/tests/test_rechunk.py::test_partial_rechunk_taskgroups PASSED [ 21%] distributed/shuffle/tests/test_shuffle_plugins.py::test_installation_on_worker PASSED [ 21%] distributed/shuffle/tests/test_shuffle_plugins.py::test_installation_on_scheduler PASSED [ 21%] distributed/tests/test_active_memory_manager.py::test_no_policies PASSED [ 21%] distributed/tests/test_active_memory_manager.py::test_drop PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_start_stop PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_auto_start PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_add_policy PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_multi_start PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_not_registered PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_client_proxy_sync PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_client_proxy_async PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_not_in_memory PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_waiter PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_double_drop PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_double_drop_stress PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_from_worker_with_least_free_memory PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_candidates PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_empty_candidates PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_from_candidates_without_key PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_bad_candidates PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_prefers_paused_workers PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_paused_workers_with_running_tasks_1 PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_paused_workers_with_running_tasks_2 PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_paused_workers_with_running_tasks_3_4[True] PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_paused_workers_with_running_tasks_3_4[False] PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_drop_with_paused_workers_with_running_tasks_5 PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_not_in_memory PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_double_replicate_stress PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_to_worker_with_most_free_memory PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_with_candidates PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_with_empty_candidates PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_to_candidates_with_key PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_avoids_paused_workers_1 PASSED [ 22%] distributed/tests/test_active_memory_manager.py::test_replicate_avoids_paused_workers_2 PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_bad_measure PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[0-0-0-1] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[1-0-0-1] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[0-0-1-1] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[2-0-0-1] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[1-1-0-2] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[1-1-1-3] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[1-1-2-4] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[2-1-1-3] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[1-1-17-4] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[17-1-1-3] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_with_waiters[18-1-1-4] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_amm_on_off[False] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_amm_on_off[True] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_no_extension PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_no_remove PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_with_ReduceReplicas[False] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_with_ReduceReplicas[True] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_all_replicas_are_being_retired PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_no_recipients PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_all_recipients_are_paused PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_new_keys_arrive_after_all_keys_moved_away PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_faulty_recipient PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_mass PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_incremental PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_dont_drop_actors PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_dont_replicate_actors PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_with_actor[False] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_with_actor[True] PASSED [ 23%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_with_actor_proxy PASSED [ 24%] distributed/tests/test_active_memory_manager.py::test_noamm_stress SKIPPED [ 24%] distributed/tests/test_active_memory_manager.py::test_drop_stress SKIPPED [ 24%] distributed/tests/test_active_memory_manager.py::test_ReduceReplicas_stress SKIPPED [ 24%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_stress[False] SKIPPED [ 24%] distributed/tests/test_active_memory_manager.py::test_RetireWorker_stress[True] SKIPPED [ 24%] distributed/tests/test_actor.py::test_client_actions[True] PASSED [ 24%] distributed/tests/test_actor.py::test_client_actions[False] PASSED [ 24%] distributed/tests/test_actor.py::test_worker_actions[False] PASSED [ 24%] distributed/tests/test_actor.py::test_worker_actions[True] PASSED [ 24%] distributed/tests/test_actor.py::test_Actor PASSED [ 24%] distributed/tests/test_actor.py::test_linear_access XFAIL (Tornado can pass things out of orderShould rely on sending small messages rather than rpc) [ 24%] distributed/tests/test_actor.py::test_exceptions_create PASSED [ 24%] distributed/tests/test_actor.py::test_exceptions_method PASSED [ 24%] distributed/tests/test_actor.py::test_gc PASSED [ 24%] distributed/tests/test_actor.py::test_track_dependencies PASSED [ 24%] distributed/tests/test_actor.py::test_future PASSED [ 24%] distributed/tests/test_actor.py::test_future_dependencies PASSED [ 24%] distributed/tests/test_actor.py::test_sync PASSED [ 24%] distributed/tests/test_actor.py::test_timeout PASSED [ 24%] distributed/tests/test_actor.py::test_failed_worker PASSED [ 24%] distributed/tests/test_actor.py::test_numpy_roundtrip PASSED [ 24%] distributed/tests/test_actor.py::test_numpy_roundtrip_getattr PASSED [ 24%] distributed/tests/test_actor.py::test_repr PASSED [ 24%] distributed/tests/test_actor.py::test_dir PASSED [ 24%] distributed/tests/test_actor.py::test_many_computations PASSED [ 24%] distributed/tests/test_actor.py::test_thread_safety PASSED [ 24%] distributed/tests/test_actor.py::test_Actors_create_dependencies PASSED [ 24%] distributed/tests/test_actor.py::test_load_balance PASSED [ 24%] distributed/tests/test_actor.py::test_load_balance_map PASSED [ 24%] distributed/tests/test_actor.py::test_compute PASSED [ 24%] distributed/tests/test_actor.py::test_compute_sync PASSED [ 25%] distributed/tests/test_actor.py::test_actors_in_profile PASSED [ 25%] distributed/tests/test_actor.py::test_waiter PASSED [ 25%] distributed/tests/test_actor.py::test_worker_actor_handle_is_weakref PASSED [ 25%] distributed/tests/test_actor.py::test_worker_actor_handle_is_weakref_sync PASSED [ 25%] distributed/tests/test_actor.py::test_worker_actor_handle_is_weakref_from_compute_sync PASSED [ 25%] distributed/tests/test_actor.py::test_one_thread_deadlock PASSED [ 25%] distributed/tests/test_actor.py::test_one_thread_deadlock_timeout PASSED [ 25%] distributed/tests/test_actor.py::test_one_thread_deadlock_sync_client PASSED [ 25%] distributed/tests/test_actor.py::test_async_deadlock PASSED [ 25%] distributed/tests/test_actor.py::test_exception PASSED [ 25%] distributed/tests/test_actor.py::test_exception_async PASSED [ 25%] distributed/tests/test_actor.py::test_as_completed PASSED [ 25%] distributed/tests/test_actor.py::test_actor_future_awaitable PASSED [ 25%] distributed/tests/test_actor.py::test_actor_future_awaitable_deadlock PASSED [ 25%] distributed/tests/test_actor.py::test_serialize_with_pickle PASSED [ 25%] distributed/tests/test_actor.py::test_worker_client_async PASSED [ 25%] distributed/tests/test_actor.py::test_worker_client_separate_thread PASSED [ 25%] distributed/tests/test_actor.py::test_get_worker PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_async PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_timeout_async PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_timeout_sync PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_sync PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_with_non_futures PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_add PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_update PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_repeats PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_is_empty PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_cancel PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_cancel_last PASSED [ 25%] distributed/tests/test_as_completed.py::test_async_for_py2_equivalent PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_error_async PASSED [ 25%] distributed/tests/test_as_completed.py::test_as_completed_error PASSED [ 26%] distributed/tests/test_as_completed.py::test_as_completed_with_results PASSED [ 26%] distributed/tests/test_as_completed.py::test_as_completed_with_results_async PASSED [ 26%] distributed/tests/test_as_completed.py::test_as_completed_with_results_no_raise PASSED [ 26%] distributed/tests/test_as_completed.py::test_str PASSED [ 26%] distributed/tests/test_as_completed.py::test_as_completed_with_results_no_raise_async PASSED [ 26%] distributed/tests/test_as_completed.py::test_clear PASSED [ 26%] distributed/tests/test_asyncio.py::test_rlock PASSED [ 26%] distributed/tests/test_asyncio.py::test_rlock_nonreentrant PASSED [ 26%] distributed/tests/test_asyncio.py::test_rlock_none PASSED [ 26%] distributed/tests/test_asyncio.py::test_rlock_release_on_raise PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_simple PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_exitcode PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_sigint_from_same_process PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_sigterm_from_parent_process PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_terminate PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_close PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_exit_callback PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_child_main_thread PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_num_fds PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_terminate_after_stop PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_kill PASSED [ 26%] distributed/tests/test_asyncprocess.py::test_asyncprocess_child_teardown_on_parent_exit PASSED [ 26%] distributed/tests/test_batched.py::test_close_not_started PASSED [ 26%] distributed/tests/test_batched.py::test_stress SKIPPED (need --runslow option to run) [ 26%] distributed/tests/test_batched.py::test_large_traffic_jam SKIPPED (need --runslow option to run) [ 26%] distributed/tests/test_cancelled_state.py::test_abort_execution_release PASSED [ 26%] distributed/tests/test_cancelled_state.py::test_abort_execution_reschedule PASSED [ 26%] distributed/tests/test_cancelled_state.py::test_abort_execution_add_as_dependency PASSED [ 26%] distributed/tests/test_cancelled_state.py::test_abort_execution_to_fetch PASSED [ 26%] distributed/tests/test_cancelled_state.py::test_worker_stream_died_during_comm PASSED [ 26%] distributed/tests/test_cancelled_state.py::test_flight_to_executing_via_cancelled_resumed PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_executing_cancelled_error PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_flight_cancelled_error PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_in_flight_lost_after_resumed PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_error PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_error_with_resources PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_resumed_after_flight_with_dependencies PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_resumed_after_flight_with_dependencies_workerstate PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_resumed_cancelled_handle_compute[True-True] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_resumed_cancelled_handle_compute[True-False] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_resumed_cancelled_handle_compute[False-True] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_resumed_cancelled_handle_compute[False-False] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_handle_compute[True] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_handle_compute[False] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_cancelled_task_error_rejected PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[False-resumed] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[False-cancelled] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-resumed] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] FAILED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_to_executing[executing] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_to_executing[long-running] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_flight_to_flight PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_skips_fetch_on_success[executing] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_skips_fetch_on_success[long-running] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_failure_to_fetch[executing] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_executing_failure_to_fetch[long-running] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_flight_skips_executing_on_success PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_flight_failure_to_executing[False] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_flight_failure_to_executing[True] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_resumed_fetch_to_cancelled_to_executing[executing] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_resumed_fetch_to_cancelled_to_executing[long-running] PASSED [ 27%] distributed/tests/test_cancelled_state.py::test_workerstate_resumed_fetch_to_executing[executing] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_workerstate_resumed_fetch_to_executing[long-running] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_workerstate_resumed_waiting_to_flight PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_execute_preamble_early_cancel[executing-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_execute_preamble_early_cancel[executing-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_execute_preamble_early_cancel[resumed-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_execute_preamble_early_cancel[resumed-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[ExecuteSuccessEvent-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[ExecuteSuccessEvent-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[ExecuteFailureEvent-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[ExecuteFailureEvent-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[RescheduleEvent-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_cancel_with_dependencies_in_memory[RescheduleEvent-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteSuccessEvent-False-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteSuccessEvent-False-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteSuccessEvent-True-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteSuccessEvent-True-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteFailureEvent-False-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteFailureEvent-False-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteFailureEvent-True-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[ExecuteFailureEvent-True-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[RescheduleEvent-False-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[RescheduleEvent-False-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[RescheduleEvent-True-False] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_workerstate[RescheduleEvent-True-True] PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_secede_cancelled_or_resumed_scheduler PASSED [ 28%] distributed/tests/test_cancelled_state.py::test_workerstate_remove_replica_of_cancelled_task_dependency PASSED [ 28%] distributed/tests/test_chaos.py::test_KillWorker[sys.exit] PASSED [ 28%] distributed/tests/test_chaos.py::test_KillWorker[graceful] PASSED [ 28%] distributed/tests/test_chaos.py::test_KillWorker[segfault] PASSED [ 28%] distributed/tests/test_client.py::test_submit PASSED [ 28%] distributed/tests/test_client.py::test_map PASSED [ 28%] distributed/tests/test_client.py::test_map_empty PASSED [ 29%] distributed/tests/test_client.py::test_map_keynames PASSED [ 29%] distributed/tests/test_client.py::test_map_retries PASSED [ 29%] distributed/tests/test_client.py::test_map_batch_size PASSED [ 29%] distributed/tests/test_client.py::test_custom_key_with_batches PASSED [ 29%] distributed/tests/test_client.py::test_compute_retries PASSED [ 29%] distributed/tests/test_client.py::test_compute_retries_annotations PASSED [ 29%] distributed/tests/test_client.py::test_retries_get PASSED [ 29%] distributed/tests/test_client.py::test_persist_retries PASSED [ 29%] distributed/tests/test_client.py::test_persist_retries_annotations PASSED [ 29%] distributed/tests/test_client.py::test_retries_dask_array PASSED [ 29%] distributed/tests/test_client.py::test_future_repr PASSED [ 29%] distributed/tests/test_client.py::test_future_tuple_repr PASSED [ 29%] distributed/tests/test_client.py::test_Future_exception PASSED [ 29%] distributed/tests/test_client.py::test_Future_exception_sync PASSED [ 29%] distributed/tests/test_client.py::test_Future_release PASSED [ 29%] distributed/tests/test_client.py::test_Future_release_sync PASSED [ 29%] distributed/tests/test_client.py::test_short_tracebacks[result] PASSED [ 29%] distributed/tests/test_client.py::test_short_tracebacks[gather] PASSED [ 29%] distributed/tests/test_client.py::test_short_tracebacks_async[await] PASSED [ 29%] distributed/tests/test_client.py::test_short_tracebacks_async[result] PASSED [ 29%] distributed/tests/test_client.py::test_short_tracebacks_async[gather] PASSED [ 29%] distributed/tests/test_client.py::test_map_naming PASSED [ 29%] distributed/tests/test_client.py::test_submit_naming PASSED [ 29%] distributed/tests/test_client.py::test_exceptions PASSED [ 29%] distributed/tests/test_client.py::test_gc PASSED [ 29%] distributed/tests/test_client.py::test_thread PASSED [ 29%] distributed/tests/test_client.py::test_sync_exceptions PASSED [ 29%] distributed/tests/test_client.py::test_gather PASSED [ 29%] distributed/tests/test_client.py::test_gather_mismatched_client PASSED [ 29%] distributed/tests/test_client.py::test_gather_lost PASSED [ 29%] distributed/tests/test_client.py::test_gather_sync PASSED [ 30%] distributed/tests/test_client.py::test_gather_strict PASSED [ 30%] distributed/tests/test_client.py::test_gather_skip PASSED [ 30%] distributed/tests/test_client.py::test_limit_concurrent_gathering PASSED [ 30%] distributed/tests/test_client.py::test_get PASSED [ 30%] distributed/tests/test_client.py::test_get_sync PASSED [ 30%] distributed/tests/test_client.py::test_no_future_references PASSED [ 30%] distributed/tests/test_client.py::test_get_sync_optimize_graph_passes_through PASSED [ 30%] distributed/tests/test_client.py::test_gather_errors PASSED [ 30%] distributed/tests/test_client.py::test_wait PASSED [ 30%] distributed/tests/test_client.py::test_wait_first_completed PASSED [ 30%] distributed/tests/test_client.py::test_wait_timeout PASSED [ 30%] distributed/tests/test_client.py::test_wait_sync PASSED [ 30%] distributed/tests/test_client.py::test_wait_informative_error_for_timeouts PASSED [ 30%] distributed/tests/test_client.py::test_garbage_collection PASSED [ 30%] distributed/tests/test_client.py::test_garbage_collection_with_scatter PASSED [ 30%] distributed/tests/test_client.py::test_recompute_released_key PASSED [ 30%] distributed/tests/test_client.py::test_long_tasks_dont_trigger_timeout SKIPPED [ 30%] distributed/tests/test_client.py::test_tokenize_on_futures PASSED [ 30%] distributed/tests/test_client.py::test_restrictions_submit PASSED [ 30%] distributed/tests/test_client.py::test_restrictions_ip_port PASSED [ 30%] distributed/tests/test_client.py::test_restrictions_map PASSED [ 30%] distributed/tests/test_client.py::test_restrictions_get PASSED [ 30%] distributed/tests/test_client.py::test_restrictions_get_annotate PASSED [ 30%] distributed/tests/test_client.py::test_remove_worker PASSED [ 30%] distributed/tests/test_client.py::test_errors_dont_block PASSED [ 30%] distributed/tests/test_client.py::test_submit_quotes PASSED [ 30%] distributed/tests/test_client.py::test_map_quotes PASSED [ 30%] distributed/tests/test_client.py::test_two_consecutive_clients_share_results PASSED [ 30%] distributed/tests/test_client.py::test_submit_then_get_with_Future PASSED [ 30%] distributed/tests/test_client.py::test_aliases PASSED [ 30%] distributed/tests/test_client.py::test_aliases_2 PASSED [ 31%] distributed/tests/test_client.py::test_scatter PASSED [ 31%] distributed/tests/test_client.py::test_scatter_types PASSED [ 31%] distributed/tests/test_client.py::test_scatter_non_list PASSED [ 31%] distributed/tests/test_client.py::test_scatter_tokenize_local PASSED [ 31%] distributed/tests/test_client.py::test_scatter_singletons PASSED [ 31%] distributed/tests/test_client.py::test_scatter_typename PASSED [ 31%] distributed/tests/test_client.py::test_scatter_hash PASSED [ 31%] distributed/tests/test_client.py::test_scatter_hash_2 PASSED [ 31%] distributed/tests/test_client.py::test_get_releases_data PASSED [ 31%] distributed/tests/test_client.py::test_current PASSED [ 31%] distributed/tests/test_client.py::test_current_nested PASSED [ 31%] distributed/tests/test_client.py::test_current_nested_async PASSED [ 31%] distributed/tests/test_client.py::test_current_concurrent PASSED [ 31%] distributed/tests/test_client.py::test_context_manager_used_from_different_tasks PASSED [ 31%] distributed/tests/test_client.py::test_context_manager_used_from_different_threads PASSED [ 31%] distributed/tests/test_client.py::test_global_clients PASSED [ 31%] distributed/tests/test_client.py::test_exception_on_exception PASSED [ 31%] distributed/tests/test_client.py::test_get_task_prefix_states PASSED [ 31%] distributed/tests/test_client.py::test_get_nbytes PASSED [ 31%] distributed/tests/test_client.py::test_nbytes_determines_worker PASSED [ 31%] distributed/tests/test_client.py::test_if_intermediates_clear_on_error PASSED [ 31%] distributed/tests/test_client.py::test_pragmatic_move_small_data_to_large_data PASSED [ 31%] distributed/tests/test_client.py::test_get_with_non_list_key PASSED [ 31%] distributed/tests/test_client.py::test_get_with_error PASSED [ 31%] distributed/tests/test_client.py::test_get_with_error_sync PASSED [ 31%] distributed/tests/test_client.py::test_directed_scatter PASSED [ 31%] distributed/tests/test_client.py::test_scatter_direct PASSED [ 31%] distributed/tests/test_client.py::test_scatter_direct_2 PASSED [ 31%] distributed/tests/test_client.py::test_scatter_direct_numpy PASSED [ 31%] distributed/tests/test_client.py::test_scatter_direct_broadcast PASSED [ 31%] distributed/tests/test_client.py::test_scatter_direct_balanced PASSED [ 32%] distributed/tests/test_client.py::test_scatter_direct_broadcast_target PASSED [ 32%] distributed/tests/test_client.py::test_scatter_direct_empty PASSED [ 32%] distributed/tests/test_client.py::test_scatter_direct_spread_evenly PASSED [ 32%] distributed/tests/test_client.py::test_scatter_gather_sync[True-True] PASSED [ 32%] distributed/tests/test_client.py::test_scatter_gather_sync[True-False] PASSED [ 32%] distributed/tests/test_client.py::test_scatter_gather_sync[False-True] PASSED [ 32%] distributed/tests/test_client.py::test_scatter_gather_sync[False-False] PASSED [ 32%] distributed/tests/test_client.py::test_gather_direct PASSED [ 32%] distributed/tests/test_client.py::test_many_submits_spread_evenly PASSED [ 32%] distributed/tests/test_client.py::test_traceback PASSED [ 32%] distributed/tests/test_client.py::test_get_traceback PASSED [ 32%] distributed/tests/test_client.py::test_gather_traceback PASSED [ 32%] distributed/tests/test_client.py::test_traceback_sync PASSED [ 32%] distributed/tests/test_client.py::test_upload_file PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_refresh_delayed PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_no_extension PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_zip PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_egg SKIPPED (need --runslow option to run) [ 32%] distributed/tests/test_client.py::test_upload_large_file PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_sync PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_exception PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_exception_sync PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_load PASSED [ 32%] distributed/tests/test_client.py::test_upload_file_new_worker PASSED [ 32%] distributed/tests/test_client.py::test_multiple_clients SKIPPED [ 32%] distributed/tests/test_client.py::test_async_compute PASSED [ 32%] distributed/tests/test_client.py::test_async_compute_with_scatter PASSED [ 32%] distributed/tests/test_client.py::test_sync_compute PASSED [ 32%] distributed/tests/test_client.py::test_remote_scatter_gather PASSED [ 32%] distributed/tests/test_client.py::test_remote_submit_on_Future PASSED [ 32%] distributed/tests/test_client.py::test_start_is_idempotent PASSED [ 32%] distributed/tests/test_client.py::test_client_with_scheduler PASSED [ 33%] distributed/tests/test_client.py::test_allow_restrictions PASSED [ 33%] distributed/tests/test_client.py::test_bad_address PASSED [ 33%] distributed/tests/test_client.py::test_informative_error_on_cluster_type PASSED [ 33%] distributed/tests/test_client.py::test_long_error PASSED [ 33%] distributed/tests/test_client.py::test_map_on_futures_with_kwargs PASSED [ 33%] distributed/tests/test_client.py::test_badly_serialized_input_stderr SKIPPED [ 33%] distributed/tests/test_client.py::test_repr[str] PASSED [ 33%] distributed/tests/test_client.py::test_repr[repr] PASSED [ 33%] distributed/tests/test_client.py::test_repr[func2] PASSED [ 33%] distributed/tests/test_client.py::test_repr_async PASSED [ 33%] distributed/tests/test_client.py::test_repr_no_memory_limit PASSED [ 33%] distributed/tests/test_client.py::test_forget_simple PASSED [ 33%] distributed/tests/test_client.py::test_forget_complex PASSED [ 33%] distributed/tests/test_client.py::test_forget_in_flight PASSED [ 33%] distributed/tests/test_client.py::test_forget_errors PASSED [ 33%] distributed/tests/test_client.py::test_repr_sync PASSED [ 33%] distributed/tests/test_client.py::test_multi_client PASSED [ 33%] distributed/tests/test_client.py::test_cleanup_after_broken_client_connection PASSED [ 33%] distributed/tests/test_client.py::test_multi_garbage_collection PASSED [ 33%] distributed/tests/test_client.py::test__broadcast PASSED [ 33%] distributed/tests/test_client.py::test__broadcast_integer PASSED [ 33%] distributed/tests/test_client.py::test__broadcast_dict PASSED [ 33%] distributed/tests/test_client.py::test_proxy PASSED [ 33%] distributed/tests/test_client.py::test_cancel PASSED [ 33%] distributed/tests/test_client.py::test_cancel_tuple_key PASSED [ 33%] distributed/tests/test_client.py::test_cancel_multi_client PASSED [ 33%] distributed/tests/test_client.py::test_cancel_before_known_to_scheduler PASSED [ 33%] distributed/tests/test_client.py::test_cancel_collection PASSED [ 33%] distributed/tests/test_client.py::test_cancel_sync PASSED [ 33%] distributed/tests/test_client.py::test_future_type PASSED [ 33%] distributed/tests/test_client.py::test_traceback_clean PASSED [ 34%] distributed/tests/test_client.py::test_map_differnet_lengths PASSED [ 34%] distributed/tests/test_client.py::test_Future_exception_sync_2 PASSED [ 34%] distributed/tests/test_client.py::test_async_persist PASSED [ 34%] distributed/tests/test_client.py::test__persist PASSED [ 34%] distributed/tests/test_client.py::test_persist PASSED [ 34%] distributed/tests/test_client.py::test_long_traceback PASSED [ 34%] distributed/tests/test_client.py::test_wait_on_collections PASSED [ 34%] distributed/tests/test_client.py::test_futures_of_get PASSED [ 34%] distributed/tests/test_client.py::test_futures_of_class PASSED [ 34%] distributed/tests/test_client.py::test_futures_of_cancelled_raises PASSED [ 34%] distributed/tests/test_client.py::test_dont_delete_recomputed_results SKIPPED [ 34%] distributed/tests/test_client.py::test_balance_tasks_by_stacks SKIPPED [ 34%] distributed/tests/test_client.py::test_run PASSED [ 34%] distributed/tests/test_client.py::test_run_handles_picklable_data PASSED [ 34%] distributed/tests/test_client.py::test_run_sync PASSED [ 34%] distributed/tests/test_client.py::test_run_coroutine PASSED [ 34%] distributed/tests/test_client.py::test_run_coroutine_sync PASSED [ 34%] distributed/tests/test_client.py::test_run_exception PASSED [ 34%] distributed/tests/test_client.py::test_run_rpc_error FAILED [ 34%] distributed/tests/test_client.py::test_diagnostic_ui PASSED [ 34%] distributed/tests/test_client.py::test_diagnostic_nbytes_sync PASSED [ 34%] distributed/tests/test_client.py::test_diagnostic_nbytes PASSED [ 34%] distributed/tests/test_client.py::test_worker_aliases PASSED [ 34%] distributed/tests/test_client.py::test_persist_get_sync PASSED [ 34%] distributed/tests/test_client.py::test_persist_get PASSED [ 34%] distributed/tests/test_client.py::test_client_num_fds PASSED [ 34%] distributed/tests/test_client.py::test_startup_close_startup PASSED [ 34%] distributed/tests/test_client.py::test_startup_close_startup_sync PASSED [ 34%] distributed/tests/test_client.py::test_badly_serialized_exceptions PASSED [ 34%] distributed/tests/test_client.py::test_rebalance PASSED [ 34%] distributed/tests/test_client.py::test_rebalance_workers_and_keys PASSED [ 35%] distributed/tests/test_client.py::test_rebalance_unprepared PASSED [ 35%] distributed/tests/test_client.py::test_rebalance_raises_on_explicit_missing_data PASSED [ 35%] distributed/tests/test_client.py::test_receive_lost_key PASSED [ 35%] distributed/tests/test_client.py::test_unrunnable_task_runs PASSED [ 35%] distributed/tests/test_client.py::test_add_worker_after_tasks PASSED [ 35%] distributed/tests/test_client.py::test_workers_register_indirect_data PASSED [ 35%] distributed/tests/test_client.py::test_submit_on_cancelled_future PASSED [ 35%] distributed/tests/test_client.py::test_replicate PASSED [ 35%] distributed/tests/test_client.py::test_replicate_tuple_keys PASSED [ 35%] distributed/tests/test_client.py::test_replicate_workers PASSED [ 35%] distributed/tests/test_client.py::test_replicate_tree_branching PASSED [ 35%] distributed/tests/test_client.py::test_client_replicate PASSED [ 35%] distributed/tests/test_client.py::test_client_replicate_host PASSED [ 35%] distributed/tests/test_client.py::test_client_replicate_sync PASSED [ 35%] distributed/tests/test_client.py::test_task_load_adapts_quickly PASSED [ 35%] distributed/tests/test_client.py::test_even_load_after_fast_functions PASSED [ 35%] distributed/tests/test_client.py::test_even_load_on_startup PASSED [ 35%] distributed/tests/test_client.py::test_contiguous_load SKIPPED [ 35%] distributed/tests/test_client.py::test_balanced_with_submit PASSED [ 35%] distributed/tests/test_client.py::test_balanced_with_submit_and_resident_data PASSED [ 35%] distributed/tests/test_client.py::test_scheduler_saturates_cores PASSED [ 35%] distributed/tests/test_client.py::test_scheduler_saturates_cores_random PASSED [ 35%] distributed/tests/test_client.py::test_cancel_clears_processing PASSED [ 35%] distributed/tests/test_client.py::test_default_get PASSED [ 35%] distributed/tests/test_client.py::test_get_scheduler_default_client_config_interleaving PASSED [ 35%] distributed/tests/test_client.py::test_ensure_default_client PASSED [ 35%] distributed/tests/test_client.py::test_set_as_default PASSED [ 35%] distributed/tests/test_client.py::test_get_foo PASSED [ 35%] distributed/tests/test_client.py::test_get_foo_lost_keys PASSED [ 35%] distributed/tests/test_client.py::test_bad_tasks_fail SKIPPED (need --runslow option to run) [ 35%] distributed/tests/test_client.py::test_get_processing_sync PASSED [ 35%] distributed/tests/test_client.py::test_close_idempotent PASSED [ 36%] distributed/tests/test_client.py::test_get_returns_early PASSED [ 36%] distributed/tests/test_client.py::test_Client_clears_references_after_restart SKIPPED [ 36%] distributed/tests/test_client.py::test_get_stops_work_after_error PASSED [ 36%] distributed/tests/test_client.py::test_as_completed_list PASSED [ 36%] distributed/tests/test_client.py::test_as_completed_results PASSED [ 36%] distributed/tests/test_client.py::test_as_completed_batches[True] PASSED [ 36%] distributed/tests/test_client.py::test_as_completed_batches[False] PASSED [ 36%] distributed/tests/test_client.py::test_as_completed_next_batch PASSED [ 36%] distributed/tests/test_client.py::test_status PASSED [ 36%] distributed/tests/test_client.py::test_persist_optimize_graph PASSED [ 36%] distributed/tests/test_client.py::test_scatter_raises_if_no_workers PASSED [ 36%] distributed/tests/test_client.py::test_reconnect SKIPPED (need --runslow option to run) [ 36%] distributed/tests/test_client.py::test_reconnect_timeout PASSED [ 36%] distributed/tests/test_client.py::test_idempotence PASSED [ 36%] distributed/tests/test_client.py::test_scheduler_info PASSED [ 36%] distributed/tests/test_client.py::test_write_scheduler_file PASSED [ 36%] distributed/tests/test_client.py::test_get_versions_sync PASSED [ 36%] distributed/tests/test_client.py::test_get_versions_async PASSED [ 36%] distributed/tests/test_client.py::test_get_versions_rpc_error PASSED [ 36%] distributed/tests/test_client.py::test_threaded_get_within_distributed PASSED [ 36%] distributed/tests/test_client.py::test_lose_scattered_data PASSED [ 36%] distributed/tests/test_client.py::test_partially_lose_scattered_data PASSED [ 36%] distributed/tests/test_client.py::test_scatter_compute_lose PASSED [ 36%] distributed/tests/test_client.py::test_scatter_compute_store_lose PASSED [ 36%] distributed/tests/test_client.py::test_scatter_compute_store_lose_processing PASSED [ 36%] distributed/tests/test_client.py::test_serialize_future PASSED [ 36%] distributed/tests/test_client.py::test_serialize_future_without_client PASSED [ 36%] distributed/tests/test_client.py::test_temp_default_client PASSED [ 36%] distributed/tests/test_client.py::test_as_current PASSED [ 36%] distributed/tests/test_client.py::test_as_current_is_thread_local PASSED [ 36%] distributed/tests/test_client.py::test_as_current_is_task_local PASSED [ 37%] distributed/tests/test_client.py::test_persist_workers_annotate PASSED [ 37%] distributed/tests/test_client.py::test_persist_workers_annotate2 PASSED [ 37%] distributed/tests/test_client.py::test_persist_workers PASSED [ 37%] distributed/tests/test_client.py::test_compute_workers_annotate PASSED [ 37%] distributed/tests/test_client.py::test_compute_workers PASSED [ 37%] distributed/tests/test_client.py::test_compute_nested_containers PASSED [ 37%] distributed/tests/test_client.py::test_scatter_type PASSED [ 37%] distributed/tests/test_client.py::test_retire_workers_2 PASSED [ 37%] distributed/tests/test_client.py::test_retire_many_workers PASSED [ 37%] distributed/tests/test_client.py::test_weight_occupancy_against_data_movement PASSED [ 37%] distributed/tests/test_client.py::test_distribute_tasks_by_nthreads PASSED [ 37%] distributed/tests/test_client.py::test_add_done_callback PASSED [ 37%] distributed/tests/test_client.py::test_normalize_collection PASSED [ 37%] distributed/tests/test_client.py::test_normalize_collection_dask_array PASSED [ 37%] distributed/tests/test_client.py::test_normalize_collection_with_released_futures SKIPPED [ 37%] distributed/tests/test_client.py::test_auto_normalize_collection XPASS [ 37%] distributed/tests/test_client.py::test_auto_normalize_collection_sync XPASS [ 37%] distributed/tests/test_client.py::test_interleave_computations PASSED [ 37%] distributed/tests/test_client.py::test_interleave_computations_map SKIPPED [ 37%] distributed/tests/test_client.py::test_scatter_dict_workers PASSED [ 37%] distributed/tests/test_client.py::test_client_timeout SKIPPED (need --runslow option to run) [ 37%] distributed/tests/test_client.py::test_submit_list_kwargs PASSED [ 37%] distributed/tests/test_client.py::test_map_list_kwargs PASSED [ 37%] distributed/tests/test_client.py::test_recreate_error_delayed PASSED [ 37%] distributed/tests/test_client.py::test_recreate_error_futures PASSED [ 37%] distributed/tests/test_client.py::test_recreate_error_collection PASSED [ 37%] distributed/tests/test_client.py::test_recreate_error_array SKIPPED [ 37%] distributed/tests/test_client.py::test_recreate_error_sync PASSED [ 37%] distributed/tests/test_client.py::test_recreate_error_not_error PASSED [ 37%] distributed/tests/test_client.py::test_recreate_task_delayed PASSED [ 37%] distributed/tests/test_client.py::test_recreate_task_futures PASSED [ 38%] distributed/tests/test_client.py::test_recreate_task_collection PASSED [ 38%] distributed/tests/test_client.py::test_recreate_task_array PASSED [ 38%] distributed/tests/test_client.py::test_recreate_task_sync PASSED [ 38%] distributed/tests/test_client.py::test_retire_workers PASSED [ 38%] distributed/tests/test_client.py::test_restart_workers SKIPPED (need --runslow option to run) [ 38%] distributed/tests/test_client.py::test_restart_workers_no_nanny_raises PASSED [ 38%] distributed/tests/test_client.py::test_restart_workers_kill_timeout[True] SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_kill_timeout[False] SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_restart_timeout[True] SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_restart_timeout[False] SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_exception SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_by_name[True] SKIPPED [ 38%] distributed/tests/test_client.py::test_restart_workers_by_name[False] SKIPPED [ 38%] distributed/tests/test_client.py::test_robust_unserializable PASSED [ 38%] distributed/tests/test_client.py::test_robust_undeserializable PASSED [ 38%] distributed/tests/test_client.py::test_robust_undeserializable_function PASSED [ 38%] distributed/tests/test_client.py::test_fire_and_forget PASSED [ 38%] distributed/tests/test_client.py::test_fire_and_forget_err PASSED [ 38%] distributed/tests/test_client.py::test_close PASSED [ 38%] distributed/tests/test_client.py::test_threadsafe PASSED [ 38%] distributed/tests/test_client.py::test_threadsafe_get SKIPPED (need --runslow option to run) [ 38%] distributed/tests/test_client.py::test_threadsafe_compute SKIPPED (need --runslow option to run) [ 38%] distributed/tests/test_client.py::test_identity PASSED [ 38%] distributed/tests/test_client.py::test_get_client PASSED [ 38%] distributed/tests/test_client.py::test_get_client_no_cluster PASSED [ 38%] distributed/tests/test_client.py::test_serialize_collections PASSED [ 38%] distributed/tests/test_client.py::test_secede_simple PASSED [ 38%] distributed/tests/test_client.py::test_secede_balances PASSED [ 38%] distributed/tests/test_client.py::test_long_running_not_in_occupancy[True] PASSED [ 38%] distributed/tests/test_client.py::test_long_running_not_in_occupancy[False] PASSED [ 38%] distributed/tests/test_client.py::test_long_running_removal_clean[True] PASSED [ 39%] distributed/tests/test_client.py::test_long_running_removal_clean[False] PASSED [ 39%] distributed/tests/test_client.py::test_sub_submit_priority PASSED [ 39%] distributed/tests/test_client.py::test_get_client_sync PASSED [ 39%] distributed/tests/test_client.py::test_serialize_collections_of_futures PASSED [ 39%] distributed/tests/test_client.py::test_serialize_collections_of_futures_sync PASSED [ 39%] distributed/tests/test_client.py::test_dynamic_workloads_sync PASSED [ 39%] distributed/tests/test_client.py::test_dynamic_workloads_sync_random SKIPPED [ 39%] distributed/tests/test_client.py::test_bytes_keys PASSED [ 39%] distributed/tests/test_client.py::test_unicode_ascii_keys PASSED [ 39%] distributed/tests/test_client.py::test_unicode_keys PASSED [ 39%] distributed/tests/test_client.py::test_use_synchronous_client_in_async_context PASSED [ 39%] distributed/tests/test_client.py::test_quiet_quit_when_cluster_leaves PASSED [ 39%] distributed/tests/test_client.py::test_call_stack_future PASSED [ 39%] distributed/tests/test_client.py::test_call_stack_all PASSED [ 39%] distributed/tests/test_client.py::test_call_stack_collections PASSED [ 39%] distributed/tests/test_client.py::test_call_stack_collections_all PASSED [ 39%] distributed/tests/test_client.py::test_profile PASSED [ 39%] distributed/tests/test_client.py::test_profile_disabled PASSED [ 39%] distributed/tests/test_client.py::test_profile_keys PASSED [ 39%] distributed/tests/test_client.py::test_client_with_name PASSED [ 39%] distributed/tests/test_client.py::test_future_defaults_to_default_client PASSED [ 39%] distributed/tests/test_client.py::test_future_auto_inform PASSED [ 39%] distributed/tests/test_client.py::test_client_async_before_loop_starts PASSED [ 39%] distributed/tests/test_client.py::test_nested_compute SKIPPED (need --runslow option to run) [ 39%] distributed/tests/test_client.py::test_task_metadata PASSED [ 39%] distributed/tests/test_client.py::test_logs PASSED [ 39%] distributed/tests/test_client.py::test_logs_from_worker_submodules PASSED [ 39%] distributed/tests/test_client.py::test_avoid_delayed_finalize PASSED [ 39%] distributed/tests/test_client.py::test_config_scheduler_address PASSED [ 39%] distributed/tests/test_client.py::test_warn_when_submitting_large_values PASSED [ 39%] distributed/tests/test_client.py::test_warn_when_submitting_large_values_memoryview PASSED [ 39%] distributed/tests/test_client.py::test_unhashable_function PASSED [ 40%] distributed/tests/test_client.py::test_client_name PASSED [ 40%] distributed/tests/test_client.py::test_client_doesnt_close_given_loop PASSED [ 40%] distributed/tests/test_client.py::test_quiet_scheduler_loss SKIPPED [ 40%] distributed/tests/test_client.py::test_dashboard_link PASSED [ 40%] distributed/tests/test_client.py::test_client_timeout_2 PASSED [ 40%] distributed/tests/test_client.py::test_turn_off_pickle[True] PASSED [ 40%] distributed/tests/test_client.py::test_turn_off_pickle[False] PASSED [ 40%] distributed/tests/test_client.py::test_de_serialization PASSED [ 40%] distributed/tests/test_client.py::test_de_serialization_none PASSED [ 40%] distributed/tests/test_client.py::test_client_repr_closed PASSED [ 40%] distributed/tests/test_client.py::test_client_repr_closed_sync SKIPPED [ 40%] distributed/tests/test_client.py::test_nested_prioritization XPASS [ 40%] distributed/tests/test_client.py::test_scatter_error_cancel PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[False-False-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[False-False-True] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[False-True-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[False-True-True] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[True-False-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[True-False-True] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[True-True-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[True-True-True] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[10-False-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[10-False-True] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[10-True-False] PASSED [ 40%] distributed/tests/test_client.py::test_scatter_and_replicate_avoid_paused_workers[10-True-True] PASSED [ 40%] distributed/tests/test_client.py::test_no_threads_lingering XPASS [ 40%] distributed/tests/test_client.py::test_direct_async PASSED [ 40%] distributed/tests/test_client.py::test_direct_sync PASSED [ 40%] distributed/tests/test_client.py::test_mixing_clients_same_scheduler PASSED [ 40%] distributed/tests/test_client.py::test_tuple_keys PASSED [ 40%] distributed/tests/test_client.py::test_multiple_scatter PASSED [ 41%] distributed/tests/test_client.py::test_map_large_kwargs_in_graph PASSED [ 41%] distributed/tests/test_client.py::test_retry PASSED [ 41%] distributed/tests/test_client.py::test_retry_dependencies PASSED [ 41%] distributed/tests/test_client.py::test_released_dependencies PASSED [ 41%] distributed/tests/test_client.py::test_profile_bokeh SKIPPED (could not import 'bokeh.plotting': No module named 'bokeh') [ 41%] distributed/tests/test_client.py::test_get_mix_futures_and_SubgraphCallable PASSED [ 41%] distributed/tests/test_client.py::test_get_mix_futures_and_SubgraphCallable_dask_dataframe PASSED [ 41%] distributed/tests/test_client.py::test_direct_to_workers PASSED [ 41%] distributed/tests/test_client.py::test_instances PASSED [ 41%] distributed/tests/test_client.py::test_wait_for_workers PASSED [ 41%] distributed/tests/test_client.py::test_config_inherited_by_subprocess PASSED [ 41%] distributed/tests/test_client.py::test_futures_of_sorted PASSED [ 41%] distributed/tests/test_client.py::test_profile_server PASSED [ 41%] distributed/tests/test_client.py::test_profile_server_disabled PASSED [ 41%] distributed/tests/test_client.py::test_await_future PASSED [ 41%] distributed/tests/test_client.py::test_as_completed_async_for PASSED [ 41%] distributed/tests/test_client.py::test_as_completed_async_for_results PASSED [ 41%] distributed/tests/test_client.py::test_as_completed_async_for_cancel PASSED [ 41%] distributed/tests/test_client.py::test_client_sync_with_async_def PASSED [ 41%] distributed/tests/test_client.py::test_dont_hold_on_to_large_messages SKIPPED [ 41%] distributed/tests/test_client.py::test_run_on_scheduler_async_def PASSED [ 41%] distributed/tests/test_client.py::test_run_on_scheduler_async_def_wait PASSED [ 41%] distributed/tests/test_client.py::test_performance_report[True] SKIPPED [ 41%] distributed/tests/test_client.py::test_performance_report[False] SKIPPED [ 41%] distributed/tests/test_client.py::test_client_gather_semaphore_loop SKIPPED [ 41%] distributed/tests/test_client.py::test_as_completed_condition_loop PASSED [ 41%] distributed/tests/test_client.py::test_client_connectionpool_semaphore_loop SKIPPED [ 41%] distributed/tests/test_client.py::test_mixed_compression SKIPPED (need --runslow option to run) [ 41%] distributed/tests/test_client.py::test_futures_in_subgraphs PASSED [ 41%] distributed/tests/test_client.py::test_get_task_metadata PASSED [ 41%] distributed/tests/test_client.py::test_get_task_metadata_multiple PASSED [ 42%] distributed/tests/test_client.py::test_register_worker_plugin_exception PASSED [ 42%] distributed/tests/test_client.py::test_log_event PASSED [ 42%] distributed/tests/test_client.py::test_log_event_multiple_clients PASSED [ 42%] distributed/tests/test_client.py::test_annotations_task_state PASSED [ 42%] distributed/tests/test_client.py::test_annotations_compute_time[compute] PASSED [ 42%] distributed/tests/test_client.py::test_annotations_compute_time[persist] PASSED [ 42%] distributed/tests/test_client.py::test_annotations_survive_optimization XFAIL [ 42%] distributed/tests/test_client.py::test_annotations_priorities PASSED [ 42%] distributed/tests/test_client.py::test_annotations_workers PASSED [ 42%] distributed/tests/test_client.py::test_annotations_retries PASSED [ 42%] distributed/tests/test_client.py::test_annotations_blockwise_unpack PASSED [ 42%] distributed/tests/test_client.py::test_annotations_resources PASSED [ 42%] distributed/tests/test_client.py::test_annotations_resources_culled PASSED [ 42%] distributed/tests/test_client.py::test_annotations_loose_restrictions PASSED [ 42%] distributed/tests/test_client.py::test_annotations_submit_map PASSED [ 42%] distributed/tests/test_client.py::test_annotations_global_vs_local PASSED [ 42%] distributed/tests/test_client.py::test_workers_collection_restriction PASSED [ 42%] distributed/tests/test_client.py::test_get_client_functions_spawn_clusters SKIPPED [ 42%] distributed/tests/test_client.py::test_computation_code_walk_frames PASSED [ 42%] distributed/tests/test_client.py::test_computation_ignore_ipython_frames[2] SKIPPED [ 42%] distributed/tests/test_client.py::test_computation_ignore_ipython_frames[3] SKIPPED [ 42%] distributed/tests/test_client.py::test_computation_store_annotations PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_dask_compute PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_dask_compute_no_frames_default PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_not_available SKIPPED [ 42%] distributed/tests/test_client.py::test_computation_object_code_dask_persist PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_client_submit_simple PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_client_submit_list_comp PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_client_submit_dict_comp PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_client_map PASSED [ 42%] distributed/tests/test_client.py::test_computation_object_code_client_compute PASSED [ 42%] distributed/tests/test_client.py::test_upload_directory SKIPPED (need --runslow option to run) [ 43%] distributed/tests/test_client.py::test_duck_typed_register_plugin_raises PASSED [ 43%] distributed/tests/test_client.py::test_exception_text PASSED [ 43%] distributed/tests/test_client.py::test_async_task PASSED [ 43%] distributed/tests/test_client.py::test_async_task_with_partial PASSED [ 43%] distributed/tests/test_client.py::test_events_subscribe_topic PASSED [ 43%] distributed/tests/test_client.py::test_events_subscribe_topic_cancelled PASSED [ 43%] distributed/tests/test_client.py::test_events_all_servers_use_same_channel PASSED [ 43%] distributed/tests/test_client.py::test_events_unsubscribe_raises_if_unknown PASSED [ 43%] distributed/tests/test_client.py::test_log_event_warn PASSED [ 43%] distributed/tests/test_client.py::test_log_event_msgpack PASSED [ 43%] distributed/tests/test_client.py::test_log_event_warn_dask_warns PASSED [ 43%] distributed/tests/test_client.py::test_print_remote PASSED [ 43%] distributed/tests/test_client.py::test_print_manual PASSED [ 43%] distributed/tests/test_client.py::test_print_manual_bad_args PASSED [ 43%] distributed/tests/test_client.py::test_print_non_msgpack_serializable PASSED [ 43%] distributed/tests/test_client.py::test_print_local PASSED [ 43%] distributed/tests/test_client.py::test_forward_logging PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_write_from_scheduler PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_sync[msgpack-True] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_sync[msgpack-False] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_sync[yaml-True] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_sync[yaml-False] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_async[msgpack-True] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_async[msgpack-False] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_async[yaml-True] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_async[yaml-False] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_json[True] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_json[False] PASSED [ 43%] distributed/tests/test_client.py::test_dump_cluster_state_exclude_default PASSED [ 43%] distributed/tests/test_client.py::test_benchmark_hardware_no_workers PASSED [ 43%] distributed/tests/test_client.py::test_wait_for_workers_updates_info PASSED [ 44%] distributed/tests/test_client.py::test_quiet_close_process[True] SKIPPED [ 44%] distributed/tests/test_client.py::test_quiet_close_process[False] SKIPPED [ 44%] distributed/tests/test_client.py::test_deprecated_loop_properties PASSED [ 44%] distributed/tests/test_client.py::test_fast_close_on_aexit_failure PASSED [ 44%] distributed/tests/test_client.py::test_wait_for_workers_n_workers_value_check[None-ValueError] PASSED [ 44%] distributed/tests/test_client.py::test_wait_for_workers_n_workers_value_check[0-ValueError] PASSED [ 44%] distributed/tests/test_client.py::test_wait_for_workers_n_workers_value_check[1.0-ValueError] PASSED [ 44%] distributed/tests/test_client.py::test_wait_for_workers_n_workers_value_check[1-None] PASSED [ 44%] distributed/tests/test_client.py::test_wait_for_workers_n_workers_value_check[2-None] PASSED [ 44%] distributed/tests/test_client.py::test_unpacks_remotedata_namedtuple[PlainNamedTuple-args0-kwargs0] PASSED [ 44%] distributed/tests/test_client.py::test_unpacks_remotedata_namedtuple[NewArgsNamedTuple-args1-kwargs1] PASSED [ 44%] distributed/tests/test_client.py::test_unpacks_remotedata_namedtuple[NewArgsExNamedTuple-args2-kwargs2] PASSED [ 44%] distributed/tests/test_client.py::test_resolves_future_in_namedtuple[PlainNamedTuple-args0-kwargs0] PASSED [ 44%] distributed/tests/test_client.py::test_resolves_future_in_namedtuple[NewArgsNamedTuple-args1-kwargs1] PASSED [ 44%] distributed/tests/test_client.py::test_resolves_future_in_namedtuple[NewArgsExNamedTuple-args2-kwargs2] PASSED [ 44%] distributed/tests/test_client.py::test_resolves_future_in_dict PASSED [ 44%] distributed/tests/test_client.py::test_gather_race_vs_AMM[False] PASSED [ 44%] distributed/tests/test_client.py::test_gather_race_vs_AMM[True] PASSED [ 44%] distributed/tests/test_client_executor.py::test_submit PASSED [ 44%] distributed/tests/test_client_executor.py::test_as_completed PASSED [ 44%] distributed/tests/test_client_executor.py::test_wait PASSED [ 44%] distributed/tests/test_client_executor.py::test_cancellation PASSED [ 44%] distributed/tests/test_client_executor.py::test_cancellation_wait PASSED [ 44%] distributed/tests/test_client_executor.py::test_cancellation_as_completed PASSED [ 44%] distributed/tests/test_client_executor.py::test_map SKIPPED (need --runslow option to run) [ 44%] distributed/tests/test_client_executor.py::test_pure PASSED [ 44%] distributed/tests/test_client_executor.py::test_workers PASSED [ 44%] distributed/tests/test_client_executor.py::test_unsupported_arguments PASSED [ 44%] distributed/tests/test_client_executor.py::test_retries PASSED [ 44%] distributed/tests/test_client_executor.py::test_shutdown_wait PASSED [ 44%] distributed/tests/test_client_executor.py::test_shutdown_nowait PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_tuple_to_list[input0-expected0] PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_tuple_to_list[input1-expected1] PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_tuple_to_list[input2-expected2] PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_tuple_to_list[foo-foo] PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_write_state_msgpack PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_write_state_yaml PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_cluster_dump_state PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_cluster_dump_story PASSED [ 45%] distributed/tests/test_cluster_dump.py::test_cluster_dump_to_yamls PASSED [ 45%] distributed/tests/test_collections.py::test_lru PASSED [ 45%] distributed/tests/test_collections.py::test_heapset PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_sorted_flag_left PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_sorted_flag_right PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_popright[False] PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_popright[True] PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_pickle PASSED [ 45%] distributed/tests/test_collections.py::test_heapset_sort_duplicate PASSED [ 45%] distributed/tests/test_collections.py::test_sum_mappings PASSED [ 45%] distributed/tests/test_compatibility.py::test_randbytes PASSED [ 45%] distributed/tests/test_compatibility.py::test_randbytes_seed PASSED [ 45%] distributed/tests/test_computations.py::test_computations PASSED [ 45%] distributed/tests/test_computations.py::test_computations_futures PASSED [ 45%] distributed/tests/test_computations.py::test_computations_no_workers PASSED [ 45%] distributed/tests/test_computations.py::test_computations_no_resources PASSED [ 45%] distributed/tests/test_computations.py::test_computations_long_running PASSED [ 45%] distributed/tests/test_config.py::test_logging_default[config0] PASSED [ 45%] distributed/tests/test_config.py::test_logging_default[None] PASSED [ 45%] distributed/tests/test_config.py::test_default_logging_does_not_override_basic_config PASSED [ 45%] distributed/tests/test_config.py::test_basic_config_does_not_override_default_logging PASSED [ 45%] distributed/tests/test_config.py::test_schema PASSED [ 45%] distributed/tests/test_config.py::test_schema_is_complete PASSED [ 46%] distributed/tests/test_config.py::test_uvloop_event_loop SKIPPED (could not import 'uvloop': No module named 'uvloop') [ 46%] distributed/tests/test_config.py::test_deprecations_on_set[args0-kwargs0] PASSED [ 46%] distributed/tests/test_config.py::test_deprecations_on_set[args1-kwargs1] PASSED [ 46%] distributed/tests/test_config.py::test_deprecations_on_set[args2-kwargs2] PASSED [ 46%] distributed/tests/test_config.py::test_deprecations_on_env_variables PASSED [ 46%] distributed/tests/test_config.py::test_deprecations_on_yaml[allowed-failures] PASSED [ 46%] distributed/tests/test_config.py::test_deprecations_on_yaml[allowed_failures] PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_initialization PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_call_soon_executes_task_in_background PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_call_later_executes_delayed_task_in_background PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_close_closes PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_close_does_not_cancel_existing_tasks PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_close_prohibits_new_tasks PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_stop_disallows_shutdown PASSED [ 46%] distributed/tests/test_core.py::test_async_task_group_stop_cancels_long_running PASSED [ 46%] distributed/tests/test_core.py::test_rpc_inputs PASSED [ 46%] distributed/tests/test_core.py::test_large_packets_tcp SKIPPED (need --runslow option to run) [ 46%] distributed/tests/test_core.py::test_connect_raises PASSED [ 46%] distributed/tests/test_core.py::test_coerce_to_address PASSED [ 46%] distributed/tests/test_core.py::test_remove_cancels_connect_attempts PASSED [ 46%] distributed/tests/test_core.py::test_remove_cancels_connect_before_task_running PASSED [ 46%] distributed/tests/test_core.py::test_ticks SKIPPED (could not import 'crick': No module named 'crick') [ 46%] distributed/tests/test_core.py::test_tick_logging SKIPPED (could not import 'crick': No module named 'crick') [ 46%] distributed/tests/test_core.py::test_thread_id PASSED [ 46%] distributed/tests/test_core.py::test_expects_comm PASSED [ 46%] distributed/tests/test_core.py::test_async_listener_stop PASSED [ 46%] distributed/tests/test_core.py::test_large_payload SKIPPED (need --runslow option to run) [ 46%] distributed/tests/test_counter.py::test_digest[Counter-] PASSED [ 46%] distributed/tests/test_counter.py::test_digest[None-] SKIPPED [ 46%] distributed/tests/test_counter.py::test_counter PASSED [ 46%] distributed/tests/test_dask_collections.py::test_dataframes PASSED [ 46%] distributed/tests/test_dask_collections.py::test_dask_array_collections PASSED [ 47%] distributed/tests/test_dask_collections.py::test_bag_groupby_tasks_default PASSED [ 47%] distributed/tests/test_dask_collections.py::test_bag_groupby_key_hashing PASSED [ 47%] distributed/tests/test_dask_collections.py::test_dataframe_set_index_sync[wait] PASSED [ 47%] distributed/tests/test_dask_collections.py::test_dataframe_set_index_sync[] PASSED [ 47%] distributed/tests/test_dask_collections.py::test_loc_sync PASSED [ 47%] distributed/tests/test_dask_collections.py::test_rolling_sync PASSED [ 47%] distributed/tests/test_dask_collections.py::test_loc PASSED [ 47%] distributed/tests/test_dask_collections.py::test_dataframe_groupby_tasks PASSED [ 47%] distributed/tests/test_dask_collections.py::test_sparse_arrays SKIPPED [ 47%] distributed/tests/test_dask_collections.py::test_delayed_none PASSED [ 47%] distributed/tests/test_dask_collections.py::test_tuple_futures_arg[tuple] PASSED [ 47%] distributed/tests/test_dask_collections.py::test_tuple_futures_arg[list] PASSED [ 47%] distributed/tests/test_deadline.py::test_deadline PASSED [ 47%] distributed/tests/test_deadline.py::test_infinite_deadline PASSED [ 47%] distributed/tests/test_deadline.py::test_deadline_progress PASSED [ 47%] distributed/tests/test_deadline.py::test_deadline_expiration PASSED [ 47%] distributed/tests/test_deadline.py::test_deadline_expiration_async PASSED [ 47%] distributed/tests/test_diskutils.py::test_workdir_simple PASSED [ 47%] distributed/tests/test_diskutils.py::test_two_workspaces_in_same_directory PASSED [ 47%] distributed/tests/test_diskutils.py::test_workspace_process_crash PASSED [ 47%] distributed/tests/test_diskutils.py::test_workspace_rmtree_failure PASSED [ 47%] distributed/tests/test_diskutils.py::test_locking_disabled PASSED [ 47%] distributed/tests/test_diskutils.py::test_workspace_concurrency SKIPPED [ 47%] distributed/tests/test_diskutils.py::test_unwritable_base_dir PASSED [ 47%] distributed/tests/test_events.py::test_event_on_workers PASSED [ 47%] distributed/tests/test_events.py::test_default_event PASSED [ 47%] distributed/tests/test_events.py::test_set_not_set PASSED [ 47%] distributed/tests/test_events.py::test_set_not_set_many_events PASSED [ 47%] distributed/tests/test_events.py::test_timeout PASSED [ 47%] distributed/tests/test_events.py::test_event_sync PASSED [ 47%] distributed/tests/test_events.py::test_event_types PASSED [ 48%] distributed/tests/test_events.py::test_serializable PASSED [ 48%] distributed/tests/test_events.py::test_two_events_on_workers PASSED [ 48%] distributed/tests/test_events.py::test_unpickle_without_client PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_sync SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[False-False-closing] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[False-False-closed] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[False-True-closing] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[False-True-closed] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[True-False-closing] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[True-False-closed] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[True-True-closing] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker_async[True-True-closed] PASSED [ 48%] distributed/tests/test_failed_workers.py::test_submit_after_failed_worker PASSED [ 48%] distributed/tests/test_failed_workers.py::test_gather_after_failed_worker SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_gather_then_submit_after_failed_workers SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_restart PASSED [ 48%] distributed/tests/test_failed_workers.py::test_restart_cleared PASSED [ 48%] distributed/tests/test_failed_workers.py::test_restart_sync PASSED [ 48%] distributed/tests/test_failed_workers.py::test_worker_doesnt_await_task_completion PASSED [ 48%] distributed/tests/test_failed_workers.py::test_multiple_clients_restart PASSED [ 48%] distributed/tests/test_failed_workers.py::test_restart_scheduler PASSED [ 48%] distributed/tests/test_failed_workers.py::test_forgotten_futures_dont_clean_up_new_futures PASSED [ 48%] distributed/tests/test_failed_workers.py::test_broken_worker_during_computation SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_restart_during_computation PASSED [ 48%] distributed/tests/test_failed_workers.py::test_worker_who_has_clears_after_failed_connection SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_worker_same_host_replicas_missing PASSED [ 48%] distributed/tests/test_failed_workers.py::test_restart_timeout_on_long_running_task SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_worker_time_to_live SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_worker_ttl_restarts_worker[False] SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_worker_ttl_restarts_worker[True] SKIPPED [ 48%] distributed/tests/test_failed_workers.py::test_restart_hung_worker SKIPPED [ 49%] distributed/tests/test_failed_workers.py::test_forget_data_not_supposed_to_have PASSED [ 49%] distributed/tests/test_failed_workers.py::test_failing_worker_with_additional_replicas_on_cluster PASSED [ 49%] distributed/tests/test_imports.py::test_can_import_distributed_in_background_thread PASSED [ 49%] distributed/tests/test_init.py::test_version PASSED [ 49%] distributed/tests/test_itertools.py::test_ffill PASSED [ 49%] distributed/tests/test_jupyter.py::test_jupyter_cli SKIPPED (need --runslow option to run) [ 49%] distributed/tests/test_locks.py::test_lock PASSED [ 49%] distributed/tests/test_locks.py::test_timeout PASSED [ 49%] distributed/tests/test_locks.py::test_acquires_with_zero_timeout PASSED [ 49%] distributed/tests/test_locks.py::test_acquires_blocking PASSED [ 49%] distributed/tests/test_locks.py::test_timeout_sync PASSED [ 49%] distributed/tests/test_locks.py::test_lock_sync PASSED [ 49%] distributed/tests/test_locks.py::test_lock_types PASSED [ 49%] distributed/tests/test_locks.py::test_serializable PASSED [ 49%] distributed/tests/test_locks.py::test_locks PASSED [ 49%] distributed/tests/test_metrics.py::test_wall_clock[time] PASSED [ 49%] distributed/tests/test_metrics.py::test_wall_clock[monotonic] PASSED [ 49%] distributed/tests/test_metrics.py::test_monotonic SKIPPED (WindowsTime doesn't work with high accuracy base timer) [ 49%] distributed/tests/test_metrics.py::test_meter PASSED [ 49%] distributed/tests/test_metrics.py::test_meter_raise PASSED [ 49%] distributed/tests/test_metrics.py::test_meter_floor[kwargs0-0] PASSED [ 49%] distributed/tests/test_metrics.py::test_meter_floor[kwargs1-0.1] PASSED [ 49%] distributed/tests/test_metrics.py::test_meter_floor[kwargs2--1] PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_raise PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_nested PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_decorator PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_nested_floor PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_pickle PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_clear_callbacks PASSED [ 49%] distributed/tests/test_metrics.py::test_context_meter_clear_callbacks_raises PASSED [ 50%] distributed/tests/test_metrics.py::test_context_meter_allow_offload PASSED [ 50%] distributed/tests/test_metrics.py::test_delayed_metrics_ledger PASSED [ 50%] distributed/tests/test_metrics.py::test_context_meter_keyed PASSED [ 50%] distributed/tests/test_metrics.py::test_delayed_metrics_ledger_keyed PASSED [ 50%] distributed/tests/test_multi_locks.py::test_single_lock PASSED [ 50%] distributed/tests/test_multi_locks.py::test_timeout PASSED [ 50%] distributed/tests/test_multi_locks.py::test_timeout_wake_waiter PASSED [ 50%] distributed/tests/test_multi_locks.py::test_multiple_locks PASSED [ 50%] distributed/tests/test_multi_locks.py::test_num_locks PASSED [ 50%] distributed/tests/test_nanny.py::test_str PASSED [ 50%] distributed/tests/test_nanny.py::test_nanny_process_failure PASSED [ 50%] distributed/tests/test_nanny.py::test_run PASSED [ 50%] distributed/tests/test_nanny.py::test_no_hang_when_scheduler_closes SKIPPED [ 50%] distributed/tests/test_nanny.py::test_close_on_disconnect SKIPPED (need --runslow option to run) [ 50%] distributed/tests/test_nanny.py::test_nanny_worker_class PASSED [ 50%] distributed/tests/test_nanny.py::test_nanny_alt_worker_class PASSED [ 50%] distributed/tests/test_nanny.py::test_nanny_death_timeout SKIPPED (need --runslow option to run) [ 50%] distributed/tests/test_nanny.py::test_random_seed PASSED [ 50%] distributed/tests/test_nanny.py::test_num_fds PASSED [ 50%] distributed/tests/test_nanny.py::test_nanny_timeout PASSED [ 50%] distributed/tests/test_nanny.py::test_throttle_outgoing_transfers PASSED [ 50%] distributed/tests/test_nanny.py::test_scheduler_address_config PASSED [ 50%] distributed/tests/test_nanny.py::test_wait_for_scheduler SKIPPED (need --runslow option to run) [ 50%] distributed/tests/test_nanny.py::test_environment_variable PASSED [ 50%] distributed/tests/test_nanny.py::test_environment_variable_by_config PASSED [ 50%] distributed/tests/test_nanny.py::test_environment_variable_config PASSED [ 50%] distributed/tests/test_nanny.py::test_environment_variable_pre_post_spawn PASSED [ 50%] distributed/tests/test_nanny.py::test_environment_variable_overlay PASSED [ 50%] distributed/tests/test_nanny.py::test_config_param_overlays PASSED [ 50%] distributed/tests/test_nanny.py::test_local_directory PASSED [ 50%] distributed/tests/test_nanny.py::test_unwriteable_dask_worker_space PASSED [ 50%] distributed/tests/test_nanny.py::test_mp_process_worker_no_daemon PASSED [ 51%] distributed/tests/test_nanny.py::test_mp_pool_worker_no_daemon PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_closes_cleanly PASSED [ 51%] distributed/tests/test_nanny.py::test_lifetime SKIPPED (need --runslow option to run) [ 51%] distributed/tests/test_nanny.py::test_nanny_closes_cleanly_if_worker_is_terminated PASSED [ 51%] distributed/tests/test_nanny.py::test_config PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_port_range PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_closed_by_keyboard_interrupt[tcp] SKIPPED [ 51%] distributed/tests/test_nanny.py::test_nanny_closed_by_keyboard_interrupt[ucx] SKIPPED [ 51%] distributed/tests/test_nanny.py::test_worker_start_exception PASSED [ 51%] distributed/tests/test_nanny.py::test_failure_during_worker_initialization PASSED [ 51%] distributed/tests/test_nanny.py::test_environ_plugin PASSED [ 51%] distributed/tests/test_nanny.py::test_no_unnecessary_imports_on_worker[scipy] PASSED [ 51%] distributed/tests/test_nanny.py::test_no_unnecessary_imports_on_worker[pandas] XFAIL [ 51%] distributed/tests/test_nanny.py::test_repeated_restarts SKIPPED (need --runslow option to run) [ 51%] distributed/tests/test_nanny.py::test_restart_memory SKIPPED (need --runslow option to run) [ 51%] distributed/tests/test_nanny.py::test_close_joins SKIPPED (need --runslow option to run) [ 51%] distributed/tests/test_nanny.py::test_scheduler_crash_doesnt_restart PASSED [ 51%] distributed/tests/test_nanny.py::test_malloc_trim_threshold SKIPPED [ 51%] distributed/tests/test_nanny.py::test_default_client_does_not_propagate_to_subprocess PASSED [ 51%] distributed/tests/test_nanny.py::test_worker_inherits_temp_config PASSED [ 51%] distributed/tests/test_nanny.py::test_log_event PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_simple PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_during_start_success[True] PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_during_start_success[False] PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_during_start_failure[True] PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_during_start_failure[False] PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_nanny_killed[True] PASSED [ 51%] distributed/tests/test_nanny.py::test_nanny_plugin_register_nanny_killed[False] PASSED [ 51%] distributed/tests/test_preload.py::test_worker_preload_file PASSED [ 51%] distributed/tests/test_preload.py::test_worker_preload_config PASSED [ 51%] distributed/tests/test_preload.py::test_worker_preload_module PASSED [ 52%] distributed/tests/test_preload.py::test_worker_preload_click PASSED [ 52%] distributed/tests/test_preload.py::test_worker_preload_click_async PASSED [ 52%] distributed/tests/test_preload.py::test_web_preload PASSED [ 52%] distributed/tests/test_preload.py::test_scheduler_startup PASSED [ 52%] distributed/tests/test_preload.py::test_scheduler_startup_nanny PASSED [ 52%] distributed/tests/test_preload.py::test_web_preload_worker PASSED [ 52%] distributed/tests/test_preload.py::test_client_preload_text XFAIL (The preload argument to the client isn't supported yet) [ 52%] distributed/tests/test_preload.py::test_client_preload_config PASSED [ 52%] distributed/tests/test_preload.py::test_client_preload_click XFAIL (The preload argument to the client isn't supported yet) [ 52%] distributed/tests/test_preload.py::test_failure_doesnt_crash_worker PASSED [ 52%] distributed/tests/test_preload.py::test_failure_doesnt_crash_nanny PASSED [ 52%] distributed/tests/test_preload.py::test_failure_doesnt_crash_client PASSED [ 52%] distributed/tests/test_preload.py::test_client_preload_config_click PASSED [ 52%] distributed/tests/test_priorities.py::test_submit[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_submit[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_map[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_map[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_compute[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_compute[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_persist[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_persist[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_annotate_compute[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_annotate_compute[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_annotate_persist[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_annotate_persist[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_repeated_persists_same_priority[queue on worker] PASSED [ 52%] distributed/tests/test_priorities.py::test_repeated_persists_same_priority[queue on scheduler] PASSED [ 52%] distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] FAILED [ 52%] distributed/tests/test_priorities.py::test_last_in_first_out[queue on scheduler] PASSED [ 52%] distributed/tests/test_profile.py::test_basic PASSED [ 52%] distributed/tests/test_profile.py::test_basic_low_level SKIPPED (could not import 'stacktrace': No module named 'stacktrace') [ 53%] distributed/tests/test_profile.py::test_merge PASSED [ 53%] distributed/tests/test_profile.py::test_merge_empty PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack PASSED [ 53%] distributed/tests/test_profile.py::test_identifier PASSED [ 53%] distributed/tests/test_profile.py::test_watch PASSED [ 53%] distributed/tests/test_profile.py::test_watch_requires_lock_to_run PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[-1-1] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[0-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[1-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[11-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[12-3] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[21-4] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[22-4] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[23-4] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[24-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[25-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[26-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[27-2] PASSED [ 53%] distributed/tests/test_profile.py::test_info_frame_f_lineno[100-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[-1-1] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[0-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[1-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[11-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[12-3] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[21-4] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[22-4] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[23-4] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[24-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[25-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[26-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[27-2] PASSED [ 53%] distributed/tests/test_profile.py::test_call_stack_f_lineno[100-2] PASSED [ 54%] distributed/tests/test_profile.py::test_builtin PASSED [ 54%] distributed/tests/test_publish.py::test_publish_simple PASSED [ 54%] distributed/tests/test_publish.py::test_publish_non_string_key PASSED [ 54%] distributed/tests/test_publish.py::test_publish_roundtrip PASSED [ 54%] distributed/tests/test_publish.py::test_unpublish PASSED [ 54%] distributed/tests/test_publish.py::test_unpublish_sync PASSED [ 54%] distributed/tests/test_publish.py::test_publish_multiple_datasets PASSED [ 54%] distributed/tests/test_publish.py::test_unpublish_multiple_datasets_sync PASSED [ 54%] distributed/tests/test_publish.py::test_publish_bag PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_setitem PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_getitem PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_getitem_default PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_delitem PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_keys PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_contains PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_republish PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_iter PASSED [ 54%] distributed/tests/test_publish.py::test_datasets_async PASSED [ 54%] distributed/tests/test_publish.py::test_pickle_safe PASSED [ 54%] distributed/tests/test_publish.py::test_deserialize_client PASSED [ 54%] distributed/tests/test_publish.py::test_publish_submit_ordering PASSED [ 54%] distributed/tests/test_pubsub.py::test_speed PASSED [ 54%] distributed/tests/test_pubsub.py::test_client PASSED [ 54%] distributed/tests/test_pubsub.py::test_timeouts PASSED [ 54%] distributed/tests/test_pubsub.py::test_repr PASSED [ 54%] distributed/tests/test_pubsub.py::test_basic XPASS (flaky and re-fails on reruns; out of order execution) [ 54%] distributed/tests/test_queues.py::test_queue PASSED [ 54%] distributed/tests/test_queues.py::test_queue_with_data PASSED [ 54%] distributed/tests/test_queues.py::test_sync PASSED [ 54%] distributed/tests/test_queues.py::test_hold_futures PASSED [ 54%] distributed/tests/test_queues.py::test_picklability PASSED [ 55%] distributed/tests/test_queues.py::test_picklability_sync PASSED [ 55%] distributed/tests/test_queues.py::test_race SKIPPED (need --runslow option to run) [ 55%] distributed/tests/test_queues.py::test_same_futures PASSED [ 55%] distributed/tests/test_queues.py::test_get_many PASSED [ 55%] distributed/tests/test_queues.py::test_Future_knows_status_immediately PASSED [ 55%] distributed/tests/test_queues.py::test_erred_future PASSED [ 55%] distributed/tests/test_queues.py::test_close PASSED [ 55%] distributed/tests/test_queues.py::test_timeout PASSED [ 55%] distributed/tests/test_queues.py::test_2220 PASSED [ 55%] distributed/tests/test_queues.py::test_unpickle_without_client PASSED [ 55%] distributed/tests/test_reschedule.py::test_scheduler_reschedule_warns PASSED [ 55%] distributed/tests/test_reschedule.py::test_raise_reschedule[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_raise_reschedule[long-running] PASSED [ 55%] distributed/tests/test_reschedule.py::test_cancelled_reschedule[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_cancelled_reschedule[long-running] PASSED [ 55%] distributed/tests/test_reschedule.py::test_cancelled_reschedule_worker_state[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_cancelled_reschedule_worker_state[long-running] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_releases[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_releases[long-running] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_cancelled[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_cancelled[long-running] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_resumed[executing] PASSED [ 55%] distributed/tests/test_reschedule.py::test_reschedule_resumed[long-running] PASSED [ 55%] distributed/tests/test_resources.py::test_resource_submit PASSED [ 55%] distributed/tests/test_resources.py::test_submit_many_non_overlapping PASSED [ 55%] distributed/tests/test_resources.py::test_submit_many_non_overlapping_2 PASSED [ 55%] distributed/tests/test_resources.py::test_move PASSED [ 55%] distributed/tests/test_resources.py::test_dont_work_steal PASSED [ 55%] distributed/tests/test_resources.py::test_map PASSED [ 55%] distributed/tests/test_resources.py::test_persist PASSED [ 55%] distributed/tests/test_resources.py::test_compute PASSED [ 56%] distributed/tests/test_resources.py::test_get PASSED [ 56%] distributed/tests/test_resources.py::test_persist_multiple_collections PASSED [ 56%] distributed/tests/test_resources.py::test_resources_str PASSED [ 56%] distributed/tests/test_resources.py::test_minimum_resource PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_1[1-0-y-False] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_1[1-0-y-True] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_1[0-1-x-False] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_1[0-1-x-True] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_2[1-0-y-False] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_2[1-0-y-True] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_2[0-1-x-False] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_vs_ready_priority_2[0-1-x-True] PASSED [ 56%] distributed/tests/test_resources.py::test_constrained_tasks_respect_priority PASSED [ 56%] distributed/tests/test_resources.py::test_task_cancelled_and_readded_with_resources PASSED [ 56%] distributed/tests/test_resources.py::test_balance_resources SKIPPED [ 56%] distributed/tests/test_resources.py::test_set_resources PASSED [ 56%] distributed/tests/test_resources.py::test_persist_collections PASSED [ 56%] distributed/tests/test_resources.py::test_dont_optimize_out SKIPPED [ 56%] distributed/tests/test_resources.py::test_full_collections SKIPPED (atop fusion seemed to break this) [ 56%] distributed/tests/test_resources.py::test_collections_get[True] XFAIL [ 56%] distributed/tests/test_resources.py::test_collections_get[False] PASSED [ 56%] distributed/tests/test_resources.py::test_resources_from_config PASSED [ 56%] distributed/tests/test_resources.py::test_resources_from_python_override_config PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[executing-ExecuteSuccessEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[executing-ExecuteFailureEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[executing-RescheduleEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[long-running-ExecuteSuccessEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[long-running-ExecuteFailureEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_cancelled_with_resources[long-running-RescheduleEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_resumed_with_resources[executing-ExecuteSuccessEvent] PASSED [ 56%] distributed/tests/test_resources.py::test_resumed_with_resources[executing-ExecuteFailureEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_resources[executing-RescheduleEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_resources[long-running-ExecuteSuccessEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_resources[long-running-ExecuteFailureEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_resources[long-running-RescheduleEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[executing-ExecuteSuccessEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[executing-ExecuteFailureEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[executing-RescheduleEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[long-running-ExecuteSuccessEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[long-running-ExecuteFailureEvent] PASSED [ 57%] distributed/tests/test_resources.py::test_resumed_with_different_resources[long-running-RescheduleEvent] PASSED [ 57%] distributed/tests/test_scheduler.py::test_administration PASSED [ 57%] distributed/tests/test_scheduler.py::test_respect_data_in_memory PASSED [ 57%] distributed/tests/test_scheduler.py::test_recompute_released_results PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_with_many_independent_leaves PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_with_restrictions PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads0-0] PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads0-1] PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads0-4] PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads1-0] PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads1-1] PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_coschedule_order_neighbors[nthreads1-4] PASSED [ 57%] distributed/tests/test_scheduler.py::test_override_is_rootish PASSED [ 57%] distributed/tests/test_scheduler.py::test_decide_worker_rootish_while_last_worker_is_retiring SKIPPEDissues/7204) [ 57%] distributed/tests/test_scheduler.py::test_graph_execution_width PASSED [ 57%] distributed/tests/test_scheduler.py::test_forget_tasks_while_processing PASSED [ 57%] distributed/tests/test_scheduler.py::test_restart_while_processing SKIPPED [ 57%] distributed/tests/test_scheduler.py::test_queued_release_multiple_workers PASSED [ 57%] distributed/tests/test_scheduler.py::test_queued_paused_new_worker PASSED [ 57%] distributed/tests/test_scheduler.py::test_queued_paused_unpaused[True] PASSED [ 57%] distributed/tests/test_scheduler.py::test_queued_paused_unpaused[False] PASSED [ 57%] distributed/tests/test_scheduler.py::test_queued_remove_add_worker PASSED [ 57%] distributed/tests/test_scheduler.py::test_secede_opens_slot PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[2.5-expected_task_counts0] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[2.0-expected_task_counts1] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[1.1-expected_task_counts2] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[1.0-expected_task_counts3] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[0.1-expected_task_counts4] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[inf-expected_task_counts5] PASSED [ 58%] distributed/tests/test_scheduler.py::test_saturation_factor[inf-expected_task_counts6] PASSED [ 58%] distributed/tests/test_scheduler.py::test_bad_saturation_factor PASSED [ 58%] distributed/tests/test_scheduler.py::test_move_data_over_break_restrictions PASSED [ 58%] distributed/tests/test_scheduler.py::test_balance_with_restrictions PASSED [ 58%] distributed/tests/test_scheduler.py::test_no_valid_workers PASSED [ 58%] distributed/tests/test_scheduler.py::test_no_valid_workers_loose_restrictions PASSED [ 58%] distributed/tests/test_scheduler.py::test_no_workers[False] PASSED [ 58%] distributed/tests/test_scheduler.py::test_no_workers[True] PASSED [ 58%] distributed/tests/test_scheduler.py::test_retire_workers_empty PASSED [ 58%] distributed/tests/test_scheduler.py::test_retire_workers_lock PASSED [ 58%] distributed/tests/test_scheduler.py::test_retire_workers_concurrently PASSED [ 58%] distributed/tests/test_scheduler.py::test_server_listens_to_other_ops PASSED [ 58%] distributed/tests/test_scheduler.py::test_remove_worker_from_scheduler PASSED [ 58%] distributed/tests/test_scheduler.py::test_remove_worker_by_name_from_scheduler PASSED [ 58%] distributed/tests/test_scheduler.py::test_clear_events_worker_removal PASSED [ 58%] distributed/tests/test_scheduler.py::test_clear_events_client_removal PASSED [ 58%] distributed/tests/test_scheduler.py::test_add_worker PASSED [ 58%] distributed/tests/test_scheduler.py::test_blocked_handlers_are_respected PASSED [ 58%] distributed/tests/test_scheduler.py::test_scheduler_init_pulls_blocked_handlers_from_config PASSED [ 58%] distributed/tests/test_scheduler.py::test_feed PASSED [ 58%] distributed/tests/test_scheduler.py::test_feed_setup_teardown PASSED [ 58%] distributed/tests/test_scheduler.py::test_feed_large_bytestring PASSED [ 58%] distributed/tests/test_scheduler.py::test_delete_data PASSED [ 58%] distributed/tests/test_scheduler.py::test_delete PASSED [ 58%] distributed/tests/test_scheduler.py::test_dumps_function PASSED [ 59%] distributed/tests/test_scheduler.py::test_ready_remove_worker[1.0] PASSED [ 59%] distributed/tests/test_scheduler.py::test_ready_remove_worker[inf] PASSED [ 59%] distributed/tests/test_scheduler.py::test_restart SKIPPED (need --runslow option to run) [ 59%] distributed/tests/test_scheduler.py::test_restart_waits_for_new_workers SKIPPED [ 59%] distributed/tests/test_scheduler.py::test_restart_nanny_timeout_exceeded SKIPPED [ 59%] distributed/tests/test_scheduler.py::test_restart_not_all_workers_return PASSED [ 59%] distributed/tests/test_scheduler.py::test_restart_worker_rejoins_after_timeout_expired PASSED [ 59%] distributed/tests/test_scheduler.py::test_restart_no_wait_for_workers PASSED [ 59%] distributed/tests/test_scheduler.py::test_restart_some_nannies_some_not SKIPPED [ 59%] distributed/tests/test_scheduler.py::test_restart_heartbeat_before_closing SKIPPED [ 59%] distributed/tests/test_scheduler.py::test_broadcast PASSED [ 59%] distributed/tests/test_scheduler.py::test_broadcast_tls PASSED [ 59%] distributed/tests/test_scheduler.py::test_broadcast_nanny PASSED [ 59%] distributed/tests/test_scheduler.py::test_broadcast_on_error PASSED [ 59%] distributed/tests/test_scheduler.py::test_broadcast_deprecation PASSED [ 59%] distributed/tests/test_scheduler.py::test_worker_name PASSED [ 59%] distributed/tests/test_scheduler.py::test_coerce_address PASSED [ 59%] distributed/tests/test_scheduler.py::test_config_stealing PASSED [ 59%] distributed/tests/test_scheduler.py::test_config_no_stealing PASSED [ 59%] distributed/tests/test_scheduler.py::test_update_graph_culls PASSED [ 59%] distributed/tests/test_scheduler.py::test_io_loop PASSED [ 59%] distributed/tests/test_scheduler.py::test_story PASSED [ 59%] distributed/tests/test_scheduler.py::test_scatter_no_workers[False] PASSED [ 59%] distributed/tests/test_scheduler.py::test_scatter_no_workers[True] PASSED [ 59%] distributed/tests/test_scheduler.py::test_scheduler_sees_memory_limits PASSED [ 59%] distributed/tests/test_scheduler.py::test_retire_workers PASSED [ 59%] distributed/tests/test_scheduler.py::test_retire_workers_n PASSED [ 59%] distributed/tests/test_scheduler.py::test_workers_to_close PASSED [ 59%] distributed/tests/test_scheduler.py::test_workers_to_close_grouped PASSED [ 59%] distributed/tests/test_scheduler.py::test_workers_to_close_never_close_long_running[True] PASSED [ 59%] distributed/tests/test_scheduler.py::test_workers_to_close_never_close_long_running[False] PASSED [ 60%] distributed/tests/test_scheduler.py::test_retire_workers_no_suspicious_tasks PASSED [ 60%] distributed/tests/test_scheduler.py::test_file_descriptors SKIPPED (need --runslow option to run) [ 60%] distributed/tests/test_scheduler.py::test_learn_occupancy SKIPPED (need --runslow option to run) [ 60%] distributed/tests/test_scheduler.py::test_learn_occupancy_2 SKIPPED [ 60%] distributed/tests/test_scheduler.py::test_balance_many_workers PASSED [ 60%] distributed/tests/test_scheduler.py::test_balance_many_workers_2 PASSED [ 60%] distributed/tests/test_scheduler.py::test_include_communication_in_occupancy PASSED [ 60%] distributed/tests/test_scheduler.py::test_nonempty_data_is_rejected PASSED [ 60%] distributed/tests/test_scheduler.py::test_worker_arrives_with_data_is_rejected PASSED [ 60%] distributed/tests/test_scheduler.py::test_run_on_scheduler_sync PASSED [ 60%] distributed/tests/test_scheduler.py::test_run_on_scheduler PASSED [ 60%] distributed/tests/test_scheduler.py::test_allow_pickle_false PASSED [ 60%] distributed/tests/test_scheduler.py::test_close_worker PASSED [ 60%] distributed/tests/test_scheduler.py::test_close_nanny SKIPPED (need --runslow option to run) [ 60%] distributed/tests/test_scheduler.py::test_retire_workers_close PASSED [ 60%] distributed/tests/test_scheduler.py::test_retire_nannies_close SKIPPED [ 60%] distributed/tests/test_scheduler.py::test_fifo_submission PASSED [ 60%] distributed/tests/test_scheduler.py::test_profile_metadata PASSED [ 60%] distributed/tests/test_scheduler.py::test_profile_metadata_timeout PASSED [ 60%] distributed/tests/test_scheduler.py::test_profile_metadata_keys PASSED [ 60%] distributed/tests/test_scheduler.py::test_statistical_profiling PASSED [ 60%] distributed/tests/test_scheduler.py::test_statistical_profiling_failure PASSED [ 60%] distributed/tests/test_scheduler.py::test_cancel_fire_and_forget PASSED [ 60%] distributed/tests/test_scheduler.py::test_log_tasks_during_restart SKIPPED [ 60%] distributed/tests/test_scheduler.py::test_get_task_status PASSED [ 60%] distributed/tests/test_scheduler.py::test_deque_handler PASSED [ 60%] distributed/tests/test_scheduler.py::test_retries PASSED [ 60%] distributed/tests/test_scheduler.py::test_missing_data_errant_worker PASSED [ 60%] distributed/tests/test_scheduler.py::test_dont_recompute_if_persisted PASSED [ 60%] distributed/tests/test_scheduler.py::test_dont_recompute_if_persisted_2 PASSED [ 60%] distributed/tests/test_scheduler.py::test_dont_recompute_if_persisted_3 PASSED [ 60%] distributed/tests/test_scheduler.py::test_dont_recompute_if_persisted_4 PASSED [ 61%] distributed/tests/test_scheduler.py::test_dont_forget_released_keys PASSED [ 61%] distributed/tests/test_scheduler.py::test_dont_recompute_if_erred PASSED [ 61%] distributed/tests/test_scheduler.py::test_closing_scheduler_closes_workers PASSED [ 61%] distributed/tests/test_scheduler.py::test_resources_reset_after_cancelled_task PASSED [ 61%] distributed/tests/test_scheduler.py::test_gh2187 PASSED [ 61%] distributed/tests/test_scheduler.py::test_collect_versions PASSED [ 61%] distributed/tests/test_scheduler.py::test_idle_timeout SKIPPED (need --runslow option to run) [ 61%] distributed/tests/test_scheduler.py::test_idle_timeout_no_workers PASSED [ 61%] distributed/tests/test_scheduler.py::test_no_workers_timeout_disabled PASSED [ 61%] distributed/tests/test_scheduler.py::test_no_workers_timeout_without_workers SKIPPED [ 61%] distributed/tests/test_scheduler.py::test_no_workers_timeout_bad_restrictions SKIPPED [ 61%] distributed/tests/test_scheduler.py::test_no_workers_timeout_queued PASSED [ 61%] distributed/tests/test_scheduler.py::test_no_workers_timeout_processing SKIPPED [ 61%] distributed/tests/test_scheduler.py::test_bandwidth PASSED [ 61%] distributed/tests/test_scheduler.py::test_bandwidth_clear SKIPPED (need --runslow option to run) [ 61%] distributed/tests/test_scheduler.py::test_workerstate_clean PASSED [ 61%] distributed/tests/test_scheduler.py::test_result_type PASSED [ 61%] distributed/tests/test_scheduler.py::test_close_workers PASSED [ 61%] distributed/tests/test_scheduler.py::test_host_address PASSED [ 61%] distributed/tests/test_scheduler.py::test_dashboard_address SKIPPED [ 61%] distributed/tests/test_scheduler.py::test_adaptive_target PASSED [ 61%] distributed/tests/test_scheduler.py::test_adaptive_target_empty_cluster[True] PASSED [ 61%] distributed/tests/test_scheduler.py::test_adaptive_target_empty_cluster[False] PASSED [ 61%] distributed/tests/test_scheduler.py::test_retire_names_str PASSED [ 61%] distributed/tests/test_scheduler.py::test_retire_workers_bad_params PASSED [ 61%] distributed/tests/test_scheduler.py::test_get_task_duration PASSED [ 61%] distributed/tests/test_scheduler.py::test_default_task_duration_splits PASSED [ 61%] distributed/tests/test_scheduler.py::test_task_groups PASSED [ 61%] distributed/tests/test_scheduler.py::test_task_groups_update_start_stop PASSED [ 61%] distributed/tests/test_scheduler.py::test_task_group_done PASSED [ 61%] distributed/tests/test_scheduler.py::test_task_group_not_done_waiting PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_group_not_done_noworker PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_group_not_done_queued PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_group_not_done_processing PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_prefix PASSED [ 62%] distributed/tests/test_scheduler.py::test_failing_task_increments_suspicious SKIPPED [ 62%] distributed/tests/test_scheduler.py::test_task_group_non_tuple_key PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_unique_groups PASSED [ 62%] distributed/tests/test_scheduler.py::test_task_group_on_fire_and_forget PASSED [ 62%] distributed/tests/test_scheduler.py::test_gather_failing_can_recover PASSED [ 62%] distributed/tests/test_scheduler.py::test_gather_failing_cnn_error PASSED [ 62%] distributed/tests/test_scheduler.py::test_gather_no_workers PASSED [ 62%] distributed/tests/test_scheduler.py::test_gather_bad_worker[False] PASSED [ 62%] distributed/tests/test_scheduler.py::test_gather_bad_worker[True] PASSED [ 62%] distributed/tests/test_scheduler.py::test_too_many_groups PASSED [ 62%] distributed/tests/test_scheduler.py::test_worker_name_collision PASSED [ 62%] distributed/tests/test_scheduler.py::test_unknown_task_duration_config PASSED [ 62%] distributed/tests/test_scheduler.py::test_unknown_task_duration_config_2 PASSED [ 62%] distributed/tests/test_scheduler.py::test_retire_state_change PASSED [ 62%] distributed/tests/test_scheduler.py::test_configurable_events_log_length PASSED [ 62%] distributed/tests/test_scheduler.py::test_get_worker_monitor_info PASSED [ 62%] distributed/tests/test_scheduler.py::test_quiet_cluster_round_robin PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_sum PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-0-0] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-0-1] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-0-2] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-0-3] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-1-0] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-1-1] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-1-2] PASSED [ 62%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-1-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-2-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-2-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-2-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-2-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-3-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-3-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-3-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-0-3-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-0-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-0-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-0-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-0-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-1-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-1-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-1-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-1-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-2-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-2-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-2-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-2-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-3-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-3-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-3-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-1-3-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-0-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-0-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-0-2] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-0-3] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-1-0] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-1-1] PASSED [ 63%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-1-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-1-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-2-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-2-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-2-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-2-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-3-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-3-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-3-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-2-3-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-0-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-0-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-0-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-0-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-1-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-1-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-1-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-1-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-2-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-2-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-2-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-2-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-3-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-3-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-3-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[0-3-3-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-0-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-0-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-0-2] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-0-3] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-1-0] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-1-1] PASSED [ 64%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-1-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-1-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-2-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-2-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-2-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-2-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-3-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-3-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-3-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-0-3-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-0-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-0-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-0-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-0-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-1-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-1-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-1-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-1-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-2-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-2-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-2-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-2-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-3-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-3-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-3-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-1-3-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-0-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-0-1] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-0-2] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-0-3] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-1-0] PASSED [ 65%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-1-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-1-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-1-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-2-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-2-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-2-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-2-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-3-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-3-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-3-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-2-3-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-0-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-0-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-0-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-0-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-1-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-1-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-1-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-1-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-2-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-2-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-2-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-2-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-3-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-3-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-3-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[1-3-3-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-0-0] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-0-1] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-0-2] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-0-3] PASSED [ 66%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-1-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-1-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-1-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-1-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-2-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-2-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-2-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-2-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-3-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-3-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-3-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-0-3-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-0-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-0-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-0-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-0-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-1-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-1-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-1-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-1-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-2-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-2-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-2-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-2-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-3-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-3-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-3-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-1-3-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-0-0] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-0-1] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-0-2] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-0-3] PASSED [ 67%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-1-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-1-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-1-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-1-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-2-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-2-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-2-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-2-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-3-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-3-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-3-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-2-3-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-0-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-0-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-0-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-0-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-1-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-1-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-1-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-1-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-2-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-2-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-2-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-2-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-3-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-3-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-3-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[2-3-3-3] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-0-0] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-0-1] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-0-2] PASSED [ 68%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-0-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-1-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-1-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-1-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-1-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-2-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-2-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-2-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-2-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-3-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-3-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-3-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-0-3-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-0-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-0-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-0-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-0-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-1-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-1-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-1-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-1-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-2-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-2-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-2-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-2-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-3-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-3-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-3-2] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-1-3-3] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-0-0] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-0-1] PASSED [ 69%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-0-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-0-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-1-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-1-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-1-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-1-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-2-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-2-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-2-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-2-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-3-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-3-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-3-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-2-3-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-0-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-0-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-0-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-0-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-1-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-1-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-1-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-1-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-2-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-2-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-2-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-2-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-3-0] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-3-1] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-3-2] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate_adds_up[3-3-3-3] PASSED [ 70%] distributed/tests/test_scheduler.py::test_memorystate__to_dict PASSED [ 70%] distributed/tests/test_scheduler.py::test_memory SKIPPED (need --runslow option to run) [ 71%] distributed/tests/test_scheduler.py::test_memory_no_zict PASSED [ 71%] distributed/tests/test_scheduler.py::test_memory_no_workers PASSED [ 71%] distributed/tests/test_scheduler.py::test_infrequent_sysmon PASSED [ 71%] distributed/tests/test_scheduler.py::test_close_scheduler__close_workers_Worker PASSED [ 71%] distributed/tests/test_scheduler.py::test_close_scheduler__close_workers_Nanny SKIPPED [ 71%] distributed/tests/test_scheduler.py::test_rebalance SKIPPED (need --runslow option to run) [ 71%] distributed/tests/test_scheduler.py::test_rebalance_managed_memory PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_workers_and_keys PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_missing_data1 PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_missing_data2 PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_raises_missing_data3[False] PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_raises_missing_data3[True] XFAIL explicit RPC). Therefore, the partial-fail is very timing sensitive and subject to a race condition. This test assumes that the data is freed before the rebalance get_data requests come in but merely deleting the futures is not sufficient to guarantee this) [ 71%] distributed/tests/test_scheduler.py::test_rebalance_no_workers PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_no_limit PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_no_recipients SKIPPED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_skip_recipient PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_skip_all_recipients PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_sender_below_mean SKIPPED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_least_recently_inserted_sender_min SKIPPED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_bad_recipient PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_bad_sender PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_bad_sender_replicated[False] PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_bad_sender_replicated[True] PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_key_not_on_sender PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_key_not_on_sender_replicated[False] PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_key_not_on_sender_replicated[True] PASSED [ 71%] distributed/tests/test_scheduler.py::test_gather_on_worker_duplicate_task PASSED [ 71%] distributed/tests/test_scheduler.py::test_rebalance_dead_recipient PASSED [ 71%] distributed/tests/test_scheduler.py::test_delete_worker_data PASSED [ 71%] distributed/tests/test_scheduler.py::test_delete_worker_data_double_delete PASSED [ 71%] distributed/tests/test_scheduler.py::test_delete_worker_data_bad_worker PASSED [ 72%] distributed/tests/test_scheduler.py::test_delete_worker_data_bad_task[False] PASSED [ 72%] distributed/tests/test_scheduler.py::test_delete_worker_data_bad_task[True] PASSED [ 72%] distributed/tests/test_scheduler.py::test_transition_counter PASSED [ 72%] distributed/tests/test_scheduler.py::test_transition_counter_max_scheduler SKIPPED [ 72%] distributed/tests/test_scheduler.py::test_transition_counter_max_worker PASSED [ 72%] distributed/tests/test_scheduler.py::test_disable_transition_counter_max PASSED [ 72%] distributed/tests/test_scheduler.py::test_worker_heartbeat_after_cancel PASSED [ 72%] distributed/tests/test_scheduler.py::test_set_restrictions PASSED [ 72%] distributed/tests/test_scheduler.py::test_avoid_paused_workers PASSED [ 72%] distributed/tests/test_scheduler.py::test_Scheduler__to_dict PASSED [ 72%] distributed/tests/test_scheduler.py::test_TaskState__to_dict PASSED [ 72%] distributed/tests/test_scheduler.py::test_get_cluster_state PASSED [ 72%] distributed/tests/test_scheduler.py::test_get_cluster_state_worker_error PASSED [ 72%] distributed/tests/test_scheduler.py::test_dump_cluster_state[msgpack] PASSED [ 72%] distributed/tests/test_scheduler.py::test_dump_cluster_state[yaml] PASSED [ 72%] distributed/tests/test_scheduler.py::test_repr PASSED [ 72%] distributed/tests/test_scheduler.py::test_ensure_events_dont_include_taskstate_objects PASSED [ 72%] distributed/tests/test_scheduler.py::test_worker_state_unique_regardless_of_address PASSED [ 72%] distributed/tests/test_scheduler.py::test_scheduler_close_fast_deprecated PASSED [ 72%] distributed/tests/test_scheduler.py::test_runspec_regression_sync PASSED [ 72%] distributed/tests/test_scheduler.py::test_KilledWorker_informative_message PASSED [ 72%] distributed/tests/test_scheduler.py::test_count_task_prefix PASSED [ 72%] distributed/tests/test_scheduler.py::test_transition_waiting_memory PASSED [ 72%] distributed/tests/test_scheduler.py::test_deadlock_resubmit_queued_tasks_fast[True] PASSED [ 72%] distributed/tests/test_scheduler.py::test_deadlock_resubmit_queued_tasks_fast[False] PASSED [ 72%] distributed/tests/test_scheduler.py::test_submit_dependency_of_erred_task PASSED [ 72%] distributed/tests/test_scheduler.py::test_tell_workers_when_peers_have_left PASSED [ 72%] distributed/tests/test_scheduler.py::test_client_desires_keys_creates_ts PASSED [ 72%] distributed/tests/test_scheduler.py::test_scatter_creates_ts PASSED [ 72%] distributed/tests/test_scheduler.py::test_refuse_to_schedule_huge_task[False] PASSED [ 72%] distributed/tests/test_scheduler.py::test_refuse_to_schedule_huge_task[True] PASSED [ 73%] distributed/tests/test_scheduler.py::test_html_repr PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_before_previous_is_done[same] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_before_previous_is_done[less] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_before_previous_is_done[more] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[False-same] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[False-less] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[False-more] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[True-same] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[True-less] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_after_previous_is_done[True-more] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_many_clients PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_nondeterministic_task_same_deps[before0-123-True] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_nondeterministic_task_same_deps[123-after1-True] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_nondeterministic_task_same_deps[before2-after2-False] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_nondeterministic_task_different_deps[False] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_nondeterministic_task_different_deps[True] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_warns_only_once[10-2] PASSED [ 73%] distributed/tests/test_scheduler.py::test_resubmit_different_task_same_key_warns_only_once[30-1] PASSED [ 73%] distributed/tests/test_scheduler.py::test_fan_out_pattern_deadlock PASSED [ 73%] distributed/tests/test_scheduler.py::test_stimulus_from_erred_task PASSED [ 73%] distributed/tests/test_security.py::test_defaults PASSED [ 73%] distributed/tests/test_security.py::test_constructor_errors PASSED [ 73%] distributed/tests/test_security.py::test_attribute_error PASSED [ 73%] distributed/tests/test_security.py::test_from_config PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[None-None] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[None-1.2] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[None-1.3] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[1.2-None] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[1.2-1.2] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[1.2-1.3] PASSED [ 73%] distributed/tests/test_security.py::test_min_max_version_from_config[1.3-None] PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_from_config[1.3-1.2] PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_from_config[1.3-1.3] PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_config_errors[min-version] PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_config_errors[max-version] PASSED [ 74%] distributed/tests/test_security.py::test_invalid_min_version_from_config_errors PASSED [ 74%] distributed/tests/test_security.py::test_kwargs PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_kwarg_errors[tls_min_version] PASSED [ 74%] distributed/tests/test_security.py::test_min_max_version_kwarg_errors[tls_max_version] PASSED [ 74%] distributed/tests/test_security.py::test_repr_temp_keys PASSED [ 74%] distributed/tests/test_security.py::test_repr_local_keys PASSED [ 74%] distributed/tests/test_security.py::test_tls_config_for_role PASSED [ 74%] distributed/tests/test_security.py::test_connection_args PASSED [ 74%] distributed/tests/test_security.py::test_extra_conn_args_connection_args PASSED [ 74%] distributed/tests/test_security.py::test_listen_args PASSED [ 74%] distributed/tests/test_security.py::test_temporary_credentials PASSED [ 74%] distributed/tests/test_security.py::test_extra_conn_args_in_temporary_credentials PASSED [ 74%] distributed/tests/test_semaphore.py::test_semaphore_trivial PASSED [ 74%] distributed/tests/test_semaphore.py::test_serializable PASSED [ 74%] distributed/tests/test_semaphore.py::test_release_simple PASSED [ 74%] distributed/tests/test_semaphore.py::test_acquires_with_timeout PASSED [ 74%] distributed/tests/test_semaphore.py::test_timeout_sync PASSED [ 74%] distributed/tests/test_semaphore.py::test_release_semaphore_after_timeout PASSED [ 74%] distributed/tests/test_semaphore.py::test_async_ctx PASSED [ 74%] distributed/tests/test_semaphore.py::test_worker_dies PASSED [ 74%] distributed/tests/test_semaphore.py::test_access_semaphore_by_name PASSED [ 74%] distributed/tests/test_semaphore.py::test_close_async PASSED [ 74%] distributed/tests/test_semaphore.py::test_close_sync PASSED [ 74%] distributed/tests/test_semaphore.py::test_release_once_too_many PASSED [ 74%] distributed/tests/test_semaphore.py::test_release_once_too_many_resilience PASSED [ 74%] distributed/tests/test_semaphore.py::test_retry_acquire PASSED [ 74%] distributed/tests/test_semaphore.py::test_oversubscribing_leases PASSED [ 75%] distributed/tests/test_semaphore.py::test_timeout_zero PASSED [ 75%] distributed/tests/test_semaphore.py::test_getvalue PASSED [ 75%] distributed/tests/test_semaphore.py::test_metrics PASSED [ 75%] distributed/tests/test_semaphore.py::test_release_retry PASSED [ 75%] distributed/tests/test_semaphore.py::test_release_failure PASSED [ 75%] distributed/tests/test_semaphore.py::test_unpickle_without_client PASSED [ 75%] distributed/tests/test_sizeof.py::test_safe_sizeof[obj0] PASSED [ 75%] distributed/tests/test_sizeof.py::test_safe_sizeof[obj1] PASSED [ 75%] distributed/tests/test_sizeof.py::test_safe_sizeof[obj2] PASSED [ 75%] distributed/tests/test_sizeof.py::test_safe_sizeof_logs_on_failure PASSED [ 75%] distributed/tests/test_spans.py::test_spans PASSED [ 75%] distributed/tests/test_spans.py::test_submit PASSED [ 75%] distributed/tests/test_spans.py::test_multiple_tags PASSED [ 75%] distributed/tests/test_spans.py::test_repeat_span PASSED [ 75%] distributed/tests/test_spans.py::test_default_span PASSED [ 75%] distributed/tests/test_spans.py::test_no_extension PASSED [ 75%] distributed/tests/test_spans.py::test_task_groups[False] PASSED [ 75%] distributed/tests/test_spans.py::test_task_groups[True] PASSED [ 75%] distributed/tests/test_spans.py::test_before_first_task_finished PASSED [ 75%] distributed/tests/test_spans.py::test_duplicate_task_group PASSED [ 75%] distributed/tests/test_spans.py::test_mismatched_span[False] PASSED [ 75%] distributed/tests/test_spans.py::test_mismatched_span[True] PASSED [ 75%] distributed/tests/test_spans.py::test_no_tags PASSED [ 75%] distributed/tests/test_spans.py::test_client_desires_keys_creates_ts PASSED [ 75%] distributed/tests/test_spans.py::test_client_desires_keys_creates_tg PASSED [ 75%] distributed/tests/test_spans.py::test_scatter_creates_ts PASSED [ 75%] distributed/tests/test_spans.py::test_scatter_creates_tg PASSED [ 75%] distributed/tests/test_spans.py::test_worker_metrics PASSED [ 75%] distributed/tests/test_spans.py::test_merge_by_tags PASSED [ 75%] distributed/tests/test_spans.py::test_merge_by_tags_metrics PASSED [ 75%] distributed/tests/test_spans.py::test_code PASSED [ 75%] distributed/tests/test_spans.py::test_no_code_by_default PASSED [ 76%] distributed/tests/test_spans.py::test_merge_all PASSED [ 76%] distributed/tests/test_spans.py::test_merge_nothing PASSED [ 76%] distributed/tests/test_spans.py::test_active_cpu_seconds_trivial PASSED [ 76%] distributed/tests/test_spans.py::test_active_cpu_seconds_not_done[False] PASSED [ 76%] distributed/tests/test_spans.py::test_active_cpu_seconds_not_done[True] PASSED [ 76%] distributed/tests/test_spans.py::test_active_cpu_seconds_change_nthreads PASSED [ 76%] distributed/tests/test_spans.py::test_active_cpu_seconds_merged PASSED [ 76%] distributed/tests/test_spans.py::test_spans_are_visible_from_tasks PASSED [ 76%] distributed/tests/test_spans.py::test_span_on_persist PASSED [ 76%] distributed/tests/test_spans.py::test_collections_metadata XPASS (Test is flaky) [ 76%] distributed/tests/test_spec.py::test_address_default_none PASSED [ 76%] distributed/tests/test_spec.py::test_child_address_persists PASSED [ 76%] distributed/tests/test_spill.py::test_psize PASSED [ 76%] distributed/tests/test_spill.py::test_spillbuffer PASSED [ 76%] distributed/tests/test_spill.py::test_disk_size_calculation PASSED [ 76%] distributed/tests/test_spill.py::test_spillbuffer_maxlim PASSED [ 76%] distributed/tests/test_spill.py::test_spillbuffer_fail_to_serialize PASSED [ 76%] distributed/tests/test_spill.py::test_spillbuffer_evict PASSED [ 76%] distributed/tests/test_spill.py::test_no_pop PASSED [ 76%] distributed/tests/test_spill.py::test_weakref_cache[60-SupportsWeakRef-True] PASSED [ 76%] distributed/tests/test_spill.py::test_weakref_cache[60-NoWeakRef-False] PASSED [ 76%] distributed/tests/test_spill.py::test_weakref_cache[110-SupportsWeakRef-True] PASSED [ 76%] distributed/tests/test_spill.py::test_weakref_cache[110-NoWeakRef-False] PASSED [ 76%] distributed/tests/test_spill.py::test_metrics PASSED [ 76%] distributed/tests/test_spill.py::test_compression_settings[zlib-100-500] PASSED [ 76%] distributed/tests/test_spill.py::test_compression_settings[None-20000-20500] PASSED [ 76%] distributed/tests/test_spill.py::test_str_collision PASSED [ 76%] distributed/tests/test_steal.py::test_work_stealing PASSED [ 76%] distributed/tests/test_steal.py::test_dont_steal_expensive_data_fast_computation PASSED [ 76%] distributed/tests/test_steal.py::test_steal_cheap_data_slow_computation PASSED [ 76%] distributed/tests/test_steal.py::test_steal_expensive_data_slow_computation SKIPPED [ 77%] distributed/tests/test_steal.py::test_worksteal_many_thieves PASSED [ 77%] distributed/tests/test_steal.py::test_stop_plugin PASSED [ 77%] distributed/tests/test_steal.py::test_stop_in_flight PASSED [ 77%] distributed/tests/test_steal.py::test_allow_tasks_stolen_before_first_completes PASSED [ 77%] distributed/tests/test_steal.py::test_eventually_steal_unknown_functions PASSED [ 77%] distributed/tests/test_steal.py::test_steal_related_tasks SKIPPED [ 77%] distributed/tests/test_steal.py::test_dont_steal_fast_tasks_compute_time PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_fast_tasks_blocklist PASSED [ 77%] distributed/tests/test_steal.py::test_new_worker_steals PASSED [ 77%] distributed/tests/test_steal.py::test_work_steal_allow_other_workers PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_worker_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_steal_worker_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_host_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_steal_host_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_resource_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_steal_resource_restrictions PASSED [ 77%] distributed/tests/test_steal.py::test_steal_resource_restrictions_asym_diff PASSED [ 77%] distributed/tests/test_steal.py::test_balance_without_dependencies PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_executing_tasks PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_executing_tasks_2 PASSED [ 77%] distributed/tests/test_steal.py::test_dont_steal_few_saturated_tasks_many_workers PASSED [ 77%] distributed/tests/test_steal.py::test_steal_when_more_tasks PASSED [ 77%] distributed/tests/test_steal.py::test_steal_more_attractive_tasks PASSED [ 77%] distributed/tests/test_steal.py::test_balance[don't move unnecessarily] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[balance] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[balance even if results in even] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[don't over balance] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[move from larger] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[move to smaller] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[choose easier first] PASSED [ 77%] distributed/tests/test_steal.py::test_balance[spread evenly] PASSED [ 78%] distributed/tests/test_steal.py::test_balance[move easier] PASSED [ 78%] distributed/tests/test_steal.py::test_balance[be willing to move costly items] PASSED [ 78%] distributed/tests/test_steal.py::test_balance[but don't move too many] PASSED [ 78%] distributed/tests/test_steal.py::test_balance[no one clearly saturated] PASSED [ 78%] distributed/tests/test_steal.py::test_balance[balance multiple saturated workers] PASSED [ 78%] distributed/tests/test_steal.py::test_restart PASSED [ 78%] distributed/tests/test_steal.py::test_paused_workers_must_not_steal PASSED [ 78%] distributed/tests/test_steal.py::test_dont_steal_already_released PASSED [ 78%] distributed/tests/test_steal.py::test_dont_steal_long_running_tasks PASSED [ 78%] distributed/tests/test_steal.py::test_cleanup_repeated_tasks PASSED [ 78%] distributed/tests/test_steal.py::test_lose_task PASSED [ 78%] distributed/tests/test_steal.py::test_parse_stealing_interval[None-100] PASSED [ 78%] distributed/tests/test_steal.py::test_parse_stealing_interval[500ms-500] PASSED [ 78%] distributed/tests/test_steal.py::test_parse_stealing_interval[2-2] PASSED [ 78%] distributed/tests/test_steal.py::test_balance_with_longer_task PASSED [ 78%] distributed/tests/test_steal.py::test_blocklist_shuffle_split PASSED [ 78%] distributed/tests/test_steal.py::test_steal_concurrent_simple PASSED [ 78%] distributed/tests/test_steal.py::test_steal_reschedule_reset_in_flight_occupancy PASSED [ 78%] distributed/tests/test_steal.py::test_get_story PASSED [ 78%] distributed/tests/test_steal.py::test_steal_worker_dies_same_ip PASSED [ 78%] distributed/tests/test_steal.py::test_reschedule_concurrent_requests_deadlock PASSED [ 78%] distributed/tests/test_steal.py::test_correct_bad_time_estimate SKIPPED [ 78%] distributed/tests/test_steal.py::test_steal_stimulus_id_unique PASSED [ 78%] distributed/tests/test_steal.py::test_steal_worker_state[executing] PASSED [ 78%] distributed/tests/test_steal.py::test_steal_worker_state[long-running] PASSED [ 78%] distributed/tests/test_steal.py::test_steal_very_fast_tasks SKIPPED [ 78%] distributed/tests/test_steal.py::test_balance_expensive_tasks[not enough work to steal] PASSED [ 78%] distributed/tests/test_steal.py::test_balance_expensive_tasks[enough work to steal] PASSED [ 78%] distributed/tests/test_steal.py::test_balance_expensive_tasks[not enough work for increased cost] PASSED [ 78%] distributed/tests/test_steal.py::test_balance_uneven_without_replica PASSED [ 78%] distributed/tests/test_steal.py::test_balance_eventually_steals_large_dependency_without_replica PASSED [ 78%] distributed/tests/test_steal.py::test_balance_even_with_replica PASSED [ 79%] distributed/tests/test_steal.py::test_balance_to_replica PASSED [ 79%] distributed/tests/test_steal.py::test_balance_multiple_to_replica PASSED [ 79%] distributed/tests/test_steal.py::test_balance_to_larger_dependency PASSED [ 79%] distributed/tests/test_steal.py::test_balance_prefers_busier_with_dependency PASSED [ 79%] distributed/tests/test_steal.py::test_trivial_workload_should_not_cause_work_stealing PASSED [ 79%] distributed/tests/test_stories.py::test_scheduler_story_stimulus_success PASSED [ 79%] distributed/tests/test_stories.py::test_scheduler_story_stimulus_retry PASSED [ 79%] distributed/tests/test_stories.py::test_client_story PASSED [ 79%] distributed/tests/test_stories.py::test_client_story_failed_worker[ignore] PASSED [ 79%] distributed/tests/test_stories.py::test_client_story_failed_worker[raise] PASSED [ 79%] distributed/tests/test_stories.py::test_worker_story_with_deps PASSED [ 79%] distributed/tests/test_stress.py::test_stress_1 PASSED [ 79%] distributed/tests/test_stress.py::test_stress_gc[slowinc-100] SKIPPED [ 79%] distributed/tests/test_stress.py::test_stress_gc[inc-1000] SKIPPED (need --runslow option to run) [ 79%] distributed/tests/test_stress.py::test_cancel_stress PASSED [ 79%] distributed/tests/test_stress.py::test_cancel_stress_sync PASSED [ 79%] distributed/tests/test_stress.py::test_stress_creation_and_deletion SKIPPED [ 79%] distributed/tests/test_stress.py::test_stress_scatter_death PASSED [ 79%] distributed/tests/test_stress.py::test_stress_steal SKIPPED [ 79%] distributed/tests/test_stress.py::test_close_connections SKIPPED (need --runslow option to run) [ 79%] distributed/tests/test_stress.py::test_no_delay_during_large_transfer SKIPPED [ 79%] distributed/tests/test_stress.py::test_chaos_rechunk SKIPPED (need --runslow option to run) [ 79%] distributed/tests/test_system.py::test_memory_limit PASSED [ 79%] distributed/tests/test_system.py::test_hard_memory_limit_cgroups PASSED [ 79%] distributed/tests/test_system.py::test_soft_memory_limit_cgroups PASSED [ 79%] distributed/tests/test_system.py::test_hard_memory_limit_cgroups2 PASSED [ 79%] distributed/tests/test_system.py::test_soft_memory_limit_cgroups2 PASSED [ 79%] distributed/tests/test_system.py::test_softest_memory_limit_cgroups2 PASSED [ 79%] distributed/tests/test_system.py::test_rlimit PASSED [ 79%] distributed/tests/test_system_monitor.py::test_SystemMonitor PASSED [ 79%] distributed/tests/test_system_monitor.py::test_maxlen_zero PASSED [ 80%] distributed/tests/test_system_monitor.py::test_count PASSED [ 80%] distributed/tests/test_system_monitor.py::test_range_query PASSED [ 80%] distributed/tests/test_system_monitor.py::test_disk_config PASSED [ 80%] distributed/tests/test_system_monitor.py::test_host_cpu PASSED [ 80%] distributed/tests/test_system_monitor.py::test_gil_contention SKIPPED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_tpe PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_shutdown_timeout PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_shutdown_timeout_raises PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_shutdown_wait PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_secede_rejoin_busy PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_secede_rejoin_quiet PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_rejoin_idempotent PASSED [ 80%] distributed/tests/test_threadpoolexecutor.py::test_thread_name PASSED [ 80%] distributed/tests/test_tls_functional.py::test_basic PASSED [ 80%] distributed/tests/test_tls_functional.py::test_Queue PASSED [ 80%] distributed/tests/test_tls_functional.py::test_client_submit PASSED [ 80%] distributed/tests/test_tls_functional.py::test_gather PASSED [ 80%] distributed/tests/test_tls_functional.py::test_scatter PASSED [ 80%] distributed/tests/test_tls_functional.py::test_nanny PASSED [ 80%] distributed/tests/test_tls_functional.py::test_rebalance PASSED [ 80%] distributed/tests/test_tls_functional.py::test_work_stealing PASSED [ 80%] distributed/tests/test_tls_functional.py::test_worker_client PASSED [ 80%] distributed/tests/test_tls_functional.py::test_worker_client_gather PASSED [ 80%] distributed/tests/test_tls_functional.py::test_worker_client_executor PASSED [ 80%] distributed/tests/test_tls_functional.py::test_retire_workers PASSED [ 80%] distributed/tests/test_tls_functional.py::test_security_dict_input PASSED [ 80%] distributed/tests/test_utils.py::test_All PASSED [ 80%] distributed/tests/test_utils.py::test_sync PASSED [ 80%] distributed/tests/test_utils.py::test_sync_error PASSED [ 80%] distributed/tests/test_utils.py::test_sync_timeout PASSED [ 80%] distributed/tests/test_utils.py::test_sync_closed_loop PASSED [ 81%] distributed/tests/test_utils.py::test_sync_contextvars PASSED [ 81%] distributed/tests/test_utils.py::test_is_kernel SKIPPED (could not import 'IPython': No module named 'IPython') [ 81%] distributed/tests/test_utils.py::test_ensure_ip PASSED [ 81%] distributed/tests/test_utils.py::test_get_ip_interface PASSED [ 81%] distributed/tests/test_utils.py::test_get_mp_context PASSED [ 81%] distributed/tests/test_utils.py::test_truncate_exception PASSED [ 81%] distributed/tests/test_utils.py::test_get_traceback PASSED [ 81%] distributed/tests/test_utils.py::test_maybe_complex PASSED [ 81%] distributed/tests/test_utils.py::test_read_block PASSED [ 81%] distributed/tests/test_utils.py::test_seek_delimiter_endline PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data1] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[1] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data3] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data4] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data5] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data6] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data7] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data8] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data9] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data10] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data11] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data12] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data13] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview[data14] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_ndarray[i8-12-shape0-strides0] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_ndarray[i8-12-shape1-strides1] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_ndarray[i8-12-shape2-strides2] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_ndarray[i8-12-shape3-strides3] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_ndarray[i8-12-shape4-strides4] PASSED [ 81%] distributed/tests/test_utils.py::test_ensure_memoryview_pyarrow_buffer SKIPPED [ 82%] distributed/tests/test_utils.py::test_nbytes PASSED [ 82%] distributed/tests/test_utils.py::test_open_port PASSED [ 82%] distributed/tests/test_utils.py::test_set_thread_state PASSED [ 82%] distributed/tests/test_utils.py::test_loop_runner PASSED [ 82%] distributed/tests/test_utils.py::test_two_loop_runners PASSED [ 82%] distributed/tests/test_utils.py::test_loop_runner_gen PASSED [ 82%] distributed/tests/test_utils.py::test_loop_runner_exception_in_start PASSED [ 82%] distributed/tests/test_utils.py::test_loop_runner_exception_in_teardown PASSED [ 82%] distributed/tests/test_utils.py::test_all_quiet_exceptions PASSED [ 82%] distributed/tests/test_utils.py::test_warn_on_duration PASSED [ 82%] distributed/tests/test_utils.py::test_logs PASSED [ 82%] distributed/tests/test_utils.py::test_is_valid_xml PASSED [ 82%] distributed/tests/test_utils.py::test_format_dashboard_link PASSED [ 82%] distributed/tests/test_utils.py::test_parse_ports PASSED [ 82%] distributed/tests/test_utils.py::test_run_in_executor_with_context PASSED [ 82%] distributed/tests/test_utils.py::test_run_in_executor_with_context_preserves_contextvars PASSED [ 82%] distributed/tests/test_utils.py::test_offload PASSED [ 82%] distributed/tests/test_utils.py::test_offload_preserves_contextvars PASSED [ 82%] distributed/tests/test_utils.py::test_no_default_deprecated PASSED [ 82%] distributed/tests/test_utils.py::test_iscoroutinefunction_unhashable_input PASSED [ 82%] distributed/tests/test_utils.py::test_iscoroutinefunction_nested_partial PASSED [ 82%] distributed/tests/test_utils.py::test_recursive_to_dict PASSED [ 82%] distributed/tests/test_utils.py::test_recursive_to_dict_no_nest PASSED [ 82%] distributed/tests/test_utils.py::test_log_errors PASSED [ 82%] distributed/tests/test_utils.py::test_log_errors_unroll_stack[0-test_utils] PASSED [ 82%] distributed/tests/test_utils.py::test_log_errors_unroll_stack[1-a.b.c] PASSED [ 82%] distributed/tests/test_utils.py::test_load_json_robust_timeout PASSED [ 82%] distributed/tests/test_utils.py::test_rate_limiter_filter PASSED [ 82%] distributed/tests/test_utils.py::test_tuple_comparable_lt[obj10-obj20-False] PASSED [ 82%] distributed/tests/test_utils.py::test_tuple_comparable_lt[obj11-obj21-True] PASSED [ 82%] distributed/tests/test_utils.py::test_tuple_comparable_lt[1-1-False] PASSED [ 82%] distributed/tests/test_utils.py::test_tuple_comparable_lt[1-2-True] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_lt[None-0-False] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_lt[None-obj25-True] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[obj10-obj20-True] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[obj11-obj21-False] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[1-1-True] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[1-2-False] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[None-0-True] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_eq[None-obj25-False] PASSED [ 83%] distributed/tests/test_utils.py::test_tuple_comparable_error PASSED [ 83%] distributed/tests/test_utils_comm.py::test_pack_data PASSED [ 83%] distributed/tests/test_utils_comm.py::test_subs_multiple PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_missing_replicas PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_permissive PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_permissive_flaky PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_busy PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_serialization_error[pickle] PASSED [ 83%] distributed/tests/test_utils_comm.py::test_gather_from_workers_serialization_error[unpickle] PASSED [ 83%] distributed/tests/test_utils_comm.py::test_retry_no_exception PASSED [ 83%] distributed/tests/test_utils_comm.py::test_retry0_raises_immediately PASSED [ 83%] distributed/tests/test_utils_comm.py::test_retry_does_retry_and_sleep PASSED [ 83%] distributed/tests/test_utils_comm.py::test_unpack_remotedata PASSED [ 83%] distributed/tests/test_utils_perf.py::test_fractional_timer PASSED [ 83%] distributed/tests/test_utils_perf.py::test_gc_diagnosis_cpu_time SKIPPED [ 83%] distributed/tests/test_utils_perf.py::test_gc_diagnosis_rss_win XFAIL [ 83%] distributed/tests/test_utils_test.py::test_bare_cluster PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster_pytest_fixture PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster_parametrized[True] PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster_multi_parametrized[a-True] PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster_multi_parametrized[b-True] PASSED [ 83%] distributed/tests/test_utils_test.py::test_gen_cluster_parametrized_variadic_workers[True] PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_set_config_nanny PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_cleans_up_client SKIPPED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_without_client PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_tls PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_validate PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_cluster_validate_override PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_test XFAIL (Test should always fail to ensure the body of the test function was run) [ 84%] distributed/tests/test_utils_test.py::test_gen_test_legacy_implicit XFAILn) [ 84%] distributed/tests/test_utils_test.py::test_gen_test_legacy_explicit XFAILn) [ 84%] distributed/tests/test_utils_test.py::test_gen_test_parametrized[True] PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_test_double_parametrized[False-True] PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_test_pytest_fixture PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_test_config PASSED [ 84%] distributed/tests/test_utils_test.py::test_gen_test_config_default PASSED [ 84%] distributed/tests/test_utils_test.py::test_new_config PASSED [ 84%] distributed/tests/test_utils_test.py::test_lingering_client PASSED [ 84%] distributed/tests/test_utils_test.py::test_lingering_client_2 PASSED [ 84%] distributed/tests/test_utils_test.py::test_tls_cluster PASSED [ 84%] distributed/tests/test_utils_test.py::test_tls_scheduler PASSED [ 84%] distributed/tests/test_utils_test.py::test__UnhashableCallable PASSED [ 84%] distributed/tests/test_utils_test.py::test_dump_cluster_state_timeout SKIPPED [ 84%] distributed/tests/test_utils_test.py::test_assert_story PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[Missing payload, stimulus_id, ts] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[Missing (stimulus_id, ts)] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[Missing ts] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[ts is not a float] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[ts is in the future] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[ts is too old] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[stimulus_id is not a str] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[stimulus_id is an empty str] PASSED [ 84%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[no payload] PASSED [ 85%] distributed/tests/test_utils_test.py::test_assert_story_malformed_story[timestamps out of order] PASSED [ 85%] distributed/tests/test_utils_test.py::test_assert_story_identity[True] PASSED [ 85%] distributed/tests/test_utils_test.py::test_assert_story_identity[False] PASSED [ 85%] distributed/tests/test_utils_test.py::test_dump_cluster_state PASSED [ 85%] distributed/tests/test_utils_test.py::test_dump_cluster_state_no_workers PASSED [ 85%] distributed/tests/test_utils_test.py::test_dump_cluster_state_nannies PASSED [ 85%] distributed/tests/test_utils_test.py::test_dump_cluster_state_unresponsive_local_worker PASSED [ 85%] distributed/tests/test_utils_test.py::test_dump_cluster_unresponsive_remote_worker SKIPPED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak_slow_cleanup PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak_pre_cleanup[False] PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak_pre_cleanup[True] PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak_post_cleanup[False] PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_process_leak_post_cleanup[True] PASSED [ 85%] distributed/tests/test_utils_test.py::test_start_failure_worker[True] PASSED [ 85%] distributed/tests/test_utils_test.py::test_start_failure_worker[False] PASSED [ 85%] distributed/tests/test_utils_test.py::test_start_failure_scheduler PASSED [ 85%] distributed/tests/test_utils_test.py::test_invalid_transitions PASSED [ 85%] distributed/tests/test_utils_test.py::test_invalid_worker_state PASSED [ 85%] distributed/tests/test_utils_test.py::test_raises_with_cause PASSED [ 85%] distributed/tests/test_utils_test.py::test_check_thread_leak SKIPPED [ 85%] distributed/tests/test_utils_test.py::test_fail_hard[True] PASSED [ 85%] distributed/tests/test_utils_test.py::test_fail_hard[False] PASSED [ 85%] distributed/tests/test_utils_test.py::test_popen_write_during_terminate_deadlock PASSED [ 85%] distributed/tests/test_utils_test.py::test_popen_timeout PASSED [ 85%] distributed/tests/test_utils_test.py::test_popen_always_prints_output PASSED [ 85%] distributed/tests/test_utils_test.py::test_wait_for_state PASSED [ 85%] distributed/tests/test_utils_test.py::test_wait_for_stimulus PASSED [ 85%] distributed/tests/test_utils_test.py::test_ws_with_running_task[executing] PASSED [ 85%] distributed/tests/test_utils_test.py::test_ws_with_running_task[long-running] PASSED [ 85%] distributed/tests/test_utils_test.py::test_sizeof PASSED [ 85%] distributed/tests/test_utils_test.py::test_sizeof_error[12345.0-TypeError-Expected integer] PASSED [ 86%] distributed/tests/test_utils_test.py::test_sizeof_error[-1-ValueError-larger than] PASSED [ 86%] distributed/tests/test_utils_test.py::test_sizeof_error[0-ValueError-larger than] PASSED [ 86%] distributed/tests/test_utils_test.py::test_sizeof_error[10-ValueError-larger than] PASSED [ 86%] distributed/tests/test_utils_test.py::test_captured_logger PASSED [ 86%] distributed/tests/test_utils_test.py::test_captured_context_meter PASSED [ 86%] distributed/tests/test_variable.py::test_variable PASSED [ 86%] distributed/tests/test_variable.py::test_delete_unset_variable PASSED [ 86%] distributed/tests/test_variable.py::test_queue_with_data PASSED [ 86%] distributed/tests/test_variable.py::test_sync PASSED [ 86%] distributed/tests/test_variable.py::test_hold_futures PASSED [ 86%] distributed/tests/test_variable.py::test_timeout PASSED [ 86%] distributed/tests/test_variable.py::test_timeout_sync PASSED [ 86%] distributed/tests/test_variable.py::test_cleanup PASSED [ 86%] distributed/tests/test_variable.py::test_pickleable PASSED [ 86%] distributed/tests/test_variable.py::test_timeout_get PASSED [ 86%] distributed/tests/test_variable.py::test_race SKIPPED (need --runslow option to run) [ 86%] distributed/tests/test_variable.py::test_Future_knows_status_immediately PASSED [ 86%] distributed/tests/test_variable.py::test_erred_future PASSED [ 86%] distributed/tests/test_variable.py::test_future_erred_sync PASSED [ 86%] distributed/tests/test_variable.py::test_variables_do_not_leak_client PASSED [ 86%] distributed/tests/test_variable.py::test_unpickle_without_client PASSED [ 86%] distributed/tests/test_versions.py::test_versions_match PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[source-MISMATCHED] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[source-MISSING] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[source-KEY_ERROR] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[source-NONE] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[scheduler-MISMATCHED] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[scheduler-MISSING] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[scheduler-KEY_ERROR] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[scheduler-NONE] PASSED [ 86%] distributed/tests/test_versions.py::test_version_mismatch[worker-1-MISMATCHED] PASSED [ 87%] distributed/tests/test_versions.py::test_version_mismatch[worker-1-MISSING] PASSED [ 87%] distributed/tests/test_versions.py::test_version_mismatch[worker-1-KEY_ERROR] PASSED [ 87%] distributed/tests/test_versions.py::test_version_mismatch[worker-1-NONE] PASSED [ 87%] distributed/tests/test_versions.py::test_scheduler_mismatched_irrelevant_package PASSED [ 87%] distributed/tests/test_versions.py::test_scheduler_additional_irrelevant_package PASSED [ 87%] distributed/tests/test_versions.py::test_python_mismatch[source] PASSED [ 87%] distributed/tests/test_versions.py::test_python_mismatch[scheduler] PASSED [ 87%] distributed/tests/test_versions.py::test_python_mismatch[worker-1] PASSED [ 87%] distributed/tests/test_versions.py::test_version_warning_in_cluster PASSED [ 87%] distributed/tests/test_versions.py::test_python_version PASSED [ 87%] distributed/tests/test_versions.py::test_version_custom_pkgs PASSED [ 87%] distributed/tests/test_worker.py::test_worker_nthreads PASSED [ 87%] distributed/tests/test_worker.py::test_str PASSED [ 87%] distributed/tests/test_worker.py::test_identity PASSED [ 87%] distributed/tests/test_worker.py::test_worker_bad_args PASSED [ 87%] distributed/tests/test_worker.py::test_upload_file PASSED [ 87%] distributed/tests/test_worker.py::test_upload_file_pyc SKIPPED (don't yet support uploading pyc files) [ 87%] distributed/tests/test_worker.py::test_upload_egg PASSED [ 87%] distributed/tests/test_worker.py::test_upload_pyz PASSED [ 87%] distributed/tests/test_worker.py::test_upload_large_file SKIPPED (could not import 'crick': No module named 'crick') [ 87%] distributed/tests/test_worker.py::test_broadcast PASSED [ 87%] distributed/tests/test_worker.py::test_worker_with_port_zero PASSED [ 87%] distributed/tests/test_worker.py::test_worker_port_range PASSED [ 87%] distributed/tests/test_worker.py::test_worker_waits_for_scheduler SKIPPED [ 87%] distributed/tests/test_worker.py::test_worker_task_data PASSED [ 87%] distributed/tests/test_worker.py::test_error_message PASSED [ 87%] distributed/tests/test_worker.py::test_chained_error_message PASSED [ 87%] distributed/tests/test_worker.py::test_base_exception_in_task[BaseException-True] SKIPPED [ 87%] distributed/tests/test_worker.py::test_base_exception_in_task[BaseException-False] SKIPPED [ 87%] distributed/tests/test_worker.py::test_base_exception_in_task[SystemExit-True] SKIPPED [ 87%] distributed/tests/test_worker.py::test_base_exception_in_task[SystemExit-False] SKIPPED [ 88%] distributed/tests/test_worker.py::test_base_exception_in_task[KeyboardInterrupt-True] SKIPPED [ 88%] distributed/tests/test_worker.py::test_base_exception_in_task[KeyboardInterrupt-False] SKIPPED [ 88%] distributed/tests/test_worker.py::test_base_exception_in_task[CancelledError-True] SKIPPED [ 88%] distributed/tests/test_worker.py::test_base_exception_in_task[CancelledError-False] SKIPPED [ 88%] distributed/tests/test_worker.py::test_gather PASSED [ 88%] distributed/tests/test_worker.py::test_gather_missing_keys PASSED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers PASSED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers_replicated[False] SKIPPED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers_replicated[True0] SKIPPED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers_replicated[True1] SKIPPED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers_replicated[True2] SKIPPED [ 88%] distributed/tests/test_worker.py::test_gather_missing_workers_replicated[True3] SKIPPED [ 88%] distributed/tests/test_worker.py::test_io_loop PASSED [ 88%] distributed/tests/test_worker.py::test_io_loop_alternate_loop PASSED [ 88%] distributed/tests/test_worker.py::test_access_key PASSED [ 88%] distributed/tests/test_worker.py::test_run_dask_worker PASSED [ 88%] distributed/tests/test_worker.py::test_run_dask_worker_kwonlyarg PASSED [ 88%] distributed/tests/test_worker.py::test_run_coroutine_dask_worker PASSED [ 88%] distributed/tests/test_worker.py::test_Executor PASSED [ 88%] distributed/tests/test_worker.py::test_close_on_disconnect PASSED [ 88%] distributed/tests/test_worker.py::test_memory_limit_auto PASSED [ 88%] distributed/tests/test_worker.py::test_inter_worker_communication PASSED [ 88%] distributed/tests/test_worker.py::test_clean PASSED [ 88%] distributed/tests/test_worker.py::test_message_breakup PASSED [ 88%] distributed/tests/test_worker.py::test_types PASSED [ 88%] distributed/tests/test_worker.py::test_system_monitor PASSED [ 88%] distributed/tests/test_worker.py::test_restrictions PASSED [ 88%] distributed/tests/test_worker.py::test_clean_nbytes PASSED [ 88%] distributed/tests/test_worker.py::test_gather_many_small[True] PASSED [ 88%] distributed/tests/test_worker.py::test_gather_many_small[False] PASSED [ 88%] distributed/tests/test_worker.py::test_multiple_transfers PASSED [ 89%] distributed/tests/test_worker.py::test_share_communication XFAIL (very high flakiness) [ 89%] distributed/tests/test_worker.py::test_dont_overlap_communications_to_same_worker XFAIL [ 89%] distributed/tests/test_worker.py::test_log_event PASSED [ 89%] distributed/tests/test_worker.py::test_log_exception_on_failed_task PASSED [ 89%] distributed/tests/test_worker.py::test_clean_up_dependencies PASSED [ 89%] distributed/tests/test_worker.py::test_hold_onto_dependents PASSED [ 89%] distributed/tests/test_worker.py::test_worker_death_timeout XPASS [ 89%] distributed/tests/test_worker.py::test_stop_doing_unnecessary_work PASSED [ 89%] distributed/tests/test_worker.py::test_priorities PASSED [ 89%] distributed/tests/test_worker.py::test_heartbeats PASSED [ 89%] distributed/tests/test_worker.py::test_worker_dir[Worker] PASSED [ 89%] distributed/tests/test_worker.py::test_worker_dir[Nanny] PASSED [ 89%] distributed/tests/test_worker.py::test_default_worker_dir PASSED [ 89%] distributed/tests/test_worker.py::test_dataframe_attribute_error PASSED [ 89%] distributed/tests/test_worker.py::test_pid PASSED [ 89%] distributed/tests/test_worker.py::test_get_client PASSED [ 89%] distributed/tests/test_worker.py::test_get_client_sync PASSED [ 89%] distributed/tests/test_worker.py::test_get_client_coroutine PASSED [ 89%] distributed/tests/test_worker.py::test_get_client_coroutine_sync XFAILk/distributed/pull/7339) [ 89%] distributed/tests/test_worker.py::test_global_workers PASSED [ 89%] distributed/tests/test_worker.py::test_worker_fds PASSED [ 89%] distributed/tests/test_worker.py::test_service_hosts_match_worker PASSED [ 89%] distributed/tests/test_worker.py::test_start_services PASSED [ 89%] distributed/tests/test_worker.py::test_scheduler_delay PASSED [ 89%] distributed/tests/test_worker.py::test_statistical_profiling PASSED [ 89%] distributed/tests/test_worker.py::test_statistical_profiling_2 SKIPPED [ 89%] distributed/tests/test_worker.py::test_statistical_profiling_cycle PASSED [ 89%] distributed/tests/test_worker.py::test_get_current_task PASSED [ 89%] distributed/tests/test_worker.py::test_deque_handler PASSED [ 89%] distributed/tests/test_worker.py::test_scheduler_address_config PASSED [ 89%] distributed/tests/test_worker.py::test_wait_for_outgoing SKIPPED (need --runslow option to run) [ 89%] distributed/tests/test_worker.py::test_prefer_gather_from_local_address PASSED [ 90%] distributed/tests/test_worker.py::test_avoid_oversubscription PASSED [ 90%] distributed/tests/test_worker.py::test_custom_metrics PASSED [ 90%] distributed/tests/test_worker.py::test_register_worker_callbacks PASSED [ 90%] distributed/tests/test_worker.py::test_register_worker_callbacks_err PASSED [ 90%] distributed/tests/test_worker.py::test_local_directory PASSED [ 90%] distributed/tests/test_worker.py::test_local_directory_make_new_directory PASSED [ 90%] distributed/tests/test_worker.py::test_host_address PASSED [ 90%] distributed/tests/test_worker.py::test_interface_async[Worker] PASSED [ 90%] distributed/tests/test_worker.py::test_interface_async[Nanny] PASSED [ 90%] distributed/tests/test_worker.py::test_protocol_from_scheduler_address[Worker] SKIPPED [ 90%] distributed/tests/test_worker.py::test_protocol_from_scheduler_address[Nanny] SKIPPED [ 90%] distributed/tests/test_worker.py::test_worker_listens_on_same_interface_by_default[Worker] PASSED [ 90%] distributed/tests/test_worker.py::test_worker_listens_on_same_interface_by_default[Nanny] PASSED [ 90%] distributed/tests/test_worker.py::test_close_gracefully SKIPPED (need --runslow option to run) [ 90%] distributed/tests/test_worker.py::test_close_gracefully_no_suspicious_tasks SKIPPED [ 90%] distributed/tests/test_worker.py::test_close_while_executing[False] PASSED [ 90%] distributed/tests/test_worker.py::test_close_while_executing[True] SKIPPED [ 90%] distributed/tests/test_worker.py::test_close_async_task_handles_cancellation SKIPPED [ 90%] distributed/tests/test_worker.py::test_lifetime SKIPPED (need --runslow option to run) [ 90%] distributed/tests/test_worker.py::test_lifetime_stagger PASSED [ 90%] distributed/tests/test_worker.py::test_bad_metrics PASSED [ 90%] distributed/tests/test_worker.py::test_bad_startup PASSED [ 90%] distributed/tests/test_worker.py::test_update_latency PASSED [ 90%] distributed/tests/test_worker.py::test_workerstate_executing SKIPPED [ 90%] distributed/tests/test_worker.py::test_shutdown_on_scheduler_comm_closed PASSED [ 90%] distributed/tests/test_worker.py::test_heartbeat_comm_closed PASSED [ 90%] distributed/tests/test_worker.py::test_heartbeat_missing PASSED [ 90%] distributed/tests/test_worker.py::test_heartbeat_missing_real_cluster PASSED [ 90%] distributed/tests/test_worker.py::test_heartbeat_missing_doesnt_restart SKIPPED [ 90%] distributed/tests/test_worker.py::test_bad_local_directory PASSED [ 90%] distributed/tests/test_worker.py::test_taskstate_metadata PASSED [ 91%] distributed/tests/test_worker.py::test_executor_offload PASSED [ 91%] distributed/tests/test_worker.py::test_story PASSED [ 91%] distributed/tests/test_worker.py::test_stimulus_story PASSED [ 91%] distributed/tests/test_worker.py::test_worker_descopes_data PASSED [ 91%] distributed/tests/test_worker.py::test_gather_dep_one_worker_always_busy PASSED [ 91%] distributed/tests/test_worker.py::test_gather_dep_local_workers_first PASSED [ 91%] distributed/tests/test_worker.py::test_gather_dep_from_remote_workers_if_all_local_workers_are_busy PASSED [ 91%] distributed/tests/test_worker.py::test_worker_client_uses_default_no_close PASSED [ 91%] distributed/tests/test_worker.py::test_worker_client_closes_if_created_on_worker_one_worker PASSED [ 91%] distributed/tests/test_worker.py::test_worker_client_closes_if_created_on_worker_last_worker_alive PASSED [ 91%] distributed/tests/test_worker.py::test_multiple_executors PASSED [ 91%] distributed/tests/test_worker.py::test_bad_executor_annotation PASSED [ 91%] distributed/tests/test_worker.py::test_process_executor PASSED [ 91%] distributed/tests/test_worker.py::test_process_executor_raise_exception PASSED [ 91%] distributed/tests/test_worker.py::test_worker_state_error_release_error_last PASSED [ 91%] distributed/tests/test_worker.py::test_worker_state_error_release_error_first PASSED [ 91%] distributed/tests/test_worker.py::test_worker_state_error_release_error_int PASSED [ 91%] distributed/tests/test_worker.py::test_worker_state_error_long_chain PASSED [ 91%] distributed/tests/test_worker.py::test_hold_on_to_replicas PASSED [ 91%] distributed/tests/test_worker.py::test_forget_dependents_after_release PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas_same_channel PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas_many PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas_already_in_flight PASSED [ 91%] distributed/tests/test_worker.py::test_forget_acquire_replicas SKIPPED [ 91%] distributed/tests/test_worker.py::test_remove_replicas_simple PASSED [ 91%] distributed/tests/test_worker.py::test_who_has_consistent_remove_replicas PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas_with_no_priority PASSED [ 91%] distributed/tests/test_worker.py::test_acquire_replicas_large_data PASSED [ 91%] distributed/tests/test_worker.py::test_missing_released_zombie_tasks PASSED [ 91%] distributed/tests/test_worker.py::test_missing_released_zombie_tasks_2 PASSED [ 92%] distributed/tests/test_worker.py::test_worker_status_sync PASSED [ 92%] distributed/tests/test_worker.py::test_log_remove_worker PASSED [ 92%] distributed/tests/test_worker.py::test_task_flight_compute_oserror PASSED [ 92%] distributed/tests/test_worker.py::test_gather_dep_cancelled_rescheduled PASSED [ 92%] distributed/tests/test_worker.py::test_gather_dep_do_not_handle_response_of_not_requested_tasks PASSED [ 92%] distributed/tests/test_worker.py::test_gather_dep_no_longer_in_flight_tasks PASSED [ 92%] distributed/tests/test_worker.py::test_get_data_cancelled_error PASSED [ 92%] distributed/tests/test_worker.py::test_Worker__to_dict PASSED [ 92%] distributed/tests/test_worker.py::test_extension_methods PASSED [ 92%] distributed/tests/test_worker.py::test_benchmark_hardware PASSED [ 92%] distributed/tests/test_worker.py::test_tick_interval SKIPPED (need --runslow option to run) [ 92%] distributed/tests/test_worker.py::test_broken_comm SKIPPED (need --runslow option to run) [ 92%] distributed/tests/test_worker.py::test_do_not_block_event_loop_during_shutdown PASSED [ 92%] distributed/tests/test_worker.py::test_reconnect_argument_deprecated PASSED [ 92%] distributed/tests/test_worker.py::test_worker_running_before_running_plugins PASSED [ 92%] distributed/tests/test_worker.py::test_execute_preamble_abort_retirement PASSED [ 92%] distributed/tests/test_worker.py::test_deprecation_of_renamed_worker_attributes PASSED [ 92%] distributed/tests/test_worker.py::test_forward_output PASSED [ 92%] distributed/tests/test_worker.py::test_offload_getdata PASSED [ 92%] distributed/tests/test_worker.py::test_startstops PASSED [ 92%] distributed/tests/test_worker.py::test_suppress_keyerror_for_cancelled_tasks[cancelled] PASSED [ 92%] distributed/tests/test_worker.py::test_suppress_keyerror_for_cancelled_tasks[resumed] PASSED [ 92%] distributed/tests/test_worker_client.py::test_submit_from_worker PASSED [ 92%] distributed/tests/test_worker_client.py::test_submit_from_worker_async PASSED [ 92%] distributed/tests/test_worker_client.py::test_scatter_from_worker PASSED [ 92%] distributed/tests/test_worker_client.py::test_scatter_singleton PASSED [ 92%] distributed/tests/test_worker_client.py::test_gather_multi_machine PASSED [ 92%] distributed/tests/test_worker_client.py::test_same_loop PASSED [ 92%] distributed/tests/test_worker_client.py::test_sync PASSED [ 92%] distributed/tests/test_worker_client.py::test_async PASSED [ 92%] distributed/tests/test_worker_client.py::test_separate_thread_false PASSED [ 92%] distributed/tests/test_worker_client.py::test_client_executor PASSED [ 93%] distributed/tests/test_worker_client.py::test_local_client_warning PASSED [ 93%] distributed/tests/test_worker_client.py::test_closing_worker_doesnt_close_client PASSED [ 93%] distributed/tests/test_worker_client.py::test_timeout PASSED [ 93%] distributed/tests/test_worker_client.py::test_secede_without_stealing_issue_1262 PASSED [ 93%] distributed/tests/test_worker_client.py::test_compute_within_worker_client PASSED [ 93%] distributed/tests/test_worker_client.py::test_worker_client_rejoins PASSED [ 93%] distributed/tests/test_worker_client.py::test_submit_different_names XPASS [ 93%] distributed/tests/test_worker_client.py::test_secede_does_not_claim_worker PASSED [ 93%] distributed/tests/test_worker_memory.py::test_parse_memory_limit_zero PASSED [ 93%] distributed/tests/test_worker_memory.py::test_resource_limit PASSED [ 93%] distributed/tests/test_worker_memory.py::test_parse_memory_limit_worker PASSED [ 93%] distributed/tests/test_worker_memory.py::test_parse_memory_limit_worker_relative PASSED [ 93%] distributed/tests/test_worker_memory.py::test_parse_memory_limit_nanny PASSED [ 93%] distributed/tests/test_worker_memory.py::test_dict_data_if_no_spill_to_disk PASSED [ 93%] distributed/tests/test_worker_memory.py::test_worker_data_callable_local_directory PASSED [ 93%] distributed/tests/test_worker_memory.py::test_worker_data_callable_local_directory_kwargs PASSED [ 93%] distributed/tests/test_worker_memory.py::test_worker_data_callable_kwargs PASSED [ 93%] distributed/tests/test_worker_memory.py::test_workerstate_fail_to_pickle_execute_1[executing] PASSED [ 93%] distributed/tests/test_worker_memory.py::test_workerstate_fail_to_pickle_execute_1[long-running] PASSED [ 93%] distributed/tests/test_worker_memory.py::test_workerstate_fail_to_pickle_flight PASSED [ 93%] distributed/tests/test_worker_memory.py::test_fail_to_pickle_execute_2 PASSED [ 93%] distributed/tests/test_worker_memory.py::test_fail_to_pickle_spill PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_target_threshold PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_constrained PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_spill_threshold PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_hysteresis[False-10000000000-1] PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_hysteresis[0.7-0-1] PASSED [ 93%] distributed/tests/test_worker_memory.py::test_spill_hysteresis[0.4-0-7] PASSED [ 93%] distributed/tests/test_worker_memory.py::test_pause_executor_manual PASSED [ 93%] distributed/tests/test_worker_memory.py::test_pause_executor_with_memory_monitor PASSED [ 93%] distributed/tests/test_worker_memory.py::test_pause_prevents_deps_fetch PASSED [ 94%] distributed/tests/test_worker_memory.py::test_avoid_memory_monitor_if_zero_limit_worker PASSED [ 94%] distributed/tests/test_worker_memory.py::test_avoid_memory_monitor_if_zero_limit_nanny PASSED [ 94%] distributed/tests/test_worker_memory.py::test_override_data_worker PASSED [ 94%] distributed/tests/test_worker_memory.py::test_override_data_nanny PASSED [ 94%] distributed/tests/test_worker_memory.py::test_override_data_vs_memory_monitor PASSED [ 94%] distributed/tests/test_worker_memory.py::test_manual_evict_proto PASSED [ 94%] distributed/tests/test_worker_memory.py::test_nanny_terminate SKIPPED [ 94%] distributed/tests/test_worker_memory.py::test_disk_cleanup_on_terminate[False] SKIPPED [ 94%] distributed/tests/test_worker_memory.py::test_disk_cleanup_on_terminate[True] SKIPPED [ 94%] distributed/tests/test_worker_memory.py::test_pause_while_spilling PASSED [ 94%] distributed/tests/test_worker_memory.py::test_release_evloop_while_spilling SKIPPED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Worker-memory_limit-123000000000.0] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Worker-memory_target_fraction-0.789] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Worker-memory_spill_fraction-0.789] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Worker-memory_pause_fraction-0.789] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Nanny-memory_limit-123000000000.0] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_attributes[Nanny-memory_terminate_fraction-0.789] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_memory_monitor_method_worker PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_memory_monitor_method_nanny PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_params[memory_target_fraction] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_params[memory_spill_fraction] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_deprecated_params[memory_pause_fraction] PASSED [ 94%] distributed/tests/test_worker_memory.py::test_pause_while_idle PASSED [ 94%] distributed/tests/test_worker_memory.py::test_pause_while_saturated PASSED [ 94%] distributed/tests/test_worker_memory.py::test_worker_log_memory_limit_too_high PASSED [ 94%] distributed/tests/test_worker_memory.py::test_high_unmanaged_memory_warning PASSED [ 94%] distributed/tests/test_worker_memory.py::test_delete_spilled_keys PASSED [ 94%] distributed/tests/test_worker_metrics.py::test_task_lifecycle PASSED [ 94%] distributed/tests/test_worker_metrics.py::test_async_task PASSED [ 94%] distributed/tests/test_worker_metrics.py::test_custom_executor PASSED [ 94%] distributed/tests/test_worker_metrics.py::test_offload PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_execute_failed PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_cancelled_execute PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_cancelled_flight PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_gather_dep_busy PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_gather_dep_no_task PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_gather_dep_failed PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_gather_dep_network_error PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_memory_monitor PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_user_metrics_sync PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_user_metrics_async PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_user_metrics_fail PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_user_metrics_weird PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_do_not_leak_metrics PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_reschedule PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_send_metrics_to_scheduler PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_no_spans_extension PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_new_metrics_during_heartbeat PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_delayed_ledger_is_not_reentrant PASSED [ 95%] distributed/tests/test_worker_metrics.py::test_int_metrics PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_instruction_match PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_TaskState_tracking PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_TaskState_get_nbytes PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_TaskState_eq PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_TaskState__to_dict PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_TaskState_repr PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_WorkerState__to_dict PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_WorkerState_pickle PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[False-InvalidTransition-kwargs0] PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[False-TransitionCounterMaxExceeded-kwargs1] PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[False-InvalidTaskState-kwargs2] PASSED [ 95%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[True-InvalidTransition-kwargs0] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[True-TransitionCounterMaxExceeded-kwargs1] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_pickle_exceptions[True-InvalidTaskState-kwargs2] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[TaskState] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[Instruction] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDep] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[Execute] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RetryBusyWorkerLater] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[SendMessageToScheduler] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[TaskFinishedMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[TaskErredMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[ReleaseWorkerDataMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RescheduleMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[LongRunningMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[AddKeysMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RequestRefreshWhoHasMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[StealResponseMsg] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[StateMachineEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[PauseEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[UnpauseEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RetryBusyWorkerEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDepDoneEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDepSuccessEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDepBusyEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDepNetworkFailureEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[GatherDepFailureEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RemoveWorkerEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[ComputeTaskEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[ExecuteDoneEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[ExecuteSuccessEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[ExecuteFailureEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[RescheduleEvent] PASSED [ 96%] distributed/tests/test_worker_state_machine.py::test_slots[CancelComputeEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[FindMissingEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[RefreshWhoHasEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[AcquireReplicasEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[RemoveReplicasEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[FreeKeysEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[StealRequestEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[UpdateDataEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_slots[SecedeEvent] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_sendmsg_to_dict PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_merge_recs_instructions PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_event_to_dict_with_annotations PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_event_to_dict_without_annotations PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_computetask_to_dict PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_computetask_dummy PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_updatedata_to_dict PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_executesuccess_to_dict PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_executesuccess_dummy PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_executefailure_to_dict PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_executefailure_dummy PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_fetch_to_compute PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_fetch_via_amm_to_compute PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_lose_replica_during_fetch[False] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_lose_replica_during_fetch[True] PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_fetch_to_missing_on_busy PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_new_replica_while_all_workers_in_flight PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_cancelled_while_in_flight PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_in_memory_while_in_flight PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_forget_data_needed PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_missing_handle_compute_dependency PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_missing_to_waiting PASSED [ 97%] distributed/tests/test_worker_state_machine.py::test_fetch_to_missing_on_refresh_who_has PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_fetch_to_missing_on_network_failure PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_deprecated_worker_attributes PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[10000000-3-1] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[10000000-3-2] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[20000000-2-1] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[20000000-2-2] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[30000000-1-1] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[30000000-1-2] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[60000000-1-1] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_aggregate_gather_deps[60000000-1-2] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_gather_priority PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_acquires_resources[executing] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_acquires_resources[long-running] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[executing-ExecuteSuccessEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[executing-ExecuteFailureEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[executing-RescheduleEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[long-running-ExecuteSuccessEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[long-running-ExecuteFailureEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_releases_resources[long-running-RescheduleEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_task_with_dependencies_acquires_resources PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[executing-ExecuteSuccessEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[executing-ExecuteFailureEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[executing-RescheduleEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[long-running-ExecuteSuccessEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[long-running-ExecuteFailureEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_resumed_task_releases_resources[long-running-RescheduleEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_clean_log PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_running_task_in_all_running_tasks[executing] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_running_task_in_all_running_tasks[long-running] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[executing-ExecuteSuccessEvent] PASSED [ 98%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[executing-ExecuteFailureEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[executing-RescheduleEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[long-running-ExecuteSuccessEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[long-running-ExecuteFailureEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_task_not_in_all_running_tasks[long-running-RescheduleEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[executing-ExecuteSuccessEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[executing-ExecuteFailureEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[executing-RescheduleEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[long-running-ExecuteSuccessEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[long-running-ExecuteFailureEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_done_resumed_task_not_in_all_running_tasks[long-running-RescheduleEvent] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_gather_dep_failure PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_recompute_erred_task PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_transfer_incoming_metrics PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_throttling_does_not_affect_first_transfer PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_message_target_does_not_affect_first_transfer_on_different_worker PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_throttle_incoming_transfers_on_count_limit PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_throttling_incoming_transfer_on_transfer_bytes_same_worker PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_throttling_incoming_transfer_on_transfer_bytes_different_workers PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_do_not_throttle_connections_while_below_threshold PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_throttle_on_transfer_bytes_regardless_of_threshold PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_worker_nbytes[executing] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_worker_nbytes[long-running] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_fetch_count PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_task_counter PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_flight[fail] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_flight[success] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_flight[no-key] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_flight[busy] PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_flight_unused_peer PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_while_in_fetch PASSED [ 99%] distributed/tests/test_worker_state_machine.py::test_remove_worker_unknown PASSED [100%] =================================== FAILURES =================================== _____________________________ test_rechunk_method ______________________________ fut = , timeout = 29.72892400296405 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut E asyncio.exceptions.CancelledError distributed/utils.py:1960: CancelledError The above exception was the direct cause of the following exception: async def async_fn(): result = None with dask.config.set(config): async with ( _cluster_factory() as (s, workers), _client_factory(s) as c, ): args = [s] + workers if c is not None: args = [c] + args try: coro = func(*args, *outer_args, **kwargs) task = asyncio.create_task(coro) coro2 = utils_wait_for( asyncio.shield(task), timeout=deadline.remaining ) > result = await coro2 distributed/utils_test.py:1011: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError During handling of the above exception, another exception occurred: fut = ._..test_func..async_fn at 0x7f220244f4d0> timeout = 60 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut distributed/utils.py:1960: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils_test.py:1031: in async_fn await asyncio.sleep(0.01) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ delay = 0.01, result = None async def sleep(delay, result=None): """Coroutine that completes after a given time (in seconds).""" if delay <= 0: await __sleep0() return result loop = events.get_running_loop() future = loop.create_future() h = loop.call_later(delay, futures._set_result_unless_cancelled, future, result) try: > return await future E asyncio.exceptions.CancelledError /usr/lib/python3.12/asyncio/tasks.py:665: CancelledError The above exception was the direct cause of the following exception: args = (), kwds = {} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) /usr/lib/python3.12/contextlib.py:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.12/contextlib.py:81: in inner return func(*args, **kwds) distributed/utils_test.py:1102: in test_func return _run_and_close_tornado(async_fn_outer) distributed/utils_test.py:378: in _run_and_close_tornado return asyncio_run(inner_fn(), loop_factory=get_loop_factory()) /usr/lib/python3.12/asyncio/runners.py:194: in run return runner.run(main) /usr/lib/python3.12/asyncio/runners.py:118: in run return self._loop.run_until_complete(task) /usr/lib/python3.12/asyncio/base_events.py:687: in run_until_complete return future.result() distributed/utils_test.py:375: in inner_fn return await async_fn(*args, **kwargs) distributed/utils_test.py:1099: in async_fn_outer return await utils_wait_for(async_fn(), timeout=timeout * 2) distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_rechunk_method.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 06:38:13,077 - distributed.scheduler - INFO - State start 2024-10-22 06:38:13,090 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:35785 2024-10-22 06:38:13,090 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:32813/status 2024-10-22 06:38:13,091 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 06:38:13,146 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:43603 2024-10-22 06:38:13,146 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:43603 2024-10-22 06:38:13,147 - distributed.worker - INFO - Worker name: 0 2024-10-22 06:38:13,147 - distributed.worker - INFO - dashboard at: 127.0.0.1:43003 2024-10-22 06:38:13,147 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:35785 2024-10-22 06:38:13,147 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,147 - distributed.worker - INFO - Threads: 1 2024-10-22 06:38:13,147 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:38:13,147 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-41ycmmlp 2024-10-22 06:38:13,147 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,148 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:36355 2024-10-22 06:38:13,148 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:36355 2024-10-22 06:38:13,161 - distributed.worker - INFO - Worker name: 1 2024-10-22 06:38:13,161 - distributed.worker - INFO - dashboard at: 127.0.0.1:40887 2024-10-22 06:38:13,161 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:35785 2024-10-22 06:38:13,161 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,161 - distributed.worker - INFO - Threads: 2 2024-10-22 06:38:13,161 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:38:13,161 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-pnf_e471 2024-10-22 06:38:13,161 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,215 - distributed.scheduler - INFO - Register worker 2024-10-22 06:38:13,238 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:43603 2024-10-22 06:38:13,238 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:33384 2024-10-22 06:38:13,239 - distributed.scheduler - INFO - Register worker 2024-10-22 06:38:13,265 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:36355 2024-10-22 06:38:13,265 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:33392 2024-10-22 06:38:13,266 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:38:13,266 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:38:13,267 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:35785 2024-10-22 06:38:13,267 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,268 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:35785 2024-10-22 06:38:13,268 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:38:13,268 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:35785 2024-10-22 06:38:13,277 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:35785 2024-10-22 06:38:13,315 - distributed.scheduler - INFO - Receive client connection: Client-dd6be4fc-8fca-11ef-ad33-7d0752ea3fc4 2024-10-22 06:38:13,325 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:33398 2024-10-22 06:38:14,627 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0041321168557503bfe77cc4c2fa8ad5 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 2, 1, 2, 1, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:14,756 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 88d51be1b20da5590bf384cab709a13c initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 2, 0, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:14,887 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d6585e17cb212ca77cc0912fa727b367 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 1, 2, 2, 2, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:15,056 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17165189af0e28bda349675e83aa4d40 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 1, 1, 2, 2, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:16,300 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e39985504976966b80ba04bba3046983 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 1, 0, 2, 1, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:16,466 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 00ee5e70b3d35104959c482bab9c0218 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 2, 1, 2, 1, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:16,655 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 473b2aff313c060220c21ae000400c5e initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 2, 2, 2, 1, 1, 1) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:16,739 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 316f4fc008c10683a5956fc2c5eb530c initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 2, 0, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,008 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ff3a74af796c0c0c0bac8500f84a9b80 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 1, 1, 2, 1, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,040 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 45b61ea6e3ad26f4d76d17ad5bc53779 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 1, 2, 2, 1, 1, 1) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:17,216 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 32b027e1b4dc3e43d2fa6f8c98119678 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 1, 0, 2, 1, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,262 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1fbb7727ee6f46056e97d95d524fb463 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 2, 1, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,575 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a802f8a55cac32673836f4a1905c0c9f initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 2, 0, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,698 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 72f360d254fb62e177e2e195b3766722 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 1, 2, 1, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:17,787 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3551bf46927be8ed9ead691352051359 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 2, 2, 2, 2, 2, 1) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:17,902 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 71bb09e1ea92dcc5ff33bc820a96326a initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 1, 2, 0, 1, 1, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:19,396 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ed7b1e59d6dd11800295684a812a06a initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 2, 2, 1, 1, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:19,877 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 60c8256a13826c1d697ed71561097ee6 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 1, 2, 1, 1, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:19,943 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f94b3b86bf8ff017af1b7b4dcf4b6518 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 1, 1, 1, 2, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:20,084 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle de1c3beca688f341f59797f18c823496 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 1, 2, 2, 1, 1, 2, 1) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:20,156 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a527bfb95948458ee116d8f15fd10351 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 1, 0, 1, 2, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:20,478 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cd8bcdfa6ff3c5f40ab1da84d00cf452 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 1, 1, 2, 1, 1, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:24,268 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 43e10d58d11aae5ceb65e03b73aaafd7 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 0, 2, 2, 2, 0, 2) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:24,709 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0041321168557503bfe77cc4c2fa8ad5 deactivated due to stimulus 'task-finished-1729528704.705562' 2024-10-22 06:38:25,177 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 71c5b020e22591132efe133e3709d1f4 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 0, 1, 2, 2, 0, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:25,877 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 88d51be1b20da5590bf384cab709a13c deactivated due to stimulus 'task-finished-1729528705.7707915' 2024-10-22 06:38:26,923 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 87d19895dd066470e26dc64fd4cb9582 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 0, 2, 2, 1, 0, 2) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:27,239 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0a07c2129781bf3bf34aa97f8ebc2023 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 0, 1, 2, 1, 0, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:27,565 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b82ed302966a2d3e696a15394198e6c3 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 0, 2, 1, 2, 0, 2) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:28,620 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a199acda10ad5b63f165cc2bca5cd9e3 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 0, 1, 1, 2, 0, 1) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:30,460 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ae340cebcfd3c13c9a8b488237d4a541 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 2, 0, 0, 2, 2, 0, 0) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:30,474 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a607bd2bdd49a31c1cf81f58192aee43 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 1, 0, 2, 1, 1, 0, 2) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:31,264 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d6585e17cb212ca77cc0912fa727b367 deactivated due to stimulus 'task-finished-1729528711.2612853' 2024-10-22 06:38:31,715 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 43e10d58d11aae5ceb65e03b73aaafd7 deactivated due to stimulus 'task-finished-1729528711.6895661' 2024-10-22 06:38:32,146 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f4fdfbabe2648c343bace34c4f069508 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 1, 0, 0, 2, 1, 0, 0) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:34,773 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b4f1a217afa2b30d17d7658f7ec52e16 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 2, 0, 0, 1, 2, 0, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:34,866 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17165189af0e28bda349675e83aa4d40 deactivated due to stimulus 'task-finished-1729528714.8628566' 2024-10-22 06:38:35,536 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e39985504976966b80ba04bba3046983 deactivated due to stimulus 'task-finished-1729528715.5335236' 2024-10-22 06:38:35,836 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ae340cebcfd3c13c9a8b488237d4a541 deactivated due to stimulus 'task-finished-1729528715.7566056' 2024-10-22 06:38:36,109 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a53734a94dad88ff2866c896818900a3 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 2, 2, 2, 0, 2, 2) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:36,262 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 71c5b020e22591132efe133e3709d1f4 deactivated due to stimulus 'task-finished-1729528716.2057667' 2024-10-22 06:38:36,263 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 25148c801305cb4d040d00a470d3a564 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 2, 1, 2, 0, 2, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:37,418 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 021652c22b78524e5ffb746fbb813b82 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 1, 2, 2, 0, 1, 2) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:37,877 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 6b0fa3f0145e9021527b443a2bd9c9cb initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 0, 2, 2, 1, 0, 2, 2) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:38,210 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8a32e2d0e79724374a4d5dea34362e3d initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 1, 1, 2, 0, 1, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:38,879 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0418c3198fb3bc1161ac90636208fd5b initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 0, 2, 1, 1, 0, 2, 1) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:39,834 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b3cbdc229060e31562dac51c85d8502b initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 2, 0, 2, 0, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:40,970 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 473b2aff313c060220c21ae000400c5e deactivated due to stimulus 'task-finished-1729528720.9155438' 2024-10-22 06:38:42,018 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d26b76a95658f00d8c2e21974fb81b88 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 2, 0, 1, 0, 2, 0, 1, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:42,097 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ed624c55ccda318bb49f0905f3a99ef initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 0, 1, 2, 1, 0, 1, 2) executed on worker tcp://127.0.0.1:43603 2024-10-22 06:38:42,562 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 76bb3ecca0c28e4b338288992a9ba670 initialized by task ('rechunk-transfer-3944b7a19999afcc7f2f0bba85002ac1', 1, 0, 2, 0, 1, 0, 2, 0) executed on worker tcp://127.0.0.1:36355 2024-10-22 06:38:42,771 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a53734a94dad88ff2866c896818900a3 deactivated due to stimulus 'task-finished-1729528722.7543442' 2024-10-22 06:41:20,903 - distributed.core - INFO - Event loop was unresponsive in Scheduler for 158.28s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:41:20,938 - distributed.core - INFO - Event loop was unresponsive in Worker for 158.12s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:41:20,964 - distributed.core - INFO - Event loop was unresponsive in Worker for 158.13s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:41:21,112 - distributed.scheduler - INFO - Remove client Client-dd6be4fc-8fca-11ef-ad33-7d0752ea3fc4 2024-10-22 06:41:21,141 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f94b3b86bf8ff017af1b7b4dcf4b6518 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,159 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a607bd2bdd49a31c1cf81f58192aee43 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,160 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 32b027e1b4dc3e43d2fa6f8c98119678 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,174 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 72f360d254fb62e177e2e195b3766722 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,208 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a802f8a55cac32673836f4a1905c0c9f deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,224 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b82ed302966a2d3e696a15394198e6c3 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,238 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a527bfb95948458ee116d8f15fd10351 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,240 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 45b61ea6e3ad26f4d76d17ad5bc53779 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,254 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle de1c3beca688f341f59797f18c823496 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,256 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 00ee5e70b3d35104959c482bab9c0218 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,271 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b4f1a217afa2b30d17d7658f7ec52e16 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,271 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 316f4fc008c10683a5956fc2c5eb530c deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,272 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 87d19895dd066470e26dc64fd4cb9582 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,286 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ff3a74af796c0c0c0bac8500f84a9b80 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,301 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b3cbdc229060e31562dac51c85d8502b deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,302 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 021652c22b78524e5ffb746fbb813b82 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,303 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f4fdfbabe2648c343bace34c4f069508 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,317 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 71bb09e1ea92dcc5ff33bc820a96326a deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,318 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cd8bcdfa6ff3c5f40ab1da84d00cf452 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,320 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 6b0fa3f0145e9021527b443a2bd9c9cb deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,335 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0a07c2129781bf3bf34aa97f8ebc2023 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,349 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 25148c801305cb4d040d00a470d3a564 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,350 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ed624c55ccda318bb49f0905f3a99ef deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,352 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle a199acda10ad5b63f165cc2bca5cd9e3 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,366 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d26b76a95658f00d8c2e21974fb81b88 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,367 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8a32e2d0e79724374a4d5dea34362e3d deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,384 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 76bb3ecca0c28e4b338288992a9ba670 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,398 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0418c3198fb3bc1161ac90636208fd5b deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,400 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 60c8256a13826c1d697ed71561097ee6 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,434 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ed7b1e59d6dd11800295684a812a06a deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,451 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3551bf46927be8ed9ead691352051359 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,467 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1fbb7727ee6f46056e97d95d524fb463 deactivated due to stimulus 'remove-client-1729528881.1121778' 2024-10-22 06:41:21,498 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:33398; closing. 2024-10-22 06:41:21,514 - distributed.scheduler - INFO - Remove client Client-dd6be4fc-8fca-11ef-ad33-7d0752ea3fc4 2024-10-22 06:41:21,678 - distributed.scheduler - INFO - Close client connection: Client-dd6be4fc-8fca-11ef-ad33-7d0752ea3fc4 2024-10-22 06:41:21,678 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:43603. Reason: worker-close 2024-10-22 06:41:21,678 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:36355. Reason: worker-close 2024-10-22 06:41:21,720 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:41:21,789 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:41:21,790 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:41:21,790 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:41:22,015 - distributed.shuffle._comms - ERROR - SchedulerShuffleState<87d19895dd066470e26dc64fd4cb9582[40]> forgotten Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 203, in _send self.raise_if_closed() File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 266, in raise_if_closed raise self._exception RuntimeError: SchedulerShuffleState<87d19895dd066470e26dc64fd4cb9582[40]> forgotten 2024-10-22 06:41:22,078 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '45b61ea6e3ad26f4d76d17ad5bc53779' 2024-10-22 06:41:22,094 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '316f4fc008c10683a5956fc2c5eb530c' 2024-10-22 06:41:22,184 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '45b61ea6e3ad26f4d76d17ad5bc53779' 2024-10-22 06:41:22,224 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '316f4fc008c10683a5956fc2c5eb530c' 2024-10-22 06:41:22,279 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '87d19895dd066470e26dc64fd4cb9582' 2024-10-22 06:41:22,316 - distributed.shuffle._comms - ERROR - '45b61ea6e3ad26f4d76d17ad5bc53779' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '45b61ea6e3ad26f4d76d17ad5bc53779' 2024-10-22 06:41:22,354 - distributed.shuffle._comms - ERROR - '316f4fc008c10683a5956fc2c5eb530c' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '316f4fc008c10683a5956fc2c5eb530c' 2024-10-22 06:41:22,451 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '87d19895dd066470e26dc64fd4cb9582' 2024-10-22 06:41:22,507 - distributed.shuffle._comms - ERROR - '87d19895dd066470e26dc64fd4cb9582' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '87d19895dd066470e26dc64fd4cb9582' 2024-10-22 06:41:22,544 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:33384; closing. 2024-10-22 06:41:22,554 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729528882.5539432') 2024-10-22 06:41:22,554 - distributed.core - INFO - Connection to tcp://127.0.0.1:35785 has been closed. 2024-10-22 06:41:22,570 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:33392; closing. 2024-10-22 06:41:22,571 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729528882.5709538') 2024-10-22 06:41:22,571 - distributed.scheduler - INFO - Lost all workers 2024-10-22 06:41:22,571 - distributed.core - INFO - Connection to tcp://127.0.0.1:35785 has been closed. 2024-10-22 06:41:22,604 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 06:41:22,613 - distributed.scheduler - INFO - Scheduler closing all comms __________ test_rechunk_with_partially_unknown_dimension[x3-chunks3] ___________ fut = , timeout = 29.69062902405858 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut E asyncio.exceptions.CancelledError distributed/utils.py:1960: CancelledError The above exception was the direct cause of the following exception: async def async_fn(): result = None with dask.config.set(config): async with ( _cluster_factory() as (s, workers), _client_factory(s) as c, ): args = [s] + workers if c is not None: args = [c] + args try: coro = func(*args, *outer_args, **kwargs) task = asyncio.create_task(coro) coro2 = utils_wait_for( asyncio.shield(task), timeout=deadline.remaining ) > result = await coro2 distributed/utils_test.py:1011: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError During handling of the above exception, another exception occurred: fut = ._..test_func..async_fn at 0x7f220244f870> timeout = 60 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut distributed/utils.py:1960: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils_test.py:1031: in async_fn await asyncio.sleep(0.01) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ delay = 0.01, result = None async def sleep(delay, result=None): """Coroutine that completes after a given time (in seconds).""" if delay <= 0: await __sleep0() return result loop = events.get_running_loop() future = loop.create_future() h = loop.call_later(delay, futures._set_result_unless_cancelled, future, result) try: > return await future E asyncio.exceptions.CancelledError /usr/lib/python3.12/asyncio/tasks.py:665: CancelledError The above exception was the direct cause of the following exception: args = () kwds = {'chunks': (None, 5), 'x': dask.array} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) /usr/lib/python3.12/contextlib.py:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.12/contextlib.py:81: in inner return func(*args, **kwds) distributed/utils_test.py:1102: in test_func return _run_and_close_tornado(async_fn_outer) distributed/utils_test.py:378: in _run_and_close_tornado return asyncio_run(inner_fn(), loop_factory=get_loop_factory()) /usr/lib/python3.12/asyncio/runners.py:194: in run return runner.run(main) /usr/lib/python3.12/asyncio/runners.py:118: in run return self._loop.run_until_complete(task) /usr/lib/python3.12/asyncio/base_events.py:687: in run_until_complete return future.result() distributed/utils_test.py:375: in inner_fn return await async_fn(*args, **kwargs) distributed/utils_test.py:1099: in async_fn_outer return await utils_wait_for(async_fn(), timeout=timeout * 2) distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_rechunk_with_partially_unknown_dimension.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 06:42:49,821 - distributed.scheduler - INFO - State start 2024-10-22 06:42:49,845 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:42749 2024-10-22 06:42:49,845 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:44325/status 2024-10-22 06:42:49,846 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 06:42:49,888 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:40781 2024-10-22 06:42:49,888 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:40781 2024-10-22 06:42:49,888 - distributed.worker - INFO - Worker name: 0 2024-10-22 06:42:49,888 - distributed.worker - INFO - dashboard at: 127.0.0.1:44133 2024-10-22 06:42:49,888 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:42749 2024-10-22 06:42:49,888 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:49,888 - distributed.worker - INFO - Threads: 1 2024-10-22 06:42:49,888 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:42:49,897 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-kt8ilrzi 2024-10-22 06:42:49,897 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:49,899 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:37287 2024-10-22 06:42:49,899 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:37287 2024-10-22 06:42:49,899 - distributed.worker - INFO - Worker name: 1 2024-10-22 06:42:49,899 - distributed.worker - INFO - dashboard at: 127.0.0.1:37085 2024-10-22 06:42:49,899 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:42749 2024-10-22 06:42:49,900 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:49,900 - distributed.worker - INFO - Threads: 2 2024-10-22 06:42:49,900 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:42:49,900 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-kb3p4n4x 2024-10-22 06:42:49,900 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:49,980 - distributed.scheduler - INFO - Register worker 2024-10-22 06:42:50,017 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:40781 2024-10-22 06:42:50,017 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:34508 2024-10-22 06:42:50,018 - distributed.scheduler - INFO - Register worker 2024-10-22 06:42:50,043 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:37287 2024-10-22 06:42:50,043 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:34522 2024-10-22 06:42:50,044 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:42:50,044 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:42:50,054 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:42749 2024-10-22 06:42:50,054 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:50,054 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:42749 2024-10-22 06:42:50,054 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:42:50,055 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:42749 2024-10-22 06:42:50,056 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:42749 2024-10-22 06:42:50,095 - distributed.scheduler - INFO - Receive client connection: Client-8264f701-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:42:50,115 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:34524 2024-10-22 06:42:50,551 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17261aa4df64de4bbec9ec8f778b6988 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-17261aa4df64de4bbec9ec8f778b6988' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:50,552 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e59664493e67b4922535dece242d1aab initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e59664493e67b4922535dece242d1aab' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:50,552 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bce680412476b294e188ce6816b7e2cd initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-bce680412476b294e188ce6816b7e2cd' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:50,960 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d818f5d4a45fa1d93be4d23875883b80 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d818f5d4a45fa1d93be4d23875883b80' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:50,960 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01db6bbc0c226a1ef7d4a6b3f462f6bb initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-01db6bbc0c226a1ef7d4a6b3f462f6bb' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:50,970 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 31fcedb5edb6afb1da8e949ae718fd04 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-31fcedb5edb6afb1da8e949ae718fd04' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:51,319 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01e1fb53041e5fdb0d29bbe0a220a186 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-01e1fb53041e5fdb0d29bbe0a220a186' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:51,320 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ddb3383fd4d500ceb54cf08b5b6afae1 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-ddb3383fd4d500ceb54cf08b5b6afae1' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:51,320 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c2ee4274eae62a3e133ce55a83985341 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-c2ee4274eae62a3e133ce55a83985341' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:52,042 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e59664493e67b4922535dece242d1aab deactivated due to stimulus 'task-finished-1729528972.014734' 2024-10-22 06:42:52,205 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17261aa4df64de4bbec9ec8f778b6988 deactivated due to stimulus 'task-finished-1729528972.123714' 2024-10-22 06:42:52,499 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc599f874b11835571ee6bcce1edc1d8 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-cc599f874b11835571ee6bcce1edc1d8' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:52,518 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bce680412476b294e188ce6816b7e2cd deactivated due to stimulus 'task-finished-1729528972.470268' 2024-10-22 06:42:52,560 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d818f5d4a45fa1d93be4d23875883b80 deactivated due to stimulus 'task-finished-1729528972.5204387' 2024-10-22 06:42:52,790 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01db6bbc0c226a1ef7d4a6b3f462f6bb deactivated due to stimulus 'task-finished-1729528972.7587216' 2024-10-22 06:42:53,195 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 31fcedb5edb6afb1da8e949ae718fd04 deactivated due to stimulus 'task-finished-1729528973.193394' 2024-10-22 06:42:53,230 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e765252ecfc2b6ffd5668c3b385a3174 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e765252ecfc2b6ffd5668c3b385a3174' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:53,270 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ddb3383fd4d500ceb54cf08b5b6afae1 deactivated due to stimulus 'task-finished-1729528973.2271366' 2024-10-22 06:42:53,408 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 84386ba0af6f45a3973ad458a14d56c5 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-84386ba0af6f45a3973ad458a14d56c5' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:53,734 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c81c046e65c84ae49e04fd567198f65c initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-c81c046e65c84ae49e04fd567198f65c' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:53,946 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c2ee4274eae62a3e133ce55a83985341 deactivated due to stimulus 'task-finished-1729528973.8805308' 2024-10-22 06:42:54,212 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc599f874b11835571ee6bcce1edc1d8 deactivated due to stimulus 'task-finished-1729528974.0711687' 2024-10-22 06:42:54,345 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01e1fb53041e5fdb0d29bbe0a220a186 deactivated due to stimulus 'task-finished-1729528974.2910295' 2024-10-22 06:42:54,347 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d739332393d2d6dae0f074f82be858b3 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d739332393d2d6dae0f074f82be858b3' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:54,444 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e765252ecfc2b6ffd5668c3b385a3174 deactivated due to stimulus 'task-finished-1729528974.387503' 2024-10-22 06:42:54,803 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 59881a6059b5a68660aef4b1433f259a initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-59881a6059b5a68660aef4b1433f259a' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:55,236 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e67f678006c2dfb420382b0653e7e278 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-e67f678006c2dfb420382b0653e7e278' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:55,461 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b79d0d8845743010f9e3088a0beb18d1 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-b79d0d8845743010f9e3088a0beb18d1' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:55,463 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c81c046e65c84ae49e04fd567198f65c deactivated due to stimulus 'task-finished-1729528975.446447' 2024-10-22 06:42:55,802 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 84386ba0af6f45a3973ad458a14d56c5 deactivated due to stimulus 'task-finished-1729528975.7701173' 2024-10-22 06:42:55,888 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f9160caf490b8329e7669ecbd1ac041e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-f9160caf490b8329e7669ecbd1ac041e' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:56,117 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d739332393d2d6dae0f074f82be858b3 deactivated due to stimulus 'task-finished-1729528976.1140413' 2024-10-22 06:42:56,195 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 59881a6059b5a68660aef4b1433f259a deactivated due to stimulus 'task-finished-1729528976.1796134' 2024-10-22 06:42:56,234 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e67f678006c2dfb420382b0653e7e278 deactivated due to stimulus 'task-finished-1729528976.2199976' 2024-10-22 06:42:56,242 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec3af6e3853fbf1e077a45ec7a358cac initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-ec3af6e3853fbf1e077a45ec7a358cac' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:56,559 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec792f9b679daef65c0ec6556b5e9c9f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-ec792f9b679daef65c0ec6556b5e9c9f' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:56,732 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle df475960043b4dab9b72a8cebe24fbf0 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-df475960043b4dab9b72a8cebe24fbf0' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:56,919 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b79d0d8845743010f9e3088a0beb18d1 deactivated due to stimulus 'task-finished-1729528976.757466' 2024-10-22 06:42:57,290 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f9160caf490b8329e7669ecbd1ac041e deactivated due to stimulus 'task-finished-1729528977.2782779' 2024-10-22 06:42:57,321 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e77aa0a14aa79e0a383a0b353abba873 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e77aa0a14aa79e0a383a0b353abba873' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:58,207 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec3af6e3853fbf1e077a45ec7a358cac deactivated due to stimulus 'task-finished-1729528978.0769322' 2024-10-22 06:42:58,453 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle df475960043b4dab9b72a8cebe24fbf0 deactivated due to stimulus 'task-finished-1729528978.450613' 2024-10-22 06:42:58,571 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 714ef9e25e8b5c1d5b3451d982311c2e initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-714ef9e25e8b5c1d5b3451d982311c2e' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:59,016 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec792f9b679daef65c0ec6556b5e9c9f deactivated due to stimulus 'task-finished-1729528978.9305575' 2024-10-22 06:42:59,074 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b8fcb44d156414c630e1d6ff0df77c1b initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-b8fcb44d156414c630e1d6ff0df77c1b' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:42:59,361 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 57d52a42fcd6be719e65e6f9840d0cdb initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-57d52a42fcd6be719e65e6f9840d0cdb' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:59,470 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9c215f5d25889bd6c1d4a0fb35eb468 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-b9c215f5d25889bd6c1d4a0fb35eb468' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:42:59,682 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e77aa0a14aa79e0a383a0b353abba873 deactivated due to stimulus 'task-finished-1729528979.571674' 2024-10-22 06:42:59,699 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f385202a9df8dca26b51e3b27030056b initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-f385202a9df8dca26b51e3b27030056b' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:00,420 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 714ef9e25e8b5c1d5b3451d982311c2e deactivated due to stimulus 'task-finished-1729528980.362628' 2024-10-22 06:43:00,651 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cf4000d0bbe7b2713a9b81d90981fe9b initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-cf4000d0bbe7b2713a9b81d90981fe9b' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:00,678 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1164a44fb53f1d99c26d483a9f7618c2 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-1164a44fb53f1d99c26d483a9f7618c2' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:00,763 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 57d52a42fcd6be719e65e6f9840d0cdb deactivated due to stimulus 'task-finished-1729528980.713456' 2024-10-22 06:43:01,206 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3eabd736c25cbfb05d814f9f3d9bfc98 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-3eabd736c25cbfb05d814f9f3d9bfc98' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:01,210 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0fa082cb36216f52a7836b89276b16a9 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-0fa082cb36216f52a7836b89276b16a9' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:01,347 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f385202a9df8dca26b51e3b27030056b deactivated due to stimulus 'task-finished-1729528981.3423245' 2024-10-22 06:43:01,509 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9c215f5d25889bd6c1d4a0fb35eb468 deactivated due to stimulus 'task-finished-1729528981.5068707' 2024-10-22 06:43:01,554 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cf4000d0bbe7b2713a9b81d90981fe9b deactivated due to stimulus 'task-finished-1729528981.5318544' 2024-10-22 06:43:01,567 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 11f1f6ff4fd0902dbfb6a01bfdb5359d initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-11f1f6ff4fd0902dbfb6a01bfdb5359d' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:01,688 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b8fcb44d156414c630e1d6ff0df77c1b deactivated due to stimulus 'task-finished-1729528981.6691968' 2024-10-22 06:43:01,835 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3eabd736c25cbfb05d814f9f3d9bfc98 deactivated due to stimulus 'task-finished-1729528981.8277154' 2024-10-22 06:43:01,896 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 4a20b1bca8a7c0bc5fe798d5d474efe7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-4a20b1bca8a7c0bc5fe798d5d474efe7' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:02,122 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8f6377159296c2aea721b80be7444955 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-8f6377159296c2aea721b80be7444955' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:02,174 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1164a44fb53f1d99c26d483a9f7618c2 deactivated due to stimulus 'task-finished-1729528982.1533895' 2024-10-22 06:43:02,303 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 23c3e0a6318f8719413afcb7fb1c165c initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-23c3e0a6318f8719413afcb7fb1c165c' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:02,333 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2037175d04234fb783dd45ef54a900ac initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2037175d04234fb783dd45ef54a900ac' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:02,682 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0cc7ac64edd3e809e1ee98f4e84d65e4 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-0cc7ac64edd3e809e1ee98f4e84d65e4' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:02,787 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle fd6cd470d74c25edf76b3f4dc74c12c0 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-fd6cd470d74c25edf76b3f4dc74c12c0' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:02,788 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 4a20b1bca8a7c0bc5fe798d5d474efe7 deactivated due to stimulus 'task-finished-1729528982.78543' 2024-10-22 06:43:03,000 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8f6377159296c2aea721b80be7444955 deactivated due to stimulus 'task-finished-1729528982.9971752' 2024-10-22 06:43:03,017 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 11f1f6ff4fd0902dbfb6a01bfdb5359d deactivated due to stimulus 'task-finished-1729528982.9979885' 2024-10-22 06:43:03,116 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 23c3e0a6318f8719413afcb7fb1c165c deactivated due to stimulus 'task-finished-1729528983.113419' 2024-10-22 06:43:03,116 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0fa082cb36216f52a7836b89276b16a9 deactivated due to stimulus 'task-finished-1729528983.1135314' 2024-10-22 06:43:03,243 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 92126407b7617e8e1154192b00785cb2 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-92126407b7617e8e1154192b00785cb2' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:03,244 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 204b3c4c3770c441622961008d21fe35 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-204b3c4c3770c441622961008d21fe35' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:03,514 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2037175d04234fb783dd45ef54a900ac deactivated due to stimulus 'task-finished-1729528983.5131118' 2024-10-22 06:43:03,615 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0cc7ac64edd3e809e1ee98f4e84d65e4 deactivated due to stimulus 'task-finished-1729528983.6038828' 2024-10-22 06:43:03,779 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle fd6cd470d74c25edf76b3f4dc74c12c0 deactivated due to stimulus 'task-finished-1729528983.7774303' 2024-10-22 06:43:03,802 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 204b3c4c3770c441622961008d21fe35 deactivated due to stimulus 'task-finished-1729528983.801175' 2024-10-22 06:43:03,888 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 92126407b7617e8e1154192b00785cb2 deactivated due to stimulus 'task-finished-1729528983.8855662' 2024-10-22 06:43:06,022 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 806d3b72ee98acf548c7fd2bbb441c36 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-806d3b72ee98acf548c7fd2bbb441c36' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:06,023 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bc4eab2e4c69f1370607a11f2a9c4392 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-bc4eab2e4c69f1370607a11f2a9c4392' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:06,142 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d07954b281dba89624cbc6c762797e1f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d07954b281dba89624cbc6c762797e1f' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:07,952 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bcb2f010c2aeaa2bad84b81d2684fa7c initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-bcb2f010c2aeaa2bad84b81d2684fa7c' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:07,953 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 38071ec7b016834cb180426b1b3240a7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-38071ec7b016834cb180426b1b3240a7' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:08,129 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle aa36ff673eca80feec08ff85465e098e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-aa36ff673eca80feec08ff85465e098e' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:08,824 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3c148173ebc946767fe147506184e5d5 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-3c148173ebc946767fe147506184e5d5' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:08,867 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 444ea2ff016bfbcb8c2079368d20079f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-444ea2ff016bfbcb8c2079368d20079f' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:08,868 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0d7e0559a60e99a8106e3804f6ade934 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-0d7e0559a60e99a8106e3804f6ade934' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:09,667 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 806d3b72ee98acf548c7fd2bbb441c36 deactivated due to stimulus 'task-finished-1729528989.6654696' 2024-10-22 06:43:09,744 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c97865205e833b24dffa05881b7a9355 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-c97865205e833b24dffa05881b7a9355' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:09,767 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d07954b281dba89624cbc6c762797e1f deactivated due to stimulus 'task-finished-1729528989.7414484' 2024-10-22 06:43:10,460 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 38071ec7b016834cb180426b1b3240a7 deactivated due to stimulus 'task-finished-1729528990.4580576' 2024-10-22 06:43:10,803 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle aa36ff673eca80feec08ff85465e098e deactivated due to stimulus 'task-finished-1729528990.7988675' 2024-10-22 06:43:11,028 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bcb2f010c2aeaa2bad84b81d2684fa7c deactivated due to stimulus 'task-finished-1729528991.025644' 2024-10-22 06:43:11,244 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bc4eab2e4c69f1370607a11f2a9c4392 deactivated due to stimulus 'task-finished-1729528991.2393222' 2024-10-22 06:43:11,383 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 03a2135c557b0a742259d3bca3a3f5c1 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-03a2135c557b0a742259d3bca3a3f5c1' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:11,384 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 56789b3bb869078422e4e07993326c3e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-56789b3bb869078422e4e07993326c3e' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:11,491 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3c148173ebc946767fe147506184e5d5 deactivated due to stimulus 'task-finished-1729528991.4349582' 2024-10-22 06:43:11,803 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 444ea2ff016bfbcb8c2079368d20079f deactivated due to stimulus 'task-finished-1729528991.8017304' 2024-10-22 06:43:11,899 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0d7e0559a60e99a8106e3804f6ade934 deactivated due to stimulus 'task-finished-1729528991.8601727' 2024-10-22 06:43:11,935 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2482501e003e2a2f1b4ca88493771ffa initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2482501e003e2a2f1b4ca88493771ffa' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:11,976 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc2821197bfe13c370db658898c2ecd7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-cc2821197bfe13c370db658898c2ecd7' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:12,156 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle adf80adecdef85f30b67c30bb5c08ded initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-adf80adecdef85f30b67c30bb5c08ded' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:12,507 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 03a2135c557b0a742259d3bca3a3f5c1 deactivated due to stimulus 'task-finished-1729528992.505512' 2024-10-22 06:43:12,770 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c97865205e833b24dffa05881b7a9355 deactivated due to stimulus 'task-finished-1729528992.761427' 2024-10-22 06:43:12,812 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 554d0b89cc3e30c6daa014ddf6515849 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-554d0b89cc3e30c6daa014ddf6515849' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:13,216 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2482501e003e2a2f1b4ca88493771ffa deactivated due to stimulus 'task-finished-1729528992.9168625' 2024-10-22 06:43:13,451 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle daf954ece88a0906d0a52c3d63d0adbf initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-daf954ece88a0906d0a52c3d63d0adbf' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:13,755 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2c982b75b3cd8f58f452e3c502fc4e35 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2c982b75b3cd8f58f452e3c502fc4e35' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:14,497 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 56789b3bb869078422e4e07993326c3e deactivated due to stimulus 'task-finished-1729528994.4278705' 2024-10-22 06:43:14,703 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle adf80adecdef85f30b67c30bb5c08ded deactivated due to stimulus 'task-finished-1729528994.6794093' 2024-10-22 06:43:14,781 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc2821197bfe13c370db658898c2ecd7 deactivated due to stimulus 'task-finished-1729528994.738729' 2024-10-22 06:43:15,038 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 519e9a5aaf5255b78a02c8a519e86848 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-519e9a5aaf5255b78a02c8a519e86848' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:15,896 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f339247e440184a6c63d166590f7ee64 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-f339247e440184a6c63d166590f7ee64' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:15,986 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c92a48712b5e61e056a3f6d130386701 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-c92a48712b5e61e056a3f6d130386701' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:16,211 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle daf954ece88a0906d0a52c3d63d0adbf deactivated due to stimulus 'task-finished-1729528996.1430075' 2024-10-22 06:43:16,378 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 30ef823fb3e4610092353d5ad026328e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-30ef823fb3e4610092353d5ad026328e' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:16,414 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 702342489700f3b202e200fe562506f8 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-702342489700f3b202e200fe562506f8' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:16,456 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 554d0b89cc3e30c6daa014ddf6515849 deactivated due to stimulus 'task-finished-1729528996.4162838' 2024-10-22 06:43:17,713 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 519e9a5aaf5255b78a02c8a519e86848 deactivated due to stimulus 'task-finished-1729528997.6774778' 2024-10-22 06:43:17,714 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2c982b75b3cd8f58f452e3c502fc4e35 deactivated due to stimulus 'task-finished-1729528997.6776395' 2024-10-22 06:43:18,051 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f339247e440184a6c63d166590f7ee64 deactivated due to stimulus 'task-finished-1729528998.0264425' 2024-10-22 06:43:18,283 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 30ef823fb3e4610092353d5ad026328e deactivated due to stimulus 'task-finished-1729528998.2666261' 2024-10-22 06:43:18,362 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ddb889047c2bc394819b94ad737c871 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-5ddb889047c2bc394819b94ad737c871' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:18,363 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cb20ca5338e4d4b923409c400a63ee36 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-cb20ca5338e4d4b923409c400a63ee36' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:18,536 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c92a48712b5e61e056a3f6d130386701 deactivated due to stimulus 'task-finished-1729528998.531684' 2024-10-22 06:43:18,795 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc3571f3c574576d181370e2b6304447 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-cc3571f3c574576d181370e2b6304447' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:43:19,390 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 52052fbfa6585c8c4a183f4f67724686 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-52052fbfa6585c8c4a183f4f67724686' executed on worker tcp://127.0.0.1:37287 2024-10-22 06:43:19,486 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 702342489700f3b202e200fe562506f8 deactivated due to stimulus 'task-finished-1729528999.4152758' 2024-10-22 06:43:19,808 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9d93f64b0b18e94e48adcc2627ec5dc initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-b9d93f64b0b18e94e48adcc2627ec5dc' executed on worker tcp://127.0.0.1:40781 2024-10-22 06:44:15,716 - distributed.core - INFO - Event loop was unresponsive in Scheduler for 56.35s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:44:15,747 - distributed.core - INFO - Event loop was unresponsive in Worker for 56.15s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:44:15,756 - distributed.core - INFO - Event loop was unresponsive in Worker for 56.13s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:44:15,799 - distributed.scheduler - INFO - Remove client Client-8264f701-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:44:15,821 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ddb889047c2bc394819b94ad737c871 deactivated due to stimulus 'remove-client-1729529055.7996662' 2024-10-22 06:44:15,823 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 52052fbfa6585c8c4a183f4f67724686 deactivated due to stimulus 'remove-client-1729529055.7996662' 2024-10-22 06:44:15,829 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc3571f3c574576d181370e2b6304447 deactivated due to stimulus 'remove-client-1729529055.7996662' 2024-10-22 06:44:15,844 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9d93f64b0b18e94e48adcc2627ec5dc deactivated due to stimulus 'remove-client-1729529055.7996662' 2024-10-22 06:44:15,851 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cb20ca5338e4d4b923409c400a63ee36 deactivated due to stimulus 'remove-client-1729529055.7996662' 2024-10-22 06:44:15,851 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:34524; closing. 2024-10-22 06:44:15,857 - distributed.scheduler - INFO - Remove client Client-8264f701-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:44:15,868 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: 'b9d93f64b0b18e94e48adcc2627ec5dc' 2024-10-22 06:44:15,876 - distributed.scheduler - INFO - Close client connection: Client-8264f701-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:44:15,941 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:40781. Reason: worker-close 2024-10-22 06:44:15,941 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:37287. Reason: worker-close 2024-10-22 06:44:15,942 - distributed.core - ERROR - Exception while handling op shuffle_barrier Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 82, in barrier shuffle = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: 'b9d93f64b0b18e94e48adcc2627ec5dc' 2024-10-22 06:44:15,954 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: 'b9d93f64b0b18e94e48adcc2627ec5dc' 2024-10-22 06:44:15,983 - distributed.scheduler - WARNING - Attempting to reschedule task ('rechunk-p2p-8324e95b1c2ac71dc4d7463e6c44102a', 13, 1), which was not found on the scheduler. Aborting reschedule. 2024-10-22 06:44:15,983 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:44:15,983 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:44:15,984 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:44:15,996 - distributed.shuffle._comms - ERROR - 'b9d93f64b0b18e94e48adcc2627ec5dc' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: 'b9d93f64b0b18e94e48adcc2627ec5dc' 2024-10-22 06:44:16,035 - distributed.comm.tcp - INFO - Connection from tcp://127.0.0.1:60428 closed before handshake completed 2024-10-22 06:44:16,047 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:34522; closing. 2024-10-22 06:44:16,048 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529056.0485575') 2024-10-22 06:44:16,069 - distributed.comm.tcp - INFO - Connection from tcp://127.0.0.1:57200 closed before handshake completed 2024-10-22 06:44:16,070 - distributed.core - INFO - Connection to tcp://127.0.0.1:42749 has been closed. 2024-10-22 06:44:16,087 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:34508; closing. 2024-10-22 06:44:16,088 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529056.0883484') 2024-10-22 06:44:16,088 - distributed.scheduler - INFO - Lost all workers 2024-10-22 06:44:16,093 - distributed.core - INFO - Connection to tcp://127.0.0.1:42749 has been closed. 2024-10-22 06:44:16,167 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 06:44:16,168 - distributed.scheduler - INFO - Scheduler closing all comms __________ test_rechunk_with_partially_unknown_dimension[x5-chunks5] ___________ fut = , timeout = 29.663684797938913 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut E asyncio.exceptions.CancelledError distributed/utils.py:1960: CancelledError The above exception was the direct cause of the following exception: async def async_fn(): result = None with dask.config.set(config): async with ( _cluster_factory() as (s, workers), _client_factory(s) as c, ): args = [s] + workers if c is not None: args = [c] + args try: coro = func(*args, *outer_args, **kwargs) task = asyncio.create_task(coro) coro2 = utils_wait_for( asyncio.shield(task), timeout=deadline.remaining ) > result = await coro2 distributed/utils_test.py:1011: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError During handling of the above exception, another exception occurred: fut = ._..test_func..async_fn at 0x7f220244f4d0> timeout = 60 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut distributed/utils.py:1960: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ distributed/utils_test.py:1031: in async_fn await asyncio.sleep(0.01) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ delay = 0.01, result = None async def sleep(delay, result=None): """Coroutine that completes after a given time (in seconds).""" if delay <= 0: await __sleep0() return result loop = events.get_running_loop() future = loop.create_future() h = loop.call_later(delay, futures._set_result_unless_cancelled, future, result) try: > return await future E asyncio.exceptions.CancelledError /usr/lib/python3.12/asyncio/tasks.py:665: CancelledError The above exception was the direct cause of the following exception: args = () kwds = {'chunks': (None, (5, 5)), 'x': dask.array} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) /usr/lib/python3.12/contextlib.py:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.12/contextlib.py:81: in inner return func(*args, **kwds) distributed/utils_test.py:1102: in test_func return _run_and_close_tornado(async_fn_outer) distributed/utils_test.py:378: in _run_and_close_tornado return asyncio_run(inner_fn(), loop_factory=get_loop_factory()) /usr/lib/python3.12/asyncio/runners.py:194: in run return runner.run(main) /usr/lib/python3.12/asyncio/runners.py:118: in run return self._loop.run_until_complete(task) /usr/lib/python3.12/asyncio/base_events.py:687: in run_until_complete return future.result() distributed/utils_test.py:375: in inner_fn return await async_fn(*args, **kwargs) distributed/utils_test.py:1099: in async_fn_outer return await utils_wait_for(async_fn(), timeout=timeout * 2) distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_rechunk_with_partially_unknown_dimension.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 06:44:48,788 - distributed.scheduler - INFO - State start 2024-10-22 06:44:48,806 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:38535 2024-10-22 06:44:48,806 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:45395/status 2024-10-22 06:44:48,807 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 06:44:48,851 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:45247 2024-10-22 06:44:48,851 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:45247 2024-10-22 06:44:48,851 - distributed.worker - INFO - Worker name: 0 2024-10-22 06:44:48,851 - distributed.worker - INFO - dashboard at: 127.0.0.1:43549 2024-10-22 06:44:48,852 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:38535 2024-10-22 06:44:48,852 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:48,852 - distributed.worker - INFO - Threads: 1 2024-10-22 06:44:48,852 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:44:48,852 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-a5ttnyz2 2024-10-22 06:44:48,852 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:48,867 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:46675 2024-10-22 06:44:48,867 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:46675 2024-10-22 06:44:48,867 - distributed.worker - INFO - Worker name: 1 2024-10-22 06:44:48,867 - distributed.worker - INFO - dashboard at: 127.0.0.1:40109 2024-10-22 06:44:48,867 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:38535 2024-10-22 06:44:48,867 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:48,867 - distributed.worker - INFO - Threads: 2 2024-10-22 06:44:48,868 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:44:48,868 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-ew4d4akn 2024-10-22 06:44:48,868 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:48,947 - distributed.scheduler - INFO - Register worker 2024-10-22 06:44:48,979 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:45247 2024-10-22 06:44:48,980 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:41298 2024-10-22 06:44:48,993 - distributed.scheduler - INFO - Register worker 2024-10-22 06:44:49,018 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:46675 2024-10-22 06:44:49,018 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:41310 2024-10-22 06:44:49,019 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:44:49,020 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:44:49,029 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:38535 2024-10-22 06:44:49,030 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:49,030 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:38535 2024-10-22 06:44:49,030 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:44:49,031 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38535 2024-10-22 06:44:49,031 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38535 2024-10-22 06:44:49,083 - distributed.scheduler - INFO - Receive client connection: Client-c94f2c2d-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:44:49,112 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:41312 2024-10-22 06:44:49,779 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e59664493e67b4922535dece242d1aab initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e59664493e67b4922535dece242d1aab' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:49,780 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01e1fb53041e5fdb0d29bbe0a220a186 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-01e1fb53041e5fdb0d29bbe0a220a186' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:49,793 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17261aa4df64de4bbec9ec8f778b6988 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-17261aa4df64de4bbec9ec8f778b6988' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:50,261 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d818f5d4a45fa1d93be4d23875883b80 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d818f5d4a45fa1d93be4d23875883b80' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:50,263 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 31fcedb5edb6afb1da8e949ae718fd04 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-31fcedb5edb6afb1da8e949ae718fd04' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:50,287 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01db6bbc0c226a1ef7d4a6b3f462f6bb initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-01db6bbc0c226a1ef7d4a6b3f462f6bb' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:50,627 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ddb3383fd4d500ceb54cf08b5b6afae1 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-ddb3383fd4d500ceb54cf08b5b6afae1' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:50,627 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc599f874b11835571ee6bcce1edc1d8 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-cc599f874b11835571ee6bcce1edc1d8' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:50,666 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bce680412476b294e188ce6816b7e2cd initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-bce680412476b294e188ce6816b7e2cd' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:51,258 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e59664493e67b4922535dece242d1aab deactivated due to stimulus 'task-finished-1729529091.245405' 2024-10-22 06:44:51,259 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 17261aa4df64de4bbec9ec8f778b6988 deactivated due to stimulus 'task-finished-1729529091.24559' 2024-10-22 06:44:51,427 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01e1fb53041e5fdb0d29bbe0a220a186 deactivated due to stimulus 'task-finished-1729529091.4106045' 2024-10-22 06:44:51,611 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d818f5d4a45fa1d93be4d23875883b80 deactivated due to stimulus 'task-finished-1729529091.534497' 2024-10-22 06:44:51,618 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c2ee4274eae62a3e133ce55a83985341 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-c2ee4274eae62a3e133ce55a83985341' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:51,785 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 01db6bbc0c226a1ef7d4a6b3f462f6bb deactivated due to stimulus 'task-finished-1729529091.7780151' 2024-10-22 06:44:51,908 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 31fcedb5edb6afb1da8e949ae718fd04 deactivated due to stimulus 'task-finished-1729529091.9021618' 2024-10-22 06:44:52,258 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e765252ecfc2b6ffd5668c3b385a3174 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e765252ecfc2b6ffd5668c3b385a3174' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:52,259 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bce680412476b294e188ce6816b7e2cd deactivated due to stimulus 'task-finished-1729529092.2337945' 2024-10-22 06:44:52,263 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 84386ba0af6f45a3973ad458a14d56c5 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-84386ba0af6f45a3973ad458a14d56c5' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:52,484 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ddb3383fd4d500ceb54cf08b5b6afae1 deactivated due to stimulus 'task-finished-1729529092.4524372' 2024-10-22 06:44:52,637 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c2ee4274eae62a3e133ce55a83985341 deactivated due to stimulus 'task-finished-1729529092.5827184' 2024-10-22 06:44:52,640 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c81c046e65c84ae49e04fd567198f65c initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-c81c046e65c84ae49e04fd567198f65c' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:52,671 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d739332393d2d6dae0f074f82be858b3 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d739332393d2d6dae0f074f82be858b3' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:52,799 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc599f874b11835571ee6bcce1edc1d8 deactivated due to stimulus 'task-finished-1729529092.7415996' 2024-10-22 06:44:52,933 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 59881a6059b5a68660aef4b1433f259a initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-59881a6059b5a68660aef4b1433f259a' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:53,356 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec3af6e3853fbf1e077a45ec7a358cac initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-ec3af6e3853fbf1e077a45ec7a358cac' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:53,486 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e765252ecfc2b6ffd5668c3b385a3174 deactivated due to stimulus 'task-finished-1729529093.4687536' 2024-10-22 06:44:53,638 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e67f678006c2dfb420382b0653e7e278 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-e67f678006c2dfb420382b0653e7e278' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:54,142 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec792f9b679daef65c0ec6556b5e9c9f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-ec792f9b679daef65c0ec6556b5e9c9f' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:54,239 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c81c046e65c84ae49e04fd567198f65c deactivated due to stimulus 'task-finished-1729529094.2016137' 2024-10-22 06:44:54,780 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 59881a6059b5a68660aef4b1433f259a deactivated due to stimulus 'task-finished-1729529094.7429855' 2024-10-22 06:44:54,807 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b79d0d8845743010f9e3088a0beb18d1 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-b79d0d8845743010f9e3088a0beb18d1' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:55,074 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 84386ba0af6f45a3973ad458a14d56c5 deactivated due to stimulus 'task-finished-1729529094.9715421' 2024-10-22 06:44:55,416 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f9160caf490b8329e7669ecbd1ac041e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-f9160caf490b8329e7669ecbd1ac041e' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:56,001 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e67f678006c2dfb420382b0653e7e278 deactivated due to stimulus 'task-finished-1729529095.9538443' 2024-10-22 06:44:56,034 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle df475960043b4dab9b72a8cebe24fbf0 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-df475960043b4dab9b72a8cebe24fbf0' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:56,187 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e77aa0a14aa79e0a383a0b353abba873 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-e77aa0a14aa79e0a383a0b353abba873' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:56,261 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d739332393d2d6dae0f074f82be858b3 deactivated due to stimulus 'task-finished-1729529096.1629183' 2024-10-22 06:44:56,484 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec3af6e3853fbf1e077a45ec7a358cac deactivated due to stimulus 'task-finished-1729529096.4629312' 2024-10-22 06:44:56,758 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle ec792f9b679daef65c0ec6556b5e9c9f deactivated due to stimulus 'task-finished-1729529096.7336948' 2024-10-22 06:44:56,970 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 714ef9e25e8b5c1d5b3451d982311c2e initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-714ef9e25e8b5c1d5b3451d982311c2e' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:57,674 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b8fcb44d156414c630e1d6ff0df77c1b initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-b8fcb44d156414c630e1d6ff0df77c1b' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:57,690 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b79d0d8845743010f9e3088a0beb18d1 deactivated due to stimulus 'task-finished-1729529097.5403302' 2024-10-22 06:44:58,018 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9c215f5d25889bd6c1d4a0fb35eb468 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-b9c215f5d25889bd6c1d4a0fb35eb468' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:58,049 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle df475960043b4dab9b72a8cebe24fbf0 deactivated due to stimulus 'task-finished-1729529097.9584105' 2024-10-22 06:44:58,248 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 57d52a42fcd6be719e65e6f9840d0cdb initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-57d52a42fcd6be719e65e6f9840d0cdb' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:44:58,523 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f385202a9df8dca26b51e3b27030056b initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-f385202a9df8dca26b51e3b27030056b' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:59,249 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 11f1f6ff4fd0902dbfb6a01bfdb5359d initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-11f1f6ff4fd0902dbfb6a01bfdb5359d' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:44:59,763 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 714ef9e25e8b5c1d5b3451d982311c2e deactivated due to stimulus 'task-finished-1729529099.747171' 2024-10-22 06:44:59,929 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f9160caf490b8329e7669ecbd1ac041e deactivated due to stimulus 'task-finished-1729529099.8982058' 2024-10-22 06:45:00,260 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f385202a9df8dca26b51e3b27030056b deactivated due to stimulus 'task-finished-1729529100.2422922' 2024-10-22 06:45:00,410 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 57d52a42fcd6be719e65e6f9840d0cdb deactivated due to stimulus 'task-finished-1729529100.3423104' 2024-10-22 06:45:00,525 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cf4000d0bbe7b2713a9b81d90981fe9b initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-cf4000d0bbe7b2713a9b81d90981fe9b' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:00,599 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1164a44fb53f1d99c26d483a9f7618c2 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-1164a44fb53f1d99c26d483a9f7618c2' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:00,615 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle e77aa0a14aa79e0a383a0b353abba873 deactivated due to stimulus 'task-finished-1729529100.5282543' 2024-10-22 06:45:01,103 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b8fcb44d156414c630e1d6ff0df77c1b deactivated due to stimulus 'task-finished-1729529101.0132995' 2024-10-22 06:45:01,221 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3eabd736c25cbfb05d814f9f3d9bfc98 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-3eabd736c25cbfb05d814f9f3d9bfc98' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:01,270 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle b9c215f5d25889bd6c1d4a0fb35eb468 deactivated due to stimulus 'task-finished-1729529101.223212' 2024-10-22 06:45:01,793 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 11f1f6ff4fd0902dbfb6a01bfdb5359d deactivated due to stimulus 'task-finished-1729529101.765424' 2024-10-22 06:45:01,871 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0fa082cb36216f52a7836b89276b16a9 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-0fa082cb36216f52a7836b89276b16a9' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:02,516 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 4a20b1bca8a7c0bc5fe798d5d474efe7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-4a20b1bca8a7c0bc5fe798d5d474efe7' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:02,816 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8f6377159296c2aea721b80be7444955 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-8f6377159296c2aea721b80be7444955' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:02,955 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cf4000d0bbe7b2713a9b81d90981fe9b deactivated due to stimulus 'task-finished-1729529102.81532' 2024-10-22 06:45:03,449 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3eabd736c25cbfb05d814f9f3d9bfc98 deactivated due to stimulus 'task-finished-1729529103.4023967' 2024-10-22 06:45:03,652 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 23c3e0a6318f8719413afcb7fb1c165c initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-23c3e0a6318f8719413afcb7fb1c165c' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:03,898 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 1164a44fb53f1d99c26d483a9f7618c2 deactivated due to stimulus 'task-finished-1729529103.768584' 2024-10-22 06:45:03,951 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2037175d04234fb783dd45ef54a900ac initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2037175d04234fb783dd45ef54a900ac' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:05,234 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 4a20b1bca8a7c0bc5fe798d5d474efe7 deactivated due to stimulus 'task-finished-1729529105.2185748' 2024-10-22 06:45:05,504 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 8f6377159296c2aea721b80be7444955 deactivated due to stimulus 'task-finished-1729529105.4625776' 2024-10-22 06:45:05,801 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0cc7ac64edd3e809e1ee98f4e84d65e4 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-0cc7ac64edd3e809e1ee98f4e84d65e4' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:05,914 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 23c3e0a6318f8719413afcb7fb1c165c deactivated due to stimulus 'task-finished-1729529105.8030348' 2024-10-22 06:45:06,240 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0fa082cb36216f52a7836b89276b16a9 deactivated due to stimulus 'task-finished-1729529105.964419' 2024-10-22 06:45:06,262 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 92126407b7617e8e1154192b00785cb2 initialized by task 'concatenate-from-dask-array-rechunk-slice-rechunk-transfer-values-shuffle-barrier-92126407b7617e8e1154192b00785cb2' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:06,637 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 204b3c4c3770c441622961008d21fe35 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-204b3c4c3770c441622961008d21fe35' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:06,782 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle fd6cd470d74c25edf76b3f4dc74c12c0 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-fd6cd470d74c25edf76b3f4dc74c12c0' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:07,000 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2037175d04234fb783dd45ef54a900ac deactivated due to stimulus 'task-finished-1729529106.9974322' 2024-10-22 06:45:07,241 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0cc7ac64edd3e809e1ee98f4e84d65e4 deactivated due to stimulus 'task-finished-1729529107.2223475' 2024-10-22 06:45:07,299 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 92126407b7617e8e1154192b00785cb2 deactivated due to stimulus 'task-finished-1729529107.2972493' 2024-10-22 06:45:07,436 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 204b3c4c3770c441622961008d21fe35 deactivated due to stimulus 'task-finished-1729529107.4297621' 2024-10-22 06:45:07,455 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle fd6cd470d74c25edf76b3f4dc74c12c0 deactivated due to stimulus 'task-finished-1729529107.4534297' 2024-10-22 06:45:08,257 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d07954b281dba89624cbc6c762797e1f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-d07954b281dba89624cbc6c762797e1f' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:08,291 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 38071ec7b016834cb180426b1b3240a7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-38071ec7b016834cb180426b1b3240a7' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:08,419 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 806d3b72ee98acf548c7fd2bbb441c36 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-806d3b72ee98acf548c7fd2bbb441c36' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:08,966 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bcb2f010c2aeaa2bad84b81d2684fa7c initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-bcb2f010c2aeaa2bad84b81d2684fa7c' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:09,210 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle aa36ff673eca80feec08ff85465e098e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-aa36ff673eca80feec08ff85465e098e' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:09,273 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3c148173ebc946767fe147506184e5d5 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-3c148173ebc946767fe147506184e5d5' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:11,421 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bc4eab2e4c69f1370607a11f2a9c4392 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-bc4eab2e4c69f1370607a11f2a9c4392' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:11,822 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 444ea2ff016bfbcb8c2079368d20079f initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-444ea2ff016bfbcb8c2079368d20079f' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:12,635 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle d07954b281dba89624cbc6c762797e1f deactivated due to stimulus 'task-finished-1729529112.5942504' 2024-10-22 06:45:12,760 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 38071ec7b016834cb180426b1b3240a7 deactivated due to stimulus 'task-finished-1729529112.7574859' 2024-10-22 06:45:12,875 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c97865205e833b24dffa05881b7a9355 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-c97865205e833b24dffa05881b7a9355' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:12,956 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 806d3b72ee98acf548c7fd2bbb441c36 deactivated due to stimulus 'task-finished-1729529112.921604' 2024-10-22 06:45:13,587 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle aa36ff673eca80feec08ff85465e098e deactivated due to stimulus 'task-finished-1729529113.5624628' 2024-10-22 06:45:13,750 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 03a2135c557b0a742259d3bca3a3f5c1 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-03a2135c557b0a742259d3bca3a3f5c1' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:13,920 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bcb2f010c2aeaa2bad84b81d2684fa7c deactivated due to stimulus 'task-finished-1729529113.8923714' 2024-10-22 06:45:14,274 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 56789b3bb869078422e4e07993326c3e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-56789b3bb869078422e4e07993326c3e' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:14,339 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 3c148173ebc946767fe147506184e5d5 deactivated due to stimulus 'task-finished-1729529114.2459788' 2024-10-22 06:45:15,103 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle bc4eab2e4c69f1370607a11f2a9c4392 deactivated due to stimulus 'task-finished-1729529115.0814767' 2024-10-22 06:45:15,270 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 03a2135c557b0a742259d3bca3a3f5c1 deactivated due to stimulus 'task-finished-1729529115.2598567' 2024-10-22 06:45:15,292 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2482501e003e2a2f1b4ca88493771ffa initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2482501e003e2a2f1b4ca88493771ffa' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:15,406 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle adf80adecdef85f30b67c30bb5c08ded initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-adf80adecdef85f30b67c30bb5c08ded' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:15,439 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 444ea2ff016bfbcb8c2079368d20079f deactivated due to stimulus 'task-finished-1729529115.429617' 2024-10-22 06:45:15,503 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0d7e0559a60e99a8106e3804f6ade934 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-0d7e0559a60e99a8106e3804f6ade934' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:15,532 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc2821197bfe13c370db658898c2ecd7 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-cc2821197bfe13c370db658898c2ecd7' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:15,761 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 554d0b89cc3e30c6daa014ddf6515849 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-554d0b89cc3e30c6daa014ddf6515849' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:16,039 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c97865205e833b24dffa05881b7a9355 deactivated due to stimulus 'task-finished-1729529116.0022666' 2024-10-22 06:45:16,376 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2482501e003e2a2f1b4ca88493771ffa deactivated due to stimulus 'task-finished-1729529116.3733304' 2024-10-22 06:45:16,659 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle adf80adecdef85f30b67c30bb5c08ded deactivated due to stimulus 'task-finished-1729529116.6575682' 2024-10-22 06:45:16,788 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle daf954ece88a0906d0a52c3d63d0adbf initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-daf954ece88a0906d0a52c3d63d0adbf' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:16,989 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 0d7e0559a60e99a8106e3804f6ade934 deactivated due to stimulus 'task-finished-1729529116.899771' 2024-10-22 06:45:16,992 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 519e9a5aaf5255b78a02c8a519e86848 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-519e9a5aaf5255b78a02c8a519e86848' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:17,116 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 56789b3bb869078422e4e07993326c3e deactivated due to stimulus 'task-finished-1729529117.0244699' 2024-10-22 06:45:17,279 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2c982b75b3cd8f58f452e3c502fc4e35 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-2c982b75b3cd8f58f452e3c502fc4e35' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:17,327 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc2821197bfe13c370db658898c2ecd7 deactivated due to stimulus 'task-finished-1729529117.303363' 2024-10-22 06:45:17,504 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c92a48712b5e61e056a3f6d130386701 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-c92a48712b5e61e056a3f6d130386701' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:17,648 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 554d0b89cc3e30c6daa014ddf6515849 deactivated due to stimulus 'task-finished-1729529117.544368' 2024-10-22 06:45:17,749 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f339247e440184a6c63d166590f7ee64 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-f339247e440184a6c63d166590f7ee64' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:17,988 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 30ef823fb3e4610092353d5ad026328e initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-30ef823fb3e4610092353d5ad026328e' executed on worker tcp://127.0.0.1:45247 2024-10-22 06:45:18,330 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc3571f3c574576d181370e2b6304447 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-cc3571f3c574576d181370e2b6304447' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:45:18,494 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle daf954ece88a0906d0a52c3d63d0adbf deactivated due to stimulus 'task-finished-1729529118.450363' 2024-10-22 06:45:18,699 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 519e9a5aaf5255b78a02c8a519e86848 deactivated due to stimulus 'task-finished-1729529118.6648827' 2024-10-22 06:46:07,026 - distributed.core - INFO - Event loop was unresponsive in Scheduler for 48.70s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:46:07,047 - distributed.core - INFO - Event loop was unresponsive in Worker for 48.49s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:46:07,068 - distributed.core - INFO - Event loop was unresponsive in Worker for 48.48s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability. 2024-10-22 06:46:07,159 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ddb889047c2bc394819b94ad737c871 initialized by task 'concatenate-rechunk-transfer-shuffle-barrier-5ddb889047c2bc394819b94ad737c871' executed on worker tcp://127.0.0.1:46675 2024-10-22 06:46:07,207 - distributed.scheduler - INFO - Remove client Client-c94f2c2d-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:46:07,218 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle f339247e440184a6c63d166590f7ee64 deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,219 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 30ef823fb3e4610092353d5ad026328e deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,220 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 2c982b75b3cd8f58f452e3c502fc4e35 deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,241 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle cc3571f3c574576d181370e2b6304447 deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,242 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle 5ddb889047c2bc394819b94ad737c871 deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,244 - distributed.shuffle._scheduler_plugin - WARNING - Shuffle c92a48712b5e61e056a3f6d130386701 deactivated due to stimulus 'remove-client-1729529167.2076478' 2024-10-22 06:46:07,260 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:41312; closing. 2024-10-22 06:46:07,273 - distributed.scheduler - INFO - Remove client Client-c94f2c2d-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:46:07,299 - distributed.scheduler - INFO - Close client connection: Client-c94f2c2d-8fcb-11ef-ad33-7d0752ea3fc4 2024-10-22 06:46:07,299 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:45247. Reason: worker-close 2024-10-22 06:46:07,299 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:46675. Reason: worker-close 2024-10-22 06:46:07,397 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:46:07,397 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:46:07,397 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 06:46:07,452 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,468 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,483 - distributed.core - ERROR - Exception while handling op shuffle_get Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ~~~~~~~~~~~~~~~~~~~~^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,504 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,555 - distributed.core - ERROR - Exception while handling op shuffle_receive Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,622 - distributed.shuffle._comms - ERROR - '5ddb889047c2bc394819b94ad737c871' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,671 - distributed.shuffle._comms - ERROR - '5ddb889047c2bc394819b94ad737c871' Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 836, in wrapper return await func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_comms.py", line 72, in _process await self.send(address, shards) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 226, in send return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 421, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_core.py", line 204, in _send return await self.rpc(address).shuffle_receive( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 970, in _handle_comm result = await result ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 314, in shuffle_receive shuffle_run = await self._get_shuffle_run(shuffle_id, run_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 369, in _get_shuffle_run return await self.shuffle_runs.get_with_run_id( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 118, in get_with_run_id shuffle_run = await self._refresh(shuffle_id=shuffle_id) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 225, in _refresh result = await self._fetch(shuffle_id=shuffle_id, spec=spec, key=key) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_worker_plugin.py", line 189, in _fetch result = await self._plugin.worker.scheduler.shuffle_get( ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1395, in send_recv_from_rpc return await send_recv(comm=comm, op=key, **kwargs) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1179, in send_recv raise exc.with_traceback(tb) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 968, in _handle_comm result = handler(**msg) ^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/shuffle/_scheduler_plugin.py", line 128, in get state = self.active_shuffles[id] ^^^^^^^^^^^^^^^^^ KeyError: '5ddb889047c2bc394819b94ad737c871' 2024-10-22 06:46:07,695 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:41298; closing. 2024-10-22 06:46:07,696 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529167.6961637') 2024-10-22 06:46:07,696 - distributed.core - INFO - Connection to tcp://127.0.0.1:38535 has been closed. 2024-10-22 06:46:07,747 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:41310; closing. 2024-10-22 06:46:07,748 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529167.748279') 2024-10-22 06:46:07,748 - distributed.scheduler - INFO - Lost all workers 2024-10-22 06:46:07,748 - distributed.core - INFO - Connection to tcp://127.0.0.1:38535 has been closed. 2024-10-22 06:46:07,790 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 06:46:07,791 - distributed.scheduler - INFO - Scheduler closing all comms _ test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] _ fut = , timeout = 0.5 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut distributed/utils.py:1960: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Scheduler local=tcp://127.0.0.1:54298 remote=tcp://127.0.0.1:37317> deserializers = None async def read(self, deserializers=None): stream = self.stream if stream is None: raise CommClosedError() fmt = "Q" fmt_size = struct.calcsize(fmt) try: # Don't store multiple numpy or parquet buffers into the same buffer, or # none will be released until all are released. > frames_nosplit_nbytes_bin = await stream.read_bytes(fmt_size) E asyncio.exceptions.CancelledError distributed/comm/tcp.py:225: CancelledError The above exception was the direct cause of the following exception: args = (), kwds = {'close_worker': True, 'intermediate_state': 'cancelled'} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) /usr/lib/python3.12/contextlib.py:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.12/contextlib.py:81: in inner return func(*args, **kwds) distributed/utils_test.py:1102: in test_func return _run_and_close_tornado(async_fn_outer) distributed/utils_test.py:378: in _run_and_close_tornado return asyncio_run(inner_fn(), loop_factory=get_loop_factory()) /usr/lib/python3.12/asyncio/runners.py:194: in run return runner.run(main) /usr/lib/python3.12/asyncio/runners.py:118: in run return self._loop.run_until_complete(task) /usr/lib/python3.12/asyncio/base_events.py:687: in run_until_complete return future.result() distributed/utils_test.py:375: in inner_fn return await async_fn(*args, **kwargs) distributed/utils_test.py:1099: in async_fn_outer return await utils_wait_for(async_fn(), timeout=timeout * 2) distributed/utils.py:1960: in wait_for return await fut distributed/utils_test.py:1000: in async_fn _client_factory(s) as c, /usr/lib/python3.12/contextlib.py:210: in __aenter__ return await anext(self.gen) distributed/utils_test.py:949: in _client_factory async with Client( distributed/client.py:1481: in __aenter__ await self distributed/client.py:1299: in _start await self._ensure_connected(timeout=timeout) distributed/client.py:1385: in _ensure_connected msg = await wait_for(comm.read(), timeout) distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError ----------------------------- Captured stderr call ----------------------------- 2024-10-22 06:51:56,776 - distributed.scheduler - INFO - State start 2024-10-22 06:51:56,781 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:37317 2024-10-22 06:51:56,781 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:45729/status 2024-10-22 06:51:56,781 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 06:51:56,798 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:45635 2024-10-22 06:51:56,798 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:45635 2024-10-22 06:51:56,799 - distributed.worker - INFO - Worker name: 0 2024-10-22 06:51:56,799 - distributed.worker - INFO - dashboard at: 127.0.0.1:43479 2024-10-22 06:51:56,799 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:37317 2024-10-22 06:51:56,799 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,799 - distributed.worker - INFO - Threads: 1 2024-10-22 06:51:56,799 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:51:56,799 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-zanm9xx6 2024-10-22 06:51:56,799 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,813 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:46199 2024-10-22 06:51:56,814 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:46199 2024-10-22 06:51:56,814 - distributed.worker - INFO - Worker name: 1 2024-10-22 06:51:56,814 - distributed.worker - INFO - dashboard at: 127.0.0.1:45065 2024-10-22 06:51:56,814 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:37317 2024-10-22 06:51:56,814 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,814 - distributed.worker - INFO - Threads: 2 2024-10-22 06:51:56,814 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 06:51:56,814 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-lniag47t 2024-10-22 06:51:56,814 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,856 - distributed.scheduler - INFO - Register worker 2024-10-22 06:51:56,883 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:45635 2024-10-22 06:51:56,901 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:54282 2024-10-22 06:51:56,902 - distributed.scheduler - INFO - Register worker 2024-10-22 06:51:56,949 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:46199 2024-10-22 06:51:56,993 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:54284 2024-10-22 06:51:56,994 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:51:56,994 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 06:51:56,995 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:37317 2024-10-22 06:51:56,995 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,995 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:37317 2024-10-22 06:51:56,995 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 06:51:56,996 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:37317 2024-10-22 06:51:56,996 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:37317 2024-10-22 06:51:57,076 - distributed.scheduler - INFO - Receive client connection: Client-c8657398-8fcc-11ef-ad33-7d0752ea3fc4 2024-10-22 06:51:57,275 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:54298 2024-10-22 06:51:57,599 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:45635. Reason: worker-close 2024-10-22 06:51:57,673 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:46199. Reason: worker-close 2024-10-22 06:51:57,673 - distributed.core - INFO - Connection to tcp://127.0.0.1:54298 has been closed. 2024-10-22 06:51:57,673 - distributed.scheduler - INFO - Remove client Client-c8657398-8fcc-11ef-ad33-7d0752ea3fc4 2024-10-22 06:51:57,676 - distributed.scheduler - INFO - Close client connection: Client-c8657398-8fcc-11ef-ad33-7d0752ea3fc4 2024-10-22 06:51:57,682 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:54282; closing. 2024-10-22 06:51:57,682 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:54284; closing. 2024-10-22 06:51:57,683 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529517.6832204') 2024-10-22 06:51:57,683 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729529517.683667') 2024-10-22 06:51:57,683 - distributed.scheduler - INFO - Lost all workers 2024-10-22 06:51:57,684 - distributed.core - INFO - Connection to tcp://127.0.0.1:37317 has been closed. 2024-10-22 06:51:57,697 - distributed.core - INFO - Connection to tcp://127.0.0.1:37317 has been closed. 2024-10-22 06:51:58,098 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 06:51:58,099 - distributed.scheduler - INFO - Scheduler closing all comms ______________________________ test_run_rpc_error ______________________________ fut = , timeout = 0.2 async def wait_for(fut: Awaitable[T], timeout: float) -> T: async with asyncio.timeout(timeout): > return await fut distributed/utils.py:1960: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Scheduler local=tcp://127.0.0.1:38324 remote=tcp://127.0.0.1:39985> deserializers = None async def read(self, deserializers=None): stream = self.stream if stream is None: raise CommClosedError() fmt = "Q" fmt_size = struct.calcsize(fmt) try: # Don't store multiple numpy or parquet buffers into the same buffer, or # none will be released until all are released. > frames_nosplit_nbytes_bin = await stream.read_bytes(fmt_size) E asyncio.exceptions.CancelledError distributed/comm/tcp.py:225: CancelledError The above exception was the direct cause of the following exception: args = (), kwds = {} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) /usr/lib/python3.12/contextlib.py:81: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /usr/lib/python3.12/contextlib.py:81: in inner return func(*args, **kwds) distributed/utils_test.py:1102: in test_func return _run_and_close_tornado(async_fn_outer) distributed/utils_test.py:378: in _run_and_close_tornado return asyncio_run(inner_fn(), loop_factory=get_loop_factory()) /usr/lib/python3.12/asyncio/runners.py:194: in run return runner.run(main) /usr/lib/python3.12/asyncio/runners.py:118: in run return self._loop.run_until_complete(task) /usr/lib/python3.12/asyncio/base_events.py:687: in run_until_complete return future.result() distributed/utils_test.py:375: in inner_fn return await async_fn(*args, **kwargs) distributed/utils_test.py:1099: in async_fn_outer return await utils_wait_for(async_fn(), timeout=timeout * 2) distributed/utils.py:1960: in wait_for return await fut distributed/utils_test.py:1000: in async_fn _client_factory(s) as c, /usr/lib/python3.12/contextlib.py:210: in __aenter__ return await anext(self.gen) distributed/utils_test.py:949: in _client_factory async with Client( distributed/client.py:1481: in __aenter__ await self distributed/client.py:1299: in _start await self._ensure_connected(timeout=timeout) distributed/client.py:1385: in _ensure_connected msg = await wait_for(comm.read(), timeout) distributed/utils.py:1959: in wait_for async with asyncio.timeout(timeout): _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = exc_type = exc_val = CancelledError(), exc_tb = async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType], ) -> Optional[bool]: assert self._state in (_State.ENTERED, _State.EXPIRING) if self._timeout_handler is not None: self._timeout_handler.cancel() self._timeout_handler = None if self._state is _State.EXPIRING: self._state = _State.EXPIRED if self._task.uncancel() <= self._cancelling and exc_type is exceptions.CancelledError: # Since there are no new cancel requests, we're # handling this. > raise TimeoutError from exc_val E TimeoutError /usr/lib/python3.12/asyncio/timeouts.py:115: TimeoutError ----------------------------- Captured stderr call ----------------------------- 2024-10-22 07:01:03,663 - distributed.scheduler - INFO - State start 2024-10-22 07:01:03,799 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:39985 2024-10-22 07:01:03,800 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:41567/status 2024-10-22 07:01:03,800 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 07:01:04,233 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:37699 2024-10-22 07:01:04,234 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:37699 2024-10-22 07:01:04,234 - distributed.worker - INFO - Worker name: 0 2024-10-22 07:01:04,234 - distributed.worker - INFO - dashboard at: 127.0.0.1:35619 2024-10-22 07:01:04,234 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:39985 2024-10-22 07:01:04,234 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:04,234 - distributed.worker - INFO - Threads: 1 2024-10-22 07:01:04,234 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 07:01:04,234 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-11c43ll0 2024-10-22 07:01:04,235 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:04,236 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:46199 2024-10-22 07:01:04,236 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:46199 2024-10-22 07:01:04,236 - distributed.worker - INFO - Worker name: 1 2024-10-22 07:01:04,236 - distributed.worker - INFO - dashboard at: 127.0.0.1:38787 2024-10-22 07:01:04,236 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:39985 2024-10-22 07:01:04,236 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:04,236 - distributed.worker - INFO - Threads: 2 2024-10-22 07:01:04,236 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 07:01:04,236 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-xlzuz_me 2024-10-22 07:01:04,485 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:04,550 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:39985 2024-10-22 07:01:04,550 - distributed.comm.tcp - INFO - Connection from tcp://127.0.0.1:38284 closed before handshake completed 2024-10-22 07:01:04,551 - distributed.scheduler - INFO - Register worker 2024-10-22 07:01:04,567 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:46199 2024-10-22 07:01:04,567 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38298 2024-10-22 07:01:04,568 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 07:01:04,568 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:39985 2024-10-22 07:01:04,568 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:04,569 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:39985 2024-10-22 07:01:04,723 - distributed.scheduler - INFO - Register worker 2024-10-22 07:01:04,958 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:37699 2024-10-22 07:01:05,157 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38312 2024-10-22 07:01:05,160 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 07:01:05,170 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:39985 2024-10-22 07:01:05,170 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:01:05,171 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:39985 2024-10-22 07:01:05,244 - distributed.scheduler - INFO - Receive client connection: Client-0f2242d6-8fce-11ef-ad33-7d0752ea3fc4 2024-10-22 07:01:05,438 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38324 2024-10-22 07:01:05,560 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:37699. Reason: worker-close 2024-10-22 07:01:05,560 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:46199. Reason: worker-close 2024-10-22 07:01:05,569 - distributed.core - INFO - Connection to tcp://127.0.0.1:38324 has been closed. 2024-10-22 07:01:05,721 - distributed.scheduler - INFO - Remove client Client-0f2242d6-8fce-11ef-ad33-7d0752ea3fc4 2024-10-22 07:01:05,736 - distributed.scheduler - INFO - Close client connection: Client-0f2242d6-8fce-11ef-ad33-7d0752ea3fc4 2024-10-22 07:01:05,819 - distributed.worker - ERROR - Failed to communicate with scheduler during heartbeat. Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1620, in connect return await self._connect(addr=addr, timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1564, in _connect comm = await connect( ^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/comm/core.py", line 342, in connect comm = await wait_for( ^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils.py", line 1960, in wait_for return await fut ^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/comm/tcp.py", line 546, in connect stream = await self.client.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/tornado/tcpclient.py", line 279, in connect af, addr, stream = await connector.start(connect_timeout=timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ asyncio.exceptions.CancelledError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1611, in connect async with asyncio.timeout(math.inf) as scope: ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/timeouts.py", line 115, in __aexit__ raise TimeoutError from exc_val TimeoutError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/worker.py", line 1252, in heartbeat response = await retry_operation( ^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 452, in retry_operation return await retry( ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/utils_comm.py", line 431, in retry return await coro() ^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1392, in send_recv_from_rpc comm = await self.pool.connect(self.addr) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/core.py", line 1632, in connect raise CommClosedError(reason) distributed.comm.core.CommClosedError: ConnectionPool closing. 2024-10-22 07:01:05,850 - distributed.comm.tcp - INFO - Connection from tcp://127.0.0.1:38344 closed before handshake completed 2024-10-22 07:01:05,851 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:38312; closing. 2024-10-22 07:01:05,851 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729530065.8513575') 2024-10-22 07:01:05,851 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:38298; closing. 2024-10-22 07:01:05,851 - distributed.core - INFO - Connection to tcp://127.0.0.1:39985 has been closed. 2024-10-22 07:01:05,866 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729530065.866048') 2024-10-22 07:01:05,866 - distributed.scheduler - INFO - Lost all workers 2024-10-22 07:01:05,866 - distributed.core - INFO - Connection to tcp://127.0.0.1:39985 has been closed. 2024-10-22 07:01:05,885 - distributed.batched - INFO - Batched Comm Closed Traceback (most recent call last): File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/batched.py", line 115, in _background_send nbytes = yield coro ^^^^^^^^^^ File "/usr/lib/python3/dist-packages/tornado/gen.py", line 766, in run value = future.result() ^^^^^^^^^^^^^^^ File "/build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build/distributed/comm/tcp.py", line 262, in write raise CommClosedError() distributed.comm.core.CommClosedError 2024-10-22 07:01:05,910 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 07:01:05,910 - distributed.scheduler - INFO - Scheduler closing all comms ___________________ test_last_in_first_out[queue on worker] ____________________ c = s = a = pause = False @gen_blockable_cluster async def test_last_in_first_out(c, s, a, pause): async with block_worker(c, s, a, pause, 15, 5): xs = [c.submit(slowinc, i, delay=0.05, key=f"x{i}") for i in range(5)] ys = [c.submit(slowinc, xs[i], delay=0.05, key=f"y{i}") for i in range(5)] zs = [c.submit(slowinc, ys[i], delay=0.05, key=f"z{i}") for i in range(5)] while not any(s.tasks[z.key].state == "memory" for z in zs): await asyncio.sleep(0.01) > assert not all(s.tasks[x.key].state == "memory" for x in xs) E assert not True E + where True = all(. at 0x7f22015492f0>) distributed/tests/test_priorities.py:234: AssertionError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_last_in_first_out.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 07:26:16,514 - distributed.scheduler - INFO - State start 2024-10-22 07:26:16,598 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:38475 2024-10-22 07:26:16,920 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:42317/status 2024-10-22 07:26:16,920 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 07:26:17,011 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:45933 2024-10-22 07:26:17,011 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:45933 2024-10-22 07:26:17,011 - distributed.worker - INFO - Worker name: 0 2024-10-22 07:26:17,012 - distributed.worker - INFO - dashboard at: 127.0.0.1:40203 2024-10-22 07:26:17,012 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:38475 2024-10-22 07:26:17,012 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:26:17,012 - distributed.worker - INFO - Threads: 1 2024-10-22 07:26:17,012 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 07:26:17,012 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-bcnblj4a 2024-10-22 07:26:17,012 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:26:17,123 - distributed.scheduler - INFO - Register worker 2024-10-22 07:26:17,195 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:45933 2024-10-22 07:26:17,389 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:48726 2024-10-22 07:26:17,390 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 07:26:17,391 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:38475 2024-10-22 07:26:17,391 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:26:17,401 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38475 2024-10-22 07:26:17,490 - distributed.scheduler - INFO - Receive client connection: Client-947e5168-8fd1-11ef-ad33-7d0752ea3fc4 2024-10-22 07:26:17,590 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:48732 2024-10-22 07:26:20,371 - distributed.scheduler - INFO - Remove client Client-947e5168-8fd1-11ef-ad33-7d0752ea3fc4 2024-10-22 07:26:20,401 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:48732; closing. 2024-10-22 07:26:20,402 - distributed.scheduler - INFO - Remove client Client-947e5168-8fd1-11ef-ad33-7d0752ea3fc4 2024-10-22 07:26:20,419 - distributed.scheduler - INFO - Close client connection: Client-947e5168-8fd1-11ef-ad33-7d0752ea3fc4 2024-10-22 07:26:20,446 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:45933. Reason: worker-close 2024-10-22 07:26:20,463 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 07:26:20,483 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:48726; closing. 2024-10-22 07:26:20,491 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729531580.4909594') 2024-10-22 07:26:20,491 - distributed.scheduler - INFO - Lost all workers 2024-10-22 07:26:20,491 - distributed.core - INFO - Connection to tcp://127.0.0.1:38475 has been closed. 2024-10-22 07:26:20,506 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 07:26:20,506 - distributed.scheduler - INFO - Scheduler closing all comms ============================= slowest 20 durations ============================= 189.59s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_method 86.42s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] 79.05s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] 30.15s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x4-chunks4] 27.10s call distributed/tests/test_nanny.py::test_environ_plugin 24.21s call distributed/tests/test_nanny.py::test_num_fds 22.93s call distributed/tests/test_scheduler.py::test_include_communication_in_occupancy 22.32s call distributed/tests/test_nanny.py::test_nanny_plugin_register_during_start_success[True] 21.95s call distributed/tests/test_failed_workers.py::test_restart_sync 21.28s call distributed/diagnostics/tests/test_progress.py::test_group_timing 20.19s call distributed/protocol/tests/test_serialize.py::test_large_pickled_object 19.08s call distributed/tests/test_failed_workers.py::test_restart_during_computation 18.74s call distributed/tests/test_failed_workers.py::test_restart_scheduler 18.65s call distributed/diagnostics/tests/test_progress.py::test_AllProgress 17.45s call distributed/tests/test_nanny.py::test_nanny_plugin_simple 17.37s call distributed/tests/test_client.py::test_repr[func2] 16.15s setup distributed/tests/test_publish.py::test_datasets_getitem_default 15.82s setup distributed/tests/test_publish.py::test_unpublish_multiple_datasets_sync 15.72s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_fully_unknown_dimension[x3-chunks3] 15.55s call distributed/tests/test_failed_workers.py::test_multiple_clients_restart =========================== short test summary info ============================ SKIPPED [1] distributed/cli/tests/test_dask_ssh.py:9: could not import 'paramiko': No module named 'paramiko' SKIPPED [1] distributed/comm/tests/test_ucx.py:15: could not import 'ucp': No module named 'ucp' SKIPPED [1] distributed/comm/tests/test_ucx_config.py:22: could not import 'ucp': No module named 'ucp' SKIPPED [1] distributed/dashboard/tests/test_components.py:5: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/dashboard/tests/test_scheduler_bokeh.py:11: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/dashboard/tests/test_worker_bokeh.py:10: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/deploy/tests/test_old_ssh.py:7: could not import 'paramiko': No module named 'paramiko' SKIPPED [1] distributed/deploy/tests/test_ssh.py:5: could not import 'asyncssh': No module named 'asyncssh' SKIPPED [1] distributed/diagnostics/tests/test_cudf_diagnostics.py:20: could not import 'cudf': No module named 'cudf' SKIPPED [1] distributed/diagnostics/tests/test_memray.py:5: could not import 'memray': No module named 'memray' SKIPPED [1] distributed/diagnostics/tests/test_nvml.py:10: could not import 'pynvml': No module named 'pynvml' SKIPPED [1] distributed/diagnostics/tests/test_progress_stream.py:5: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/diagnostics/tests/test_progress_widgets.py:14: could not import 'ipywidgets': No module named 'ipywidgets' SKIPPED [1] distributed/diagnostics/tests/test_rmm_diagnostics.py:14: could not import 'dask_cuda': No module named 'dask_cuda' SKIPPED [1] distributed/protocol/tests/test_arrow.py:5: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/protocol/tests/test_cupy.py:11: could not import 'cupy': No module named 'cupy' SKIPPED [1] distributed/protocol/tests/test_h5py.py:8: could not import 'h5py': No module named 'h5py' SKIPPED [1] distributed/protocol/tests/test_keras.py:5: could not import 'keras': No module named 'keras' SKIPPED [1] distributed/protocol/tests/test_netcdf4.py:5: could not import 'netCDF4': No module named 'netCDF4' SKIPPED [1] distributed/protocol/tests/test_numba.py:11: could not import 'numba.cuda': No module named 'numba' SKIPPED [1] distributed/protocol/tests/test_rmm.py:10: could not import 'numba.cuda': No module named 'numba' SKIPPED [1] distributed/protocol/tests/test_scipy.py:8: could not import 'scipy': No module named 'scipy' SKIPPED [1] distributed/protocol/tests/test_sparse.py:6: could not import 'sparse': No module named 'sparse' SKIPPED [1] distributed/protocol/tests/test_torch.py:8: could not import 'torch': No module named 'torch' SKIPPED [1] distributed/shuffle/tests/test_graph.py:9: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/shuffle/tests/test_merge.py:71: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/shuffle/tests/test_merge_column_and_index.py:110: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/shuffle/tests/test_metrics.py:71: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/shuffle/tests/test_shuffle.py:195: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/tests/test_parse_stdout.py:14: could not import 'parse_stdout': No module named 'parse_stdout' SKIPPED [1] distributed/cli/tests/test_dask_scheduler.py:497: need --runslow option to run SKIPPED [1] distributed/cli/tests/test_dask_scheduler.py:509: need --runslow option to run SKIPPED [2] distributed/cli/tests/test_dask_scheduler.py:569: need --runslow option to run SKIPPED [2] distributed/cli/tests/test_dask_worker.py:525: need --runslow option to run SKIPPED [1] distributed/deploy/tests/test_subprocess.py:77: need --runslow option to run SKIPPED [1] distributed/deploy/tests/test_subprocess.py:91: Windows-specific error testing (distributed#7434) SKIPPED [1] distributed/diagnostics/tests/test_eventstream.py:16: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/diagnostics/tests/test_install_plugin.py:143: need --runslow option to run SKIPPED [1] distributed/diagnostics/tests/test_memory_sampler.py:49: need --runslow option to run SKIPPED [2] distributed/diagnostics/tests/test_memory_sampler.py:79: could not import 'matplotlib': No module named 'matplotlib' SKIPPED [2] distributed/diagnostics/tests/test_memory_sampler.py:108: need --runslow option to run SKIPPED [1] distributed/diagnostics/tests/test_task_stream.py:140: could not import 'bokeh.models': No module named 'bokeh' SKIPPED [1] distributed/diagnostics/tests/test_task_stream.py:151: could not import 'bokeh.models': No module named 'bokeh' SKIPPED [1] distributed/http/scheduler/tests/test_missing_bokeh.py:35: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/http/scheduler/tests/test_missing_bokeh.py:59: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:86: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:324: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:437: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:478: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:498: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:521: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:543: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/scheduler/tests/test_scheduler_http.py:564: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/worker/tests/test_worker_http.py:149: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/worker/tests/test_worker_http.py:162: could not import 'aiohttp': No module named 'aiohttp' SKIPPED [1] distributed/http/worker/tests/test_worker_http.py:233: Fails on 32-bit, seems to be large memory request SKIPPED [4] distributed/protocol/tests/test_collection_cuda.py:12: could not import 'cupy': No module named 'cupy' SKIPPED [4] distributed/protocol/tests/test_collection_cuda.py:44: could not import 'cudf': No module named 'cudf' SKIPPED [1] distributed/protocol/tests/test_compression.py:125: could not import 'lz4': No module named 'lz4' SKIPPED [2] distributed/protocol/tests/test_compression.py:132: could not import 'lz4': No module named 'lz4' SKIPPED [2] distributed/protocol/tests/test_compression.py:132: could not import 'snappy': No module named 'snappy' SKIPPED [2] distributed/protocol/tests/test_compression.py:132: could not import 'zstandard': No module named 'zstandard' SKIPPED [1] distributed/protocol/tests/test_compression.py:143: could not import 'lz4': No module named 'lz4' SKIPPED [1] distributed/protocol/tests/test_compression.py:143: could not import 'snappy': No module named 'snappy' SKIPPED [1] distributed/protocol/tests/test_compression.py:143: could not import 'zstandard': No module named 'zstandard' SKIPPED [1] distributed/protocol/tests/test_compression.py:151: could not import 'lz4': No module named 'lz4' SKIPPED [1] distributed/protocol/tests/test_compression.py:151: could not import 'snappy': No module named 'snappy' SKIPPED [1] distributed/protocol/tests/test_compression.py:151: could not import 'zstandard': No module named 'zstandard' SKIPPED [10] distributed/protocol/tests/test_compression.py:168: need --runslow option to run SKIPPED [5] distributed/protocol/tests/test_compression.py:188: need --runslow option to run SKIPPED [1] distributed/protocol/tests/test_numpy.py:150: could not import 'numpy.core.test_rational': No module named 'numpy.core.test_rational' SKIPPED [1] distributed/protocol/tests/test_numpy.py:180: need --runslow option to run SKIPPED [1] distributed/protocol/tests/test_pickle.py:283: need --runslow option to run SKIPPED [3] distributed/protocol/tests/test_protocol.py:213: need --runslow option to run SKIPPED [1] distributed/shuffle/tests/test_rechunk.py:1252: need --runslow option to run SKIPPED [1] distributed/tests/test_active_memory_manager.py:1296: need --runslow option to run SKIPPED [1] distributed/tests/test_active_memory_manager.py:1309: need --runslow option to run SKIPPED [1] distributed/tests/test_active_memory_manager.py:1331: need --runslow option to run SKIPPED [2] distributed/tests/test_active_memory_manager.py:1352: need --runslow option to run SKIPPED [1] distributed/tests/test_batched.py:158: need --runslow option to run SKIPPED [1] distributed/tests/test_batched.py:228: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:853: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:1726: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:1860: unconditional skip SKIPPED [1] distributed/tests/test_client.py:2078: unconditional skip SKIPPED [1] distributed/tests/test_client.py:2653: unconditional skip SKIPPED [1] distributed/tests/test_client.py:2672: Use fast random selection now SKIPPED [1] distributed/tests/test_client.py:3303: unconditional skip SKIPPED [1] distributed/tests/test_client.py:3586: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:3651: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:3763: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:4586: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:4689: Now prefer first-in-first-out SKIPPED [1] distributed/tests/test_client.py:4719: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:4855: could not import 'scipy': No module named 'scipy' SKIPPED [1] distributed/tests/test_client.py:4998: need --runslow option to run SKIPPED [2] distributed/tests/test_client.py:5041: need --runslow option to run SKIPPED [2] distributed/tests/test_client.py:5061: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:5088: need --runslow option to run SKIPPED [2] distributed/tests/test_client.py:5101: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:5289: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:5308: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:5603: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:5845: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:6046: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:6163: unconditional skip SKIPPED [1] distributed/tests/test_client.py:6377: could not import 'bokeh.plotting': No module named 'bokeh' SKIPPED [1] distributed/tests/test_client.py:6726: known intermittent failure SKIPPED [2] distributed/tests/test_client.py:6807: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:6870: On Py3.10+ semaphore._loop is not bound until .acquire() blocks SKIPPED [1] distributed/tests/test_client.py:6890: On Py3.10+ semaphore._loop is not bound until .acquire() blocks SKIPPED [1] distributed/tests/test_client.py:6899: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:7305: could not import 'bokeh': No module named 'bokeh' SKIPPED [2] distributed/tests/test_client.py:7406: need --runslow option to run SKIPPED [1] distributed/tests/test_client.py:7501: numpy >=1.25 can capture ufunc code SKIPPED [1] distributed/tests/test_client.py:7645: need --runslow option to run SKIPPED [2] distributed/tests/test_client.py:8396: need --runslow option to run SKIPPED [1] distributed/tests/test_client_executor.py:146: need --runslow option to run SKIPPED [1] distributed/tests/test_config.py:356: could not import 'uvloop': No module named 'uvloop' SKIPPED [1] distributed/tests/test_core.py:602: need --runslow option to run SKIPPED [1] distributed/tests/test_core.py:1026: could not import 'crick': No module named 'crick' SKIPPED [1] distributed/tests/test_core.py:1035: could not import 'crick': No module named 'crick' SKIPPED [1] distributed/tests/test_core.py:1487: need --runslow option to run SKIPPED [1] distributed/tests/test_counter.py:13: no crick library SKIPPED [1] distributed/tests/test_dask_collections.py:221: could not import 'sparse': No module named 'sparse' SKIPPED [1] distributed/tests/test_diskutils.py:224: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:41: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:117: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:128: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:286: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:357: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:441: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:453: need --runslow option to run SKIPPED [2] distributed/tests/test_failed_workers.py:474: need --runslow option to run SKIPPED [1] distributed/tests/test_failed_workers.py:520: need --runslow option to run SKIPPED [1] distributed/tests/test_jupyter.py:47: need --runslow option to run SKIPPED [1] distributed/tests/test_metrics.py:31: WindowsTime doesn't work with high accuracy base timer SKIPPED [1] distributed/tests/test_nanny.py:93: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:107: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:141: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:269: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:468: need --runslow option to run SKIPPED [2] distributed/tests/test_nanny.py:545: could not import 'ucp': No module named 'ucp' SKIPPED [1] distributed/tests/test_nanny.py:632: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:640: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:681: need --runslow option to run SKIPPED [1] distributed/tests/test_nanny.py:724: need --runslow option to run SKIPPED [1] distributed/tests/test_profile.py:75: could not import 'stacktrace': No module named 'stacktrace' SKIPPED [1] distributed/tests/test_queues.py:113: need --runslow option to run SKIPPED [1] distributed/tests/test_resources.py:363: Skipped SKIPPED [1] distributed/tests/test_resources.py:420: Should protect resource keys from optimization SKIPPED [1] distributed/tests/test_resources.py:441: atop fusion seemed to break this SKIPPED [1] distributed/tests/test_scheduler.py:305: Not relevant with queuing on; see https://github.com/dask/distributed/issues/7204 SKIPPED [1] distributed/tests/test_scheduler.py:440: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1087: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1115: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1136: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1228: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1246: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1638: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1691: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1708: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1871: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:1911: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2102: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2361: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2450: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2471: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2506: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2549: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:2604: could not import 'bokeh': No module named 'bokeh' SKIPPED [1] distributed/tests/test_scheduler.py:2980: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3394: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3573: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3605: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3748: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3805: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:3825: need --runslow option to run SKIPPED [1] distributed/tests/test_scheduler.py:4064: need --runslow option to run SKIPPED [1] distributed/tests/test_steal.py:106: need --runslow option to run SKIPPED [1] distributed/tests/test_steal.py:285: Skipped SKIPPED [1] distributed/tests/test_steal.py:1287: executing heartbeats not considered yet SKIPPED [1] distributed/tests/test_steal.py:1353: need --runslow option to run SKIPPED [2] distributed/tests/test_stress.py:51: need --runslow option to run SKIPPED [1] distributed/tests/test_stress.py:96: need --runslow option to run SKIPPED [1] distributed/tests/test_stress.py:202: unconditional skip SKIPPED [1] distributed/tests/test_stress.py:233: need --runslow option to run SKIPPED [1] distributed/tests/test_stress.py:260: need --runslow option to run SKIPPED [1] distributed/tests/test_stress.py:301: need --runslow option to run SKIPPED [1] distributed/tests/test_system_monitor.py:111: could not import 'gilknocker': No module named 'gilknocker' SKIPPED [1] distributed/tests/test_utils.py:167: could not import 'IPython': No module named 'IPython' SKIPPED [1] distributed/tests/test_utils.py:357: could not import 'pyarrow': No module named 'pyarrow' SKIPPED [1] distributed/tests/test_utils_perf.py:86: need --runslow option to run SKIPPED [1] distributed/tests/test_utils_test.py:149: This hangs on travis SKIPPED [1] distributed/tests/test_utils_test.py:442: need --runslow option to run SKIPPED [1] distributed/tests/test_utils_test.py:603: need --runslow option to run SKIPPED [1] distributed/tests/test_utils_test.py:816: need --runslow option to run SKIPPED [1] distributed/tests/test_variable.py:194: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:219: don't yet support uploading pyc files SKIPPED [1] distributed/tests/test_worker.py:309: could not import 'crick': No module named 'crick' SKIPPED [1] distributed/tests/test_worker.py:345: need --runslow option to run SKIPPED [8] distributed/tests/test_worker.py:420: need --runslow option to run SKIPPED [5] distributed/tests/test_worker.py:561: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1207: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1287: need --runslow option to run SKIPPED [2] distributed/tests/test_worker.py:1476: could not import 'ucp': No module named 'ucp' SKIPPED [1] distributed/tests/test_worker.py:1540: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1572: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1603: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1630: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1658: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1725: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:1830: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:2729: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:3421: need --runslow option to run SKIPPED [1] distributed/tests/test_worker.py:3451: need --runslow option to run SKIPPED [1] distributed/tests/test_worker_memory.py:871: need --runslow option to run SKIPPED [2] distributed/tests/test_worker_memory.py:883: need --runslow option to run SKIPPED [1] distributed/tests/test_worker_memory.py:997: need --runslow option to run FAILED distributed/shuffle/tests/test_rechunk.py::test_rechunk_method - TimeoutError FAILED distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] - TimeoutError FAILED distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] - TimeoutError FAILED distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] - TimeoutError FAILED distributed/tests/test_client.py::test_run_rpc_error - TimeoutError FAILED distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] - assert not True + where True = all(. at 0x7f22015492f0>) = 6 failed, 2859 passed, 270 skipped, 221 deselected, 15 xfailed, 8 xpassed, 1 rerun in 4756.13s (1:19:16) = *** END OF RUN 1: NOT ALL TESTS HAVE YET PASSED/XFAILED *** *** STARTING RUN 2: python3.12 -m pytest --pyargs distributed --verbose --color=no --timeout-method=signal --timeout=300 -m not avoid_ci -rfE --last-failed --last-failed-no-failures none --ignore=distributed/comm/tests/test_comms.py --ignore=distributed/comm/tests/test_ws.py --ignore=distributed/deploy/tests/test_adaptive.py --ignore=distributed/deploy/tests/test_local.py --ignore=distributed/deploy/tests/test_slow_adaptive.py --ignore=distributed/deploy/tests/test_spec_cluster.py --deselect=distributed/cli/tests/test_dask_scheduler.py::test_no_dashboard --deselect=distributed/deploy/tests/test_local.py::test_localcluster_get_client --deselect=distributed/deploy/tests/test_old_ssh.py::test_cluster --deselect=distributed/deploy/tests/test_old_ssh.py::test_old_ssh_nprocs_renamed_to_n_workers --deselect=distributed/deploy/tests/test_old_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/deploy/tests/test_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/http/tests/test_core.py::test_prometheus_api_doc --deselect=distributed/tests/test_init.py::test_git_revision --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout_returned --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_jupyter.py::test_shutsdown_cleanly --deselect=distributed/tests/test_profile.py::test_stack_overflow --deselect=distributed/tests/test_pubsub.py::test_client_worker --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_spill.py::test_spillbuffer_oserror --deselect=distributed/tests/test_steal.py::test_steal_twice --deselect=distributed/tests/test_utils_test.py::test_cluster --deselect=distributed/tests/test_variable.py::test_variable_in_task --deselect=distributed/tests/test_worker.py::test_process_executor_kills_process --deselect=distributed/tests/test_worker_memory.py::test_fail_to_pickle_execute_1 --deselect=distributed/tests/test_worker_state_machine.py::test_task_state_instance_are_garbage_collected --deselect=distributed/protocol/tests/test_protocol.py::test_deeply_nested_structures --deselect=distributed/protocol/tests/test_serialize.py::test_deeply_nested_structures --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_spec.py::test_errors --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/dashboard/tests/test_scheduler_bokeh.py::test_counters --deselect=distributed/dashboard/tests/test_worker_bokeh.py::test_counters --deselect=distributed/deploy/tests/test_local.py::test_adapt_then_manual --deselect=distributed/deploy/tests/test_local.py::test_async_with --deselect=distributed/deploy/tests/test_local.py::test_close_twice --deselect=distributed/deploy/tests/test_local.py::test_cluster_info_sync --deselect=distributed/deploy/tests/test_local.py::test_local_tls --deselect=distributed/deploy/tests/test_local.py::test_no_dangling_asyncio_tasks --deselect=distributed/deploy/tests/test_local.py::test_only_local_access --deselect=distributed/deploy/tests/test_local.py::test_remote_access --deselect=distributed/diagnostics/tests/test_progress_widgets.py::test_serializers --deselect=distributed/diagnostics/tests/test_scheduler_plugin.py::test_lifecycle --deselect=distributed/http/scheduler/tests/test_missing_bokeh.py::test_missing_bokeh --deselect=distributed/http/scheduler/tests/test_scheduler_http.py::test_metrics_when_prometheus_client_not_installed --deselect=distributed/protocol/tests/test_serialize.py::test_errors --deselect=distributed/tests/test_batched.py::test_BatchedSend --deselect=distributed/tests/test_batched.py::test_close_closed --deselect=distributed/tests/test_batched.py::test_close_twice --deselect=distributed/tests/test_batched.py::test_send_after_stream_start --deselect=distributed/tests/test_batched.py::test_send_before_close --deselect=distributed/tests/test_batched.py::test_send_before_start --deselect=distributed/tests/test_batched.py::test_sending_traffic_jam --deselect=distributed/tests/test_batched.py::test_serializers --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_async_with --deselect=distributed/tests/test_client.py::test_client_is_quiet_cluster_close --deselect=distributed/tests/test_client.py::test_dashboard_link_cluster --deselect=distributed/tests/test_client.py::test_dashboard_link_inproc --deselect=distributed/tests/test_client.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_client.py::test_mixing_clients_different_scheduler --deselect=distributed/tests/test_client.py::test_quiet_client_close --deselect=distributed/tests/test_client.py::test_rebalance_sync --deselect=distributed/tests/test_client.py::test_repr_localcluster --deselect=distributed/tests/test_client.py::test_security_loader --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_shutdown --deselect=distributed/tests/test_client.py::test_shutdown_is_quiet_with_cluster --deselect=distributed/tests/test_client.py::test_shutdown_localcluster --deselect=distributed/tests/test_client.py::test_shutdown_stops_callbacks --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_start_new_loop --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_use_running_loop --deselect=distributed/tests/test_core.py::test_close_fast_without_active_handlers --deselect=distributed/tests/test_core.py::test_close_grace_period_for_handlers --deselect=distributed/tests/test_core.py::test_close_properly --deselect=distributed/tests/test_core.py::test_compression --deselect=distributed/tests/test_core.py::test_connection_pool --deselect=distributed/tests/test_core.py::test_connection_pool_close_while_connecting --deselect=distributed/tests/test_core.py::test_connection_pool_detects_remote_close --deselect=distributed/tests/test_core.py::test_connection_pool_outside_cancellation --deselect=distributed/tests/test_core.py::test_connection_pool_remove --deselect=distributed/tests/test_core.py::test_connection_pool_respects_limit --deselect=distributed/tests/test_core.py::test_connection_pool_tls --deselect=distributed/tests/test_core.py::test_counters --deselect=distributed/tests/test_core.py::test_deserialize_error --deselect=distributed/tests/test_core.py::test_errors --deselect=distributed/tests/test_core.py::test_identity_inproc --deselect=distributed/tests/test_core.py::test_identity_tcp --deselect=distributed/tests/test_core.py::test_large_packets_inproc --deselect=distributed/tests/test_core.py::test_messages_are_ordered_bsend --deselect=distributed/tests/test_core.py::test_messages_are_ordered_raw --deselect=distributed/tests/test_core.py::test_ports --deselect=distributed/tests/test_core.py::test_rpc_default --deselect=distributed/tests/test_core.py::test_rpc_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_default --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_tcp --deselect=distributed/tests/test_core.py::test_rpc_serialization --deselect=distributed/tests/test_core.py::test_rpc_tcp --deselect=distributed/tests/test_core.py::test_rpc_tls --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_inproc --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_tcp --deselect=distributed/tests/test_core.py::test_send_recv_args --deselect=distributed/tests/test_core.py::test_send_recv_cancelled --deselect=distributed/tests/test_core.py::test_server --deselect=distributed/tests/test_core.py::test_server_comms_mark_active_handlers --deselect=distributed/tests/test_core.py::test_server_raises_on_blocked_handlers --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_locks.py::test_errors --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_worker_uses_same_host_as_nanny --deselect=distributed/tests/test_preload.py::test_failure_doesnt_crash_scheduler --deselect=distributed/tests/test_preload.py::test_preload_import_time --deselect=distributed/tests/test_preload.py::test_preload_manager_sequence --deselect=distributed/tests/test_preload.py::test_worker_preload_text --deselect=distributed/tests/test_scheduler.py::test_allowed_failures_config --deselect=distributed/tests/test_scheduler.py::test_async_context_manager --deselect=distributed/tests/test_scheduler.py::test_dashboard_host --deselect=distributed/tests/test_scheduler.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_scheduler.py::test_finished --deselect=distributed/tests/test_scheduler.py::test_multiple_listeners --deselect=distributed/tests/test_scheduler.py::test_no_dangling_asyncio_tasks --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_security.py::test_require_encryption --deselect=distributed/tests/test_security.py::test_tls_listen_connect --deselect=distributed/tests/test_security.py::test_tls_temporary_credentials_functional --deselect=distributed/tests/test_semaphore.py::test_threadpoolworkers_pick_correct_ioloop --deselect=distributed/tests/test_tls_functional.py::test_security_dict_input_no_security --deselect=distributed/tests/test_utils_test.py::test_ensure_no_new_clients --deselect=distributed/tests/test_utils_test.py::test_freeze_batched_send --deselect=distributed/tests/test_utils_test.py::test_locked_comm_drop_in_replacement --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_read --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_write --deselect=distributed/tests/test_worker.py::test_host_uses_scheduler_protocol --deselect=distributed/tests/test_worker.py::test_plugin_exception --deselect=distributed/tests/test_worker.py::test_plugin_internal_exception --deselect=distributed/tests/test_worker.py::test_plugin_multiple_exceptions --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker_client.py::test_dont_override_default_get --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_allowlist --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_protocols --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers_2 --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command_default --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_config --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_file --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_remote_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_scheduler_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_contact_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_error_during_startup --deselect=distributed/cli/tests/test_dask_worker.py::test_integer_names --deselect=distributed/cli/tests/test_dask_worker.py::test_listen_address_ipv6 --deselect=distributed/cli/tests/test_dask_worker.py::test_local_directory --deselect=distributed/cli/tests/test_dask_worker.py::test_memory_limit --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range_too_many_workers_raises --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_no_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_auto --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_expands_name --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_negative --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_requires_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_preload_config --deselect=distributed/cli/tests/test_dask_worker.py::test_resources --deselect=distributed/cli/tests/test_dask_worker.py::test_respect_host_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_address_env --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_restart_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_stagger_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_signal_handling --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_works --deselect=distributed/cli/tests/test_dask_worker.py::test_timeout --deselect=distributed/cli/tests/test_dask_worker.py::test_worker_class --deselect=distributed/tests/test_config.py::test_logging_extended --deselect=distributed/tests/test_config.py::test_logging_file_config --deselect=distributed/tests/test_config.py::test_logging_mutual_exclusive --deselect=distributed/tests/test_config.py::test_logging_simple --deselect=distributed/tests/test_config.py::test_logging_simple_under_distributed --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_variable.py::test_variable_in_task ============================= test session starts ============================== platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /usr/bin/python3.12 cachedir: .pytest_cache rootdir: /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build configfile: pyproject.toml plugins: rerunfailures-12.0, typeguard-4.3.0, timeout-2.3.1, anyio-4.4.0 timeout: 300.0s timeout method: signal timeout func_only: False collecting ... collected 10 items / 4 deselected / 6 selected run-last-failure: rerun previous 6 failures (skipped 143 files) distributed/shuffle/tests/test_rechunk.py::test_rechunk_method PASSED [ 16%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] PASSED [ 33%] distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] PASSED [ 50%] distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] PASSED [ 66%] distributed/tests/test_client.py::test_run_rpc_error PASSED [ 83%] distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] FAILED [100%] =================================== FAILURES =================================== ___________________ test_last_in_first_out[queue on worker] ____________________ c = s = a = pause = False @gen_blockable_cluster async def test_last_in_first_out(c, s, a, pause): async with block_worker(c, s, a, pause, 15, 5): xs = [c.submit(slowinc, i, delay=0.05, key=f"x{i}") for i in range(5)] ys = [c.submit(slowinc, xs[i], delay=0.05, key=f"y{i}") for i in range(5)] zs = [c.submit(slowinc, ys[i], delay=0.05, key=f"z{i}") for i in range(5)] while not any(s.tasks[z.key].state == "memory" for z in zs): await asyncio.sleep(0.01) > assert not all(s.tasks[x.key].state == "memory" for x in xs) E assert not True E + where True = all(. at 0x7f74942dfb90>) distributed/tests/test_priorities.py:234: AssertionError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_last_in_first_out.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 07:48:23,715 - distributed.scheduler - INFO - State start 2024-10-22 07:48:23,727 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:39429 2024-10-22 07:48:23,728 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:34091/status 2024-10-22 07:48:23,728 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 07:48:23,746 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:33631 2024-10-22 07:48:23,746 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:33631 2024-10-22 07:48:23,746 - distributed.worker - INFO - Worker name: 0 2024-10-22 07:48:23,746 - distributed.worker - INFO - dashboard at: 127.0.0.1:33037 2024-10-22 07:48:23,746 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:39429 2024-10-22 07:48:23,747 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:23,747 - distributed.worker - INFO - Threads: 1 2024-10-22 07:48:23,747 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 07:48:23,747 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-lkucf07d 2024-10-22 07:48:23,747 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:23,750 - distributed.scheduler - INFO - Register worker 2024-10-22 07:48:23,751 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:33631 2024-10-22 07:48:23,751 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:54120 2024-10-22 07:48:23,752 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 07:48:23,752 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:39429 2024-10-22 07:48:23,761 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:23,762 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:39429 2024-10-22 07:48:23,772 - distributed.scheduler - INFO - Receive client connection: Client-ab10e344-8fd4-11ef-8fe1-631a890fd210 2024-10-22 07:48:23,781 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:54136 2024-10-22 07:48:25,373 - distributed.scheduler - INFO - Remove client Client-ab10e344-8fd4-11ef-8fe1-631a890fd210 2024-10-22 07:48:25,374 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:54136; closing. 2024-10-22 07:48:25,374 - distributed.scheduler - INFO - Remove client Client-ab10e344-8fd4-11ef-8fe1-631a890fd210 2024-10-22 07:48:25,377 - distributed.scheduler - INFO - Close client connection: Client-ab10e344-8fd4-11ef-8fe1-631a890fd210 2024-10-22 07:48:25,379 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:33631. Reason: worker-close 2024-10-22 07:48:25,381 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 07:48:25,383 - distributed.core - INFO - Connection to tcp://127.0.0.1:39429 has been closed. 2024-10-22 07:48:25,391 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:54120; closing. 2024-10-22 07:48:25,392 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729532905.3921595') 2024-10-22 07:48:25,392 - distributed.scheduler - INFO - Lost all workers 2024-10-22 07:48:25,394 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 07:48:25,394 - distributed.scheduler - INFO - Scheduler closing all comms ============================= slowest 20 durations ============================= 18.24s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_method 15.90s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] 7.32s call distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] 1.70s call distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.99s call distributed/tests/test_client.py::test_run_rpc_error 0.90s call distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] 0.00s setup distributed/shuffle/tests/test_rechunk.py::test_rechunk_method 0.00s setup distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] 0.00s setup distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] 0.00s teardown distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s setup distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] 0.00s teardown distributed/tests/test_cancelled_state.py::test_deadlock_cancelled_after_inflight_before_gather_from_worker[True-cancelled] 0.00s teardown distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x5-chunks5] 0.00s teardown distributed/shuffle/tests/test_rechunk.py::test_rechunk_with_partially_unknown_dimension[x3-chunks3] 0.00s teardown distributed/shuffle/tests/test_rechunk.py::test_rechunk_method 0.00s setup distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s teardown distributed/tests/test_client.py::test_run_rpc_error 0.00s setup distributed/tests/test_client.py::test_run_rpc_error =========================== short test summary info ============================ FAILED distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] - assert not True + where True = all(. at 0x7f74942dfb90>) ================== 1 failed, 5 passed, 4 deselected in 46.76s ================== *** END OF RUN 2: NOT ALL TESTS HAVE YET PASSED/XFAILED *** *** STARTING RUN 3: python3.12 -m pytest --pyargs distributed --verbose --color=no --timeout-method=signal --timeout=300 -m not avoid_ci -rfE --last-failed --last-failed-no-failures none --ignore=distributed/comm/tests/test_comms.py --ignore=distributed/comm/tests/test_ws.py --ignore=distributed/deploy/tests/test_adaptive.py --ignore=distributed/deploy/tests/test_local.py --ignore=distributed/deploy/tests/test_slow_adaptive.py --ignore=distributed/deploy/tests/test_spec_cluster.py --deselect=distributed/cli/tests/test_dask_scheduler.py::test_no_dashboard --deselect=distributed/deploy/tests/test_local.py::test_localcluster_get_client --deselect=distributed/deploy/tests/test_old_ssh.py::test_cluster --deselect=distributed/deploy/tests/test_old_ssh.py::test_old_ssh_nprocs_renamed_to_n_workers --deselect=distributed/deploy/tests/test_old_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/deploy/tests/test_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/http/tests/test_core.py::test_prometheus_api_doc --deselect=distributed/tests/test_init.py::test_git_revision --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout_returned --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_jupyter.py::test_shutsdown_cleanly --deselect=distributed/tests/test_profile.py::test_stack_overflow --deselect=distributed/tests/test_pubsub.py::test_client_worker --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_spill.py::test_spillbuffer_oserror --deselect=distributed/tests/test_steal.py::test_steal_twice --deselect=distributed/tests/test_utils_test.py::test_cluster --deselect=distributed/tests/test_variable.py::test_variable_in_task --deselect=distributed/tests/test_worker.py::test_process_executor_kills_process --deselect=distributed/tests/test_worker_memory.py::test_fail_to_pickle_execute_1 --deselect=distributed/tests/test_worker_state_machine.py::test_task_state_instance_are_garbage_collected --deselect=distributed/protocol/tests/test_protocol.py::test_deeply_nested_structures --deselect=distributed/protocol/tests/test_serialize.py::test_deeply_nested_structures --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_spec.py::test_errors --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/dashboard/tests/test_scheduler_bokeh.py::test_counters --deselect=distributed/dashboard/tests/test_worker_bokeh.py::test_counters --deselect=distributed/deploy/tests/test_local.py::test_adapt_then_manual --deselect=distributed/deploy/tests/test_local.py::test_async_with --deselect=distributed/deploy/tests/test_local.py::test_close_twice --deselect=distributed/deploy/tests/test_local.py::test_cluster_info_sync --deselect=distributed/deploy/tests/test_local.py::test_local_tls --deselect=distributed/deploy/tests/test_local.py::test_no_dangling_asyncio_tasks --deselect=distributed/deploy/tests/test_local.py::test_only_local_access --deselect=distributed/deploy/tests/test_local.py::test_remote_access --deselect=distributed/diagnostics/tests/test_progress_widgets.py::test_serializers --deselect=distributed/diagnostics/tests/test_scheduler_plugin.py::test_lifecycle --deselect=distributed/http/scheduler/tests/test_missing_bokeh.py::test_missing_bokeh --deselect=distributed/http/scheduler/tests/test_scheduler_http.py::test_metrics_when_prometheus_client_not_installed --deselect=distributed/protocol/tests/test_serialize.py::test_errors --deselect=distributed/tests/test_batched.py::test_BatchedSend --deselect=distributed/tests/test_batched.py::test_close_closed --deselect=distributed/tests/test_batched.py::test_close_twice --deselect=distributed/tests/test_batched.py::test_send_after_stream_start --deselect=distributed/tests/test_batched.py::test_send_before_close --deselect=distributed/tests/test_batched.py::test_send_before_start --deselect=distributed/tests/test_batched.py::test_sending_traffic_jam --deselect=distributed/tests/test_batched.py::test_serializers --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_async_with --deselect=distributed/tests/test_client.py::test_client_is_quiet_cluster_close --deselect=distributed/tests/test_client.py::test_dashboard_link_cluster --deselect=distributed/tests/test_client.py::test_dashboard_link_inproc --deselect=distributed/tests/test_client.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_client.py::test_mixing_clients_different_scheduler --deselect=distributed/tests/test_client.py::test_quiet_client_close --deselect=distributed/tests/test_client.py::test_rebalance_sync --deselect=distributed/tests/test_client.py::test_repr_localcluster --deselect=distributed/tests/test_client.py::test_security_loader --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_shutdown --deselect=distributed/tests/test_client.py::test_shutdown_is_quiet_with_cluster --deselect=distributed/tests/test_client.py::test_shutdown_localcluster --deselect=distributed/tests/test_client.py::test_shutdown_stops_callbacks --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_start_new_loop --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_use_running_loop --deselect=distributed/tests/test_core.py::test_close_fast_without_active_handlers --deselect=distributed/tests/test_core.py::test_close_grace_period_for_handlers --deselect=distributed/tests/test_core.py::test_close_properly --deselect=distributed/tests/test_core.py::test_compression --deselect=distributed/tests/test_core.py::test_connection_pool --deselect=distributed/tests/test_core.py::test_connection_pool_close_while_connecting --deselect=distributed/tests/test_core.py::test_connection_pool_detects_remote_close --deselect=distributed/tests/test_core.py::test_connection_pool_outside_cancellation --deselect=distributed/tests/test_core.py::test_connection_pool_remove --deselect=distributed/tests/test_core.py::test_connection_pool_respects_limit --deselect=distributed/tests/test_core.py::test_connection_pool_tls --deselect=distributed/tests/test_core.py::test_counters --deselect=distributed/tests/test_core.py::test_deserialize_error --deselect=distributed/tests/test_core.py::test_errors --deselect=distributed/tests/test_core.py::test_identity_inproc --deselect=distributed/tests/test_core.py::test_identity_tcp --deselect=distributed/tests/test_core.py::test_large_packets_inproc --deselect=distributed/tests/test_core.py::test_messages_are_ordered_bsend --deselect=distributed/tests/test_core.py::test_messages_are_ordered_raw --deselect=distributed/tests/test_core.py::test_ports --deselect=distributed/tests/test_core.py::test_rpc_default --deselect=distributed/tests/test_core.py::test_rpc_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_default --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_tcp --deselect=distributed/tests/test_core.py::test_rpc_serialization --deselect=distributed/tests/test_core.py::test_rpc_tcp --deselect=distributed/tests/test_core.py::test_rpc_tls --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_inproc --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_tcp --deselect=distributed/tests/test_core.py::test_send_recv_args --deselect=distributed/tests/test_core.py::test_send_recv_cancelled --deselect=distributed/tests/test_core.py::test_server --deselect=distributed/tests/test_core.py::test_server_comms_mark_active_handlers --deselect=distributed/tests/test_core.py::test_server_raises_on_blocked_handlers --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_locks.py::test_errors --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_worker_uses_same_host_as_nanny --deselect=distributed/tests/test_preload.py::test_failure_doesnt_crash_scheduler --deselect=distributed/tests/test_preload.py::test_preload_import_time --deselect=distributed/tests/test_preload.py::test_preload_manager_sequence --deselect=distributed/tests/test_preload.py::test_worker_preload_text --deselect=distributed/tests/test_scheduler.py::test_allowed_failures_config --deselect=distributed/tests/test_scheduler.py::test_async_context_manager --deselect=distributed/tests/test_scheduler.py::test_dashboard_host --deselect=distributed/tests/test_scheduler.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_scheduler.py::test_finished --deselect=distributed/tests/test_scheduler.py::test_multiple_listeners --deselect=distributed/tests/test_scheduler.py::test_no_dangling_asyncio_tasks --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_security.py::test_require_encryption --deselect=distributed/tests/test_security.py::test_tls_listen_connect --deselect=distributed/tests/test_security.py::test_tls_temporary_credentials_functional --deselect=distributed/tests/test_semaphore.py::test_threadpoolworkers_pick_correct_ioloop --deselect=distributed/tests/test_tls_functional.py::test_security_dict_input_no_security --deselect=distributed/tests/test_utils_test.py::test_ensure_no_new_clients --deselect=distributed/tests/test_utils_test.py::test_freeze_batched_send --deselect=distributed/tests/test_utils_test.py::test_locked_comm_drop_in_replacement --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_read --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_write --deselect=distributed/tests/test_worker.py::test_host_uses_scheduler_protocol --deselect=distributed/tests/test_worker.py::test_plugin_exception --deselect=distributed/tests/test_worker.py::test_plugin_internal_exception --deselect=distributed/tests/test_worker.py::test_plugin_multiple_exceptions --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker_client.py::test_dont_override_default_get --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_allowlist --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_protocols --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers_2 --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command_default --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_config --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_file --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_remote_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_scheduler_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_contact_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_error_during_startup --deselect=distributed/cli/tests/test_dask_worker.py::test_integer_names --deselect=distributed/cli/tests/test_dask_worker.py::test_listen_address_ipv6 --deselect=distributed/cli/tests/test_dask_worker.py::test_local_directory --deselect=distributed/cli/tests/test_dask_worker.py::test_memory_limit --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range_too_many_workers_raises --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_no_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_auto --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_expands_name --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_negative --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_requires_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_preload_config --deselect=distributed/cli/tests/test_dask_worker.py::test_resources --deselect=distributed/cli/tests/test_dask_worker.py::test_respect_host_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_address_env --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_restart_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_stagger_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_signal_handling --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_works --deselect=distributed/cli/tests/test_dask_worker.py::test_timeout --deselect=distributed/cli/tests/test_dask_worker.py::test_worker_class --deselect=distributed/tests/test_config.py::test_logging_extended --deselect=distributed/tests/test_config.py::test_logging_file_config --deselect=distributed/tests/test_config.py::test_logging_mutual_exclusive --deselect=distributed/tests/test_config.py::test_logging_simple --deselect=distributed/tests/test_config.py::test_logging_simple_under_distributed --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_variable.py::test_variable_in_task ============================= test session starts ============================== platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /usr/bin/python3.12 cachedir: .pytest_cache rootdir: /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build configfile: pyproject.toml plugins: rerunfailures-12.0, typeguard-4.3.0, timeout-2.3.1, anyio-4.4.0 timeout: 300.0s timeout method: signal timeout func_only: False collecting ... collected 1 item run-last-failure: rerun previous 1 failure (skipped 146 files) distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] FAILED [100%] =================================== FAILURES =================================== ___________________ test_last_in_first_out[queue on worker] ____________________ c = s = a = pause = False @gen_blockable_cluster async def test_last_in_first_out(c, s, a, pause): async with block_worker(c, s, a, pause, 15, 5): xs = [c.submit(slowinc, i, delay=0.05, key=f"x{i}") for i in range(5)] ys = [c.submit(slowinc, xs[i], delay=0.05, key=f"y{i}") for i in range(5)] zs = [c.submit(slowinc, ys[i], delay=0.05, key=f"z{i}") for i in range(5)] while not any(s.tasks[z.key].state == "memory" for z in zs): await asyncio.sleep(0.01) > assert not all(s.tasks[x.key].state == "memory" for x in xs) E assert not True E + where True = all(. at 0x7f01f9cfcad0>) distributed/tests/test_priorities.py:234: AssertionError ----------------------------- Captured stdout call ----------------------------- Dumped cluster state to test_cluster_dump/test_last_in_first_out.yaml ----------------------------- Captured stderr call ----------------------------- 2024-10-22 07:48:28,657 - distributed.http.proxy - INFO - To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy 2024-10-22 07:48:28,663 - distributed.scheduler - INFO - State start 2024-10-22 07:48:28,670 - distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:38255 2024-10-22 07:48:28,671 - distributed.scheduler - INFO - dashboard at: http://127.0.0.1:34479/status 2024-10-22 07:48:28,671 - distributed.scheduler - INFO - Registering Worker plugin shuffle 2024-10-22 07:48:28,679 - distributed.worker - INFO - Start worker at: tcp://127.0.0.1:46809 2024-10-22 07:48:28,680 - distributed.worker - INFO - Listening to: tcp://127.0.0.1:46809 2024-10-22 07:48:28,680 - distributed.worker - INFO - Worker name: 0 2024-10-22 07:48:28,680 - distributed.worker - INFO - dashboard at: 127.0.0.1:34407 2024-10-22 07:48:28,680 - distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:38255 2024-10-22 07:48:28,680 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:28,680 - distributed.worker - INFO - Threads: 1 2024-10-22 07:48:28,680 - distributed.worker - INFO - Memory: 78.53 GiB 2024-10-22 07:48:28,680 - distributed.worker - INFO - Local Directory: /tmp/dask-scratch-space/worker-xw7lruxg 2024-10-22 07:48:28,680 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:29,304 - distributed.scheduler - INFO - Register worker 2024-10-22 07:48:29,305 - distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:46809 2024-10-22 07:48:29,305 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:39666 2024-10-22 07:48:29,306 - distributed.worker - INFO - Starting Worker plugin shuffle 2024-10-22 07:48:29,306 - distributed.worker - INFO - Registered to: tcp://127.0.0.1:38255 2024-10-22 07:48:29,306 - distributed.worker - INFO - ------------------------------------------------- 2024-10-22 07:48:29,307 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:38255 2024-10-22 07:48:29,313 - distributed.scheduler - INFO - Receive client connection: Client-ae5f0c43-8fd4-11ef-8449-51ebeaa3d26a 2024-10-22 07:48:29,314 - distributed.core - INFO - Starting established connection to tcp://127.0.0.1:39670 2024-10-22 07:48:30,489 - distributed.scheduler - INFO - Remove client Client-ae5f0c43-8fd4-11ef-8449-51ebeaa3d26a 2024-10-22 07:48:30,490 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:39670; closing. 2024-10-22 07:48:30,491 - distributed.scheduler - INFO - Remove client Client-ae5f0c43-8fd4-11ef-8449-51ebeaa3d26a 2024-10-22 07:48:30,493 - distributed.scheduler - INFO - Close client connection: Client-ae5f0c43-8fd4-11ef-8449-51ebeaa3d26a 2024-10-22 07:48:30,496 - distributed.worker - INFO - Stopping worker at tcp://127.0.0.1:46809. Reason: worker-close 2024-10-22 07:48:30,498 - distributed.worker.state_machine - WARNING - Async instruction for > ended with CancelledError 2024-10-22 07:48:30,500 - distributed.core - INFO - Connection to tcp://127.0.0.1:38255 has been closed. 2024-10-22 07:48:30,523 - distributed.core - INFO - Received 'close-stream' from tcp://127.0.0.1:39666; closing. 2024-10-22 07:48:30,523 - distributed.scheduler - INFO - Remove worker (stimulus_id='handle-worker-cleanup-1729532910.5238948') 2024-10-22 07:48:30,524 - distributed.scheduler - INFO - Lost all workers 2024-10-22 07:48:30,538 - distributed.scheduler - INFO - Scheduler closing due to unknown reason... 2024-10-22 07:48:30,538 - distributed.scheduler - INFO - Scheduler closing all comms ============================= slowest 20 durations ============================= 1.95s call distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s setup distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s teardown distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] =========================== short test summary info ============================ FAILED distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] - assert not True + where True = all(. at 0x7f01f9cfcad0>) ============================== 1 failed in 2.54s =============================== *** END OF RUN 3: NOT ALL TESTS HAVE YET PASSED/XFAILED *** *** STARTING RUN 4: python3.12 -m pytest --pyargs distributed --verbose --color=no --timeout-method=signal --timeout=300 -m not avoid_ci -rfE --last-failed --last-failed-no-failures none --ignore=distributed/comm/tests/test_comms.py --ignore=distributed/comm/tests/test_ws.py --ignore=distributed/deploy/tests/test_adaptive.py --ignore=distributed/deploy/tests/test_local.py --ignore=distributed/deploy/tests/test_slow_adaptive.py --ignore=distributed/deploy/tests/test_spec_cluster.py --deselect=distributed/cli/tests/test_dask_scheduler.py::test_no_dashboard --deselect=distributed/deploy/tests/test_local.py::test_localcluster_get_client --deselect=distributed/deploy/tests/test_old_ssh.py::test_cluster --deselect=distributed/deploy/tests/test_old_ssh.py::test_old_ssh_nprocs_renamed_to_n_workers --deselect=distributed/deploy/tests/test_old_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/deploy/tests/test_ssh.py::test_nprocs_attribute_is_deprecated --deselect=distributed/http/tests/test_core.py::test_prometheus_api_doc --deselect=distributed/tests/test_init.py::test_git_revision --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout_returned --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_jupyter.py::test_shutsdown_cleanly --deselect=distributed/tests/test_profile.py::test_stack_overflow --deselect=distributed/tests/test_pubsub.py::test_client_worker --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_spill.py::test_spillbuffer_oserror --deselect=distributed/tests/test_steal.py::test_steal_twice --deselect=distributed/tests/test_utils_test.py::test_cluster --deselect=distributed/tests/test_variable.py::test_variable_in_task --deselect=distributed/tests/test_worker.py::test_process_executor_kills_process --deselect=distributed/tests/test_worker_memory.py::test_fail_to_pickle_execute_1 --deselect=distributed/tests/test_worker_state_machine.py::test_task_state_instance_are_garbage_collected --deselect=distributed/protocol/tests/test_protocol.py::test_deeply_nested_structures --deselect=distributed/protocol/tests/test_serialize.py::test_deeply_nested_structures --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_spec.py::test_errors --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/dashboard/tests/test_scheduler_bokeh.py::test_counters --deselect=distributed/dashboard/tests/test_worker_bokeh.py::test_counters --deselect=distributed/deploy/tests/test_local.py::test_adapt_then_manual --deselect=distributed/deploy/tests/test_local.py::test_async_with --deselect=distributed/deploy/tests/test_local.py::test_close_twice --deselect=distributed/deploy/tests/test_local.py::test_cluster_info_sync --deselect=distributed/deploy/tests/test_local.py::test_local_tls --deselect=distributed/deploy/tests/test_local.py::test_no_dangling_asyncio_tasks --deselect=distributed/deploy/tests/test_local.py::test_only_local_access --deselect=distributed/deploy/tests/test_local.py::test_remote_access --deselect=distributed/diagnostics/tests/test_progress_widgets.py::test_serializers --deselect=distributed/diagnostics/tests/test_scheduler_plugin.py::test_lifecycle --deselect=distributed/http/scheduler/tests/test_missing_bokeh.py::test_missing_bokeh --deselect=distributed/http/scheduler/tests/test_scheduler_http.py::test_metrics_when_prometheus_client_not_installed --deselect=distributed/protocol/tests/test_serialize.py::test_errors --deselect=distributed/tests/test_batched.py::test_BatchedSend --deselect=distributed/tests/test_batched.py::test_close_closed --deselect=distributed/tests/test_batched.py::test_close_twice --deselect=distributed/tests/test_batched.py::test_send_after_stream_start --deselect=distributed/tests/test_batched.py::test_send_before_close --deselect=distributed/tests/test_batched.py::test_send_before_start --deselect=distributed/tests/test_batched.py::test_sending_traffic_jam --deselect=distributed/tests/test_batched.py::test_serializers --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::TestClientSecurityLoader::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_async_with --deselect=distributed/tests/test_client.py::test_client_is_quiet_cluster_close --deselect=distributed/tests/test_client.py::test_dashboard_link_cluster --deselect=distributed/tests/test_client.py::test_dashboard_link_inproc --deselect=distributed/tests/test_client.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_client.py::test_mixing_clients_different_scheduler --deselect=distributed/tests/test_client.py::test_quiet_client_close --deselect=distributed/tests/test_client.py::test_rebalance_sync --deselect=distributed/tests/test_client.py::test_repr_localcluster --deselect=distributed/tests/test_client.py::test_security_loader --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_explicit_security_provided --deselect=distributed/tests/test_client.py::test_security_loader_ignored_if_returns_none --deselect=distributed/tests/test_client.py::test_shutdown --deselect=distributed/tests/test_client.py::test_shutdown_is_quiet_with_cluster --deselect=distributed/tests/test_client.py::test_shutdown_localcluster --deselect=distributed/tests/test_client.py::test_shutdown_stops_callbacks --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_start_new_loop --deselect=distributed/tests/test_client_loop.py::test_close_loop_sync_use_running_loop --deselect=distributed/tests/test_core.py::test_close_fast_without_active_handlers --deselect=distributed/tests/test_core.py::test_close_grace_period_for_handlers --deselect=distributed/tests/test_core.py::test_close_properly --deselect=distributed/tests/test_core.py::test_compression --deselect=distributed/tests/test_core.py::test_connection_pool --deselect=distributed/tests/test_core.py::test_connection_pool_close_while_connecting --deselect=distributed/tests/test_core.py::test_connection_pool_detects_remote_close --deselect=distributed/tests/test_core.py::test_connection_pool_outside_cancellation --deselect=distributed/tests/test_core.py::test_connection_pool_remove --deselect=distributed/tests/test_core.py::test_connection_pool_respects_limit --deselect=distributed/tests/test_core.py::test_connection_pool_tls --deselect=distributed/tests/test_core.py::test_counters --deselect=distributed/tests/test_core.py::test_deserialize_error --deselect=distributed/tests/test_core.py::test_errors --deselect=distributed/tests/test_core.py::test_identity_inproc --deselect=distributed/tests/test_core.py::test_identity_tcp --deselect=distributed/tests/test_core.py::test_large_packets_inproc --deselect=distributed/tests/test_core.py::test_messages_are_ordered_bsend --deselect=distributed/tests/test_core.py::test_messages_are_ordered_raw --deselect=distributed/tests/test_core.py::test_ports --deselect=distributed/tests/test_core.py::test_rpc_default --deselect=distributed/tests/test_core.py::test_rpc_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_default --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_inproc --deselect=distributed/tests/test_core.py::test_rpc_message_lifetime_tcp --deselect=distributed/tests/test_core.py::test_rpc_serialization --deselect=distributed/tests/test_core.py::test_rpc_tcp --deselect=distributed/tests/test_core.py::test_rpc_tls --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_inproc --deselect=distributed/tests/test_core.py::test_rpc_with_many_connections_tcp --deselect=distributed/tests/test_core.py::test_send_recv_args --deselect=distributed/tests/test_core.py::test_send_recv_cancelled --deselect=distributed/tests/test_core.py::test_server --deselect=distributed/tests/test_core.py::test_server_comms_mark_active_handlers --deselect=distributed/tests/test_core.py::test_server_raises_on_blocked_handlers --deselect=distributed/tests/test_jupyter.py::test_jupyter_idle_timeout --deselect=distributed/tests/test_jupyter.py::test_jupyter_server --deselect=distributed/tests/test_locks.py::test_errors --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_scheduler_file --deselect=distributed/tests/test_nanny.py::test_worker_uses_same_host_as_nanny --deselect=distributed/tests/test_preload.py::test_failure_doesnt_crash_scheduler --deselect=distributed/tests/test_preload.py::test_preload_import_time --deselect=distributed/tests/test_preload.py::test_preload_manager_sequence --deselect=distributed/tests/test_preload.py::test_worker_preload_text --deselect=distributed/tests/test_scheduler.py::test_allowed_failures_config --deselect=distributed/tests/test_scheduler.py::test_async_context_manager --deselect=distributed/tests/test_scheduler.py::test_dashboard_host --deselect=distributed/tests/test_scheduler.py::test_file_descriptors_dont_leak --deselect=distributed/tests/test_scheduler.py::test_finished --deselect=distributed/tests/test_scheduler.py::test_multiple_listeners --deselect=distributed/tests/test_scheduler.py::test_no_dangling_asyncio_tasks --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_scheduler.py::test_scheduler_file --deselect=distributed/tests/test_security.py::test_require_encryption --deselect=distributed/tests/test_security.py::test_tls_listen_connect --deselect=distributed/tests/test_security.py::test_tls_temporary_credentials_functional --deselect=distributed/tests/test_semaphore.py::test_threadpoolworkers_pick_correct_ioloop --deselect=distributed/tests/test_tls_functional.py::test_security_dict_input_no_security --deselect=distributed/tests/test_utils_test.py::test_ensure_no_new_clients --deselect=distributed/tests/test_utils_test.py::test_freeze_batched_send --deselect=distributed/tests/test_utils_test.py::test_locked_comm_drop_in_replacement --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_read --deselect=distributed/tests/test_utils_test.py::test_locked_comm_intercept_write --deselect=distributed/tests/test_worker.py::test_host_uses_scheduler_protocol --deselect=distributed/tests/test_worker.py::test_plugin_exception --deselect=distributed/tests/test_worker.py::test_plugin_internal_exception --deselect=distributed/tests/test_worker.py::test_plugin_multiple_exceptions --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker.py::test_scheduler_file --deselect=distributed/tests/test_worker_client.py::test_dont_override_default_get --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_allowlist --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_scheduler.py::test_dashboard_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_defaults --deselect=distributed/cli/tests/test_dask_scheduler.py::test_hostport --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_protocols --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers --deselect=distributed/cli/tests/test_dask_scheduler.py::test_multiple_workers_2 --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_command_default --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_config --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_file --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_preload_remote_module --deselect=distributed/cli/tests/test_dask_scheduler.py::test_scheduler_port_zero --deselect=distributed/cli/tests/test_dask_scheduler.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_contact_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_dashboard_non_standard_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_error_during_startup --deselect=distributed/cli/tests/test_dask_worker.py::test_integer_names --deselect=distributed/cli/tests/test_dask_worker.py::test_listen_address_ipv6 --deselect=distributed/cli/tests/test_dask_worker.py::test_local_directory --deselect=distributed/cli/tests/test_dask_worker.py::test_memory_limit --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_port_range_too_many_workers_raises --deselect=distributed/cli/tests/test_dask_worker.py::test_nanny_worker_ports --deselect=distributed/cli/tests/test_dask_worker.py::test_no_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_auto --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_expands_name --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_negative --deselect=distributed/cli/tests/test_dask_worker.py::test_nworkers_requires_nanny --deselect=distributed/cli/tests/test_dask_worker.py::test_preload_config --deselect=distributed/cli/tests/test_dask_worker.py::test_resources --deselect=distributed/cli/tests/test_dask_worker.py::test_respect_host_listen_address --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_address_env --deselect=distributed/cli/tests/test_dask_worker.py::test_scheduler_file --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_restart_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_set_lifetime_stagger_via_env_var --deselect=distributed/cli/tests/test_dask_worker.py::test_signal_handling --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_deprecated --deselect=distributed/cli/tests/test_dask_worker.py::test_single_executable_works --deselect=distributed/cli/tests/test_dask_worker.py::test_timeout --deselect=distributed/cli/tests/test_dask_worker.py::test_worker_class --deselect=distributed/tests/test_config.py::test_logging_extended --deselect=distributed/tests/test_config.py::test_logging_file_config --deselect=distributed/tests/test_config.py::test_logging_mutual_exclusive --deselect=distributed/tests/test_config.py::test_logging_simple --deselect=distributed/tests/test_config.py::test_logging_simple_under_distributed --deselect=distributed/tests/test_queues.py::test_queue_in_task --deselect=distributed/tests/test_variable.py::test_variable_in_task ============================= test session starts ============================== platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0 -- /usr/bin/python3.12 cachedir: .pytest_cache rootdir: /build/reproducible-path/dask.distributed-2024.5.2+ds.1/.pybuild/cpython3_3.12_distributed/build configfile: pyproject.toml plugins: rerunfailures-12.0, typeguard-4.3.0, timeout-2.3.1, anyio-4.4.0 timeout: 300.0s timeout method: signal timeout func_only: False collecting ... collected 1 item run-last-failure: rerun previous 1 failure (skipped 146 files) distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] PASSED [100%] ============================= slowest 20 durations ============================= 1.90s call distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s setup distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] 0.00s teardown distributed/tests/test_priorities.py::test_last_in_first_out[queue on worker] ============================== 1 passed in 2.72s =============================== *** END OF RUN 4: ALL TESTS RUN HAVE NOW PASSED/XFAILED *** create-stamp debian/debhelper-build-stamp dh_testroot -O--buildsystem=pybuild dh_prep -O--buildsystem=pybuild debian/rules override_dh_auto_install make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_auto_install I: pybuild plugin_pyproject:178: Copying package built for python3.12 to destdir dh_link find debian/ -name *.ico -exec chmod a-x {} \; rm -rf debian/python3-distributed/usr/lib/python3*/dist-packages/dask-worker-space debian/python3-distributed/usr/lib/python3*/dist-packages/core debian/python3-distributed/usr/lib/python3*/dist-packages/test_cluster_dump make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' debian/rules override_dh_install make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_install dh_installchangelogs docs/source/changelog.rst make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' debian/rules override_dh_installdocs make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_installdocs -ppython-distributed-doc --doc-main-package=python3-distributed dh_installdocs --remaining-packages rm -vf debian/python-distributed-doc/usr/share/doc/python3-distributed/html/_static/vendor/fontawesome/*/LICENSE.txt removed 'debian/python-distributed-doc/usr/share/doc/python3-distributed/html/_static/vendor/fontawesome/5.13.0/LICENSE.txt' make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_sphinxdoc -O--buildsystem=pybuild dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/python-distributed-doc/usr/share/doc/python3-distributed/html/_static/js/index.be7d3bbb2ef33a8344ce.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/python-distributed-doc/usr/share/doc/python3-distributed/html/_static/sphinx-book-theme.12a9622fbb08dcb3a2a40b2c02b83a57.js dh_sphinxdoc: warning: ignoring unknown JavaScript code: debian/python-distributed-doc/usr/share/doc/python3-distributed/html/_static/design-tabs.js dh_installchangelogs -O--buildsystem=pybuild dh_python3 -O--buildsystem=pybuild dh_installsystemduser -O--buildsystem=pybuild dh_perl -O--buildsystem=pybuild dh_link -O--buildsystem=pybuild dh_strip_nondeterminism -O--buildsystem=pybuild debian/rules override_dh_compress make[1]: Entering directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_compress -Xchangelog.html make[1]: Leaving directory '/build/reproducible-path/dask.distributed-2024.5.2+ds.1' dh_fixperms -O--buildsystem=pybuild dh_missing -O--buildsystem=pybuild dh_installdeb -O--buildsystem=pybuild dh_gencontrol -O--buildsystem=pybuild dh_md5sums -O--buildsystem=pybuild dh_builddeb -O--buildsystem=pybuild dpkg-deb: building package 'python3-distributed' in '../python3-distributed_2024.5.2+ds.1-7_all.deb'. dpkg-deb: building package 'python-distributed-doc' in '../python-distributed-doc_2024.5.2+ds.1-7_all.deb'. dpkg-genbuildinfo --build=binary -O../dask.distributed_2024.5.2+ds.1-7_amd64.buildinfo dpkg-genchanges --build=binary -O../dask.distributed_2024.5.2+ds.1-7_amd64.changes dpkg-genchanges: info: binary-only upload (no source code included) dpkg-source --after-build . dpkg-source: info: using options from dask.distributed-2024.5.2+ds.1/debian/source/options: --extend-diff-ignore=^[^/]+\.egg-info/ dpkg-buildpackage: info: binary-only upload (no source included) dpkg-genchanges: info: not including original source code in upload I: copying local configuration I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/B01_cleanup starting I: user script /srv/workspace/pbuilder/1538210/tmp/hooks/B01_cleanup finished I: unmounting dev/ptmx filesystem I: unmounting dev/pts filesystem I: unmounting dev/shm filesystem I: unmounting proc filesystem I: unmounting sys filesystem I: cleaning the build env I: removing directory /srv/workspace/pbuilder/1538210 and its subdirectories I: Current time: Tue Oct 22 07:49:32 +14 2024 I: pbuilder-time-stamp: 1729532972